Add rank_posts for bounded post rankings from email statistics (#101)

[conorbronsdon]

Closes #101.

What changed

• New tool: rank_posts is read-only. It makes one GET to /api/v1/publication/stats/email_stats and ranks posts by one metric, desc or asc.
• Accepted metrics: views, opened, sent, open_rate, click_through_rate, signups, subscribes, estimated_value or post_date. The endpoint silently accepts unknown sort fields, so only fields whose sorting was verified live are allowed.
• Paging: limit is 1–20, which is Substack's cap, plus the usual offset. Results report total, returned, has_more and next_offset. An empty page ends continuation.
• Rows: each has a rank, post_id, title, post_date, type, the ranked value and a value_state (reported, null or absent). It also carries a fixed metrics set and an absent_metrics list.
• Values as reported: rows keep Substack's order. Nothing is re-sorted, filled in or recomputed, and null and absent are never zero.
• Known limits, documented: Substack doesn't document rate denominators or units. Null rates appear among the numeric rows. total may exclude posts that have no email statistics.
• Rejected responses: malformed, duplicate and oversized pages are rejected before any result is returned.
• Unchanged: get_post_analytics.
• Docs: new docs/analytics-rankings.md, plus README and CHANGELOG updates.

Evidence

Read-only probes from 2026-09-14, already posted on #101:

• limit above 20 returns 400, and the page after the last one is empty.
• Every accepted metric sorts non-null values correctly in both directions.
• Absent and null values are distinct.

A live smoke test of this code (GET-only, printing only derived facts) checked views desc, open_rate desc, subscribes asc and post_date desc. Each returned 20 rows with contiguous ranks and sorted reported values. The open_rate page had 13 reported, 6 null and 1 absent. A request past the end returned an empty page with has_more: false.

Checks (at 8e7ec8c)

• Lint and test:package pass. The package test now expects 25 tools, 21 with output schemas.
• Vitest: 973 passed, 1 skipped.
• The contract snapshot diff is additive only: rank_posts, plus the new optional get_post_analytics fields.
• Tests cover:
  • validation before any request
  • null, absent and zero
  • server order and continuation
  • malformed responses
  • MCP routing with one and two publications
  • a 429 surfacing retry_after without retrying
• Mutation checks: disabling any one of these fails a test:
  • the metric allowlist
  • the absent/null distinction
  • the row bound
  • server-order preservation
  • the empty-page has_more rule
  • duplicate rejection

Review

Round 1 on 25d3248 covered Codex, OpenCode MiMo v2.5 Pro and Hermes Inkling.

• MiMo: duplicate ### Added changelog heading. Fixed in 9b04d51.
• Hermes: nothing blocking.
• Codex: 2 BLOCKER, 2 MAJOR and 1 MINOR. How each was handled:

  • Eligibility validation before any statistics read. Not treated as blocking: the maintainer chose to ship rankings without waiting on Publication eligibility, workflow validation and analytics priorities #66. The concrete part is fixed. HTTP 403/404 from statistics now returns code: "analytics_unavailable" instead of a generic upstream error, and the docs say the server does not pre-check tier or eligibility.

  • get_post_analytics scan-bound ambiguity, Add bounded post analytics rankings with explicit coverage and metric semantics #101's original problem. Fixed. Not-found results now report:

    • search_result: "archive_exhausted" when the whole feed was searched, or "scan_bound_reached" when the 500-post bound was hit first
    • scanned
    • feed_capped, the feed's isCapped flag passed through uninterpreted

    Found posts report stats_available. getPostAnalytics keeps its signature.

  • Unknown arguments silently stripped. Fixed. The input schema is strict, so section_id is rejected before any request.

  • Contradictory pages reported as exhausted. Fixed. An empty or short page before total, or rows past total, is rejected. The earlier test that accepted this was replaced.

  • post_date unvalidated (MINOR). Fixed. It must parse as a date and is documented as Substack's string, unchanged.

Each of these fixes fails a test when disabled: strict input (library and MCP schema), mid-list pages, rows past total, date parsing, analytics_unavailable, exhausted versus bound, the isCapped passthrough and stats_available.

Round 2 on 1b556c8:

• Hermes Inkling: nothing blocking. All round-1 fixes were confirmed.

• Codex: confirmed the strict input, page consistency, post_date and analytics_unavailable fixes, and found one BLOCKER. findPostAnalytics ignored the feed's total: a short page claimed every post was searched even when total said more remained, and a feed of exactly 500 posts reported the scan bound. Fixed in bd76149:

  • a short page before the reported total is feed_incomplete
  • ten full pages that reach the total are archive_exhausted
  • with no numeric total, the old behaviour is kept

  Disabling any of these three checks makes a test fail.

• CI: the container smoke test expected exactly 24 tools and failed on every commit of this branch. Fixed in adf1d22.

• Schema regression caught before push: an inline comment in output-contracts.ts had dropped five get_post_analytics output-schema fields. The contract snapshot caught it, and the final snapshot diff is additions only.

Round 3 on bd76149:

• Codex: two BLOCKERs, both about claiming every post was searched when the feed's pages disagreed.

  • Only the last page's total was kept, so a total that changed mid-scan could still produce archive_exhausted.
  • A reported total below the posts returned (total: 0 with ten full pages) counted as proof of exhaustion.

  Fixed in 5db4587. Exhaustion is now claimed only from consistent pages. These cases all return feed_incomplete:

  • a total that changes between pages, or appears on only some of them
  • more posts than the total
  • a post repeated across pages

  Full pages count as exhausted only when they exactly reach a total reported identically on every page.

• Mutation checks: each of the three new guards fails a test when disabled.

• Live check: a read-only run against the publication still reports archive_exhausted. Its feed reports a total of 19 and returns 19 posts.

Round 4 on 5db4587:

• Hermes Inkling: nothing blocking. Its low-severity note is intended and documented behaviour: when Substack omits total on every page, a short final page still ends the feed, as before this PR.
• Codex: one BLOCKER. A balanced change between reads keeps the total constant and introduces no duplicate ID, but still shifts a post past an offset boundary. Example: one post deleted and another published. No pagination check can detect this. Rather than overclaim, the latest commit changes the wording. archive_exhausted now means the search reached the end of the feed as paged, not proof that the post never existed. The not-found note, tool description, docs and changelog say pages are separate reads, and that concurrent publishing or deletion can hide a post. This follows the existing local_snapshot and export wording for multi-read tools.

Not in scope

• An operator CLI command.
• Aggregation or segmentation (Analytics and subscriber segmentation with explicit metric contracts #52).
• Rate denominators, which Substack doesn't document.

🤖 Generated with Claude Code

https://claude.ai/code/session_01AcyA1s6dUxtZfc2bMV7fuf