v0.9.0: structured content, temporal tool, hermetic tests, MCP Registry

- search/fetch and the stats family declare outputSchema and return
  structuredContent mirrored into the text block (ChatGPT apps/deep
  research contract); row-heavy search_*/OCR tools deliberately stay
  text-only to avoid doubling their payloads
- new get_temporal_distribution: counts per year/month across the dated
  subsets with the standard filters, group_by country|newspaper, and an
  undated_count so nothing is dropped silently
- consolidate: TEXT_COLS single source for per-subset keyword columns,
  shared semantic-search runner (_semantic.ts), OCR-excerpt helper,
  limitWarning; every tool now carries a top-level title
- harden: prune stale parquet shards and .partial files on cache
  refresh, IWAC_OFFLINE mode, timing-safe bearer compare, env int
  guards, instructions built per-server so semantic tools are only
  mentioned when actually registered
- tests: unit suite (tsx + node:test) and an offline fixture MCP
  round-trip against synthetic parquet (make-fixtures.mjs); smoke test
  gains an EXPECTED pin block, manifest-sync and structuredContent
  assertions; new ci.yml runs the hermetic suite on push and the live
  smoke weekly
- publish to the official MCP Registry on version tags via
  mcp-publisher GitHub OIDC: generated server.json (schema 2025-12-11)
  with mcpb fileSha256 entries and the public islam.zmo.de/mcp remote;
  Dockerfile carries the OCI ownership label

Author: fmadore

.agents/skills/iwac-mcp/SKILL.md

@@ -14,13 +14,13 @@ description: |
 
 # IWAC MCP Research Workflow
 
-Structured methodology for academic research using the IWAC MCP server's 26 possible tools (24 core + 2 optional semantic). Adapted from ALA-compliant archival research practices. Applies to server **v0.8.0+** — all matching is accent- and case-insensitive; result objects use short English keys (`id`, `date`, `polarity`, `centrality`, `subjectivity`, `description_ai`, `url`); list/search tools return a pagination envelope (`count`, `total_matches`, `offset`, `limit`, `has_more`, `next_offset`); and enumerated filters are validated (see **Reading Results & Errors** below).
+Structured methodology for academic research using the IWAC MCP server's 27 possible tools (25 core + 2 optional semantic). Adapted from ALA-compliant archival research practices. Applies to server **v0.8.0+** — all matching is accent- and case-insensitive; result objects use short English keys (`id`, `date`, `polarity`, `centrality`, `subjectivity`, `description_ai`, `url`); list/search tools return a pagination envelope (`count`, `total_matches`, `offset`, `limit`, `has_more`, `next_offset`); and enumerated filters are validated (see **Reading Results & Errors** below).
 
 ## Prerequisites
 
 Load reference files **as needed**, not all upfront:
 
-1. **references/tools-by-phase.md** — all 26 possible tools with parameters, defaults, and verified filter vocabularies. Read before the first search of a session.
+1. **references/tools-by-phase.md** — all 27 possible tools with parameters, defaults, and verified filter vocabularies. Read before the first search of a session.
 2. **references/research-domains.md** — French search terms and transliteration variants by domain. Read when crafting search-term variants (Extended mode, or when a Brief search comes back thin).
 3. **references/biases-and-limitations.md** — collection biases, coverage gaps, sentiment caveats. Read before writing the synthesis.
 4. **references/capabilities-overview.md** — plain-language description of the collection and recommended ways into the data. Read when the user asks what you can do (see "Capability Questions" below).
@@ -29,7 +29,7 @@ For data schema and Omeka S API details, defer to the `iwac-data` skill.
 
 ## Capability Questions
 
-When the user asks what you can do with IWAC ("what can you do?", "qu'est-ce que tu peux faire ?", "what's in this collection?", "how could I search this?"), do **not** launch the research workflow, present the depth choice, or enumerate the 26 tools. Read **references/capabilities-overview.md** and answer in plain language, in the user's language:
+When the user asks what you can do with IWAC ("what can you do?", "qu'est-ce que tu peux faire ?", "what's in this collection?", "how could I search this?"), do **not** launch the research workflow, present the depth choice, or enumerate the 27 tools. Read **references/capabilities-overview.md** and answer in plain language, in the user's language:
 
 1. One short paragraph on what the collection is and covers.
 2. The main ways into the data (keyword, curated themes, people/organizations, semantic, sentiment, periodicals, scholarship) — described as research moves, not tool names.

.agents/skills/iwac-mcp/references/capabilities-overview.md

@@ -41,6 +41,7 @@ Because reports are date-stamped and location-specific, the collection also supp
 - **People, organisations, places** — find the canonical name among the 4,700 index records, then pull every article that mentions it.
 - **Conceptual (semantic) search** in any language, when keywords would miss paraphrases — "Islamic education reform" also surfaces madrasa-modernisation pieces.
 - **Tone and framing** — every article carries AI sentiment labels (polarity, centrality, subjectivity); compare how topics, countries or outlets frame Islam.
+- **Trends over time** — chart how coverage of a topic rose and fell year by year (per country or outlet), directly, without leafing through the articles themselves.
 - **By outlet and period** — filter by newspaper, country, date range; contrast state vs. private press, pre- vs. post-1990.
 - **Inside Islamic periodicals** — browse the 25 series, search tables of contents (complete for 17 smaller magazines), or pull keyword excerpts from a full issue. The same keyword-excerpt reading works on long archival documents and articles too.
 - **Scholarship** — connect press findings to the 864 academic references.

.agents/skills/iwac-mcp/references/tools-by-phase.md

@@ -1,6 +1,6 @@
 # IWAC MCP Tools by Research Phase
 
-26 possible tools (24 core + 2 optional semantic) organized by the workflow phase where they are most useful. Server **v0.8.0+**: **all keyword/filter matching is accent- and case-insensitive**; result rows use short English keys (`id`, `date`, `polarity`, `centrality`, `subjectivity`, `description_ai`, `url`) and omit empty fields. List/search tools return a pagination envelope — `count`, `total_matches`, `offset`, `limit` (applied), `has_more`, `next_offset`, plus `requested_limit` + `limit_warning` when you exceed a tool's max. Enumerated filters (`country`, `polarity`, `centrality`, `index_type`) are **validated**: an invalid value returns `{error, valid_values}` (`isError`) instead of a silent zero-result.
+27 possible tools (25 core + 2 optional semantic) organized by the workflow phase where they are most useful. Server **v0.8.0+**: **all keyword/filter matching is accent- and case-insensitive**; result rows use short English keys (`id`, `date`, `polarity`, `centrality`, `subjectivity`, `description_ai`, `url`) and omit empty fields. List/search tools return a pagination envelope — `count`, `total_matches`, `offset`, `limit` (applied), `has_more`, `next_offset`, plus `requested_limit` + `limit_warning` when you exceed a tool's max. Enumerated filters (`country`, `polarity`, `centrality`, `index_type`, and on the temporal tool `subset`, `granularity`, `group_by`) are **validated**: an invalid value returns `{error, valid_values}` (`isError`) instead of a silent zero-result. Server **v0.9.0+** adds `get_temporal_distribution` (counts per year/month — use it for any "how did coverage evolve" question instead of paging through searches).
 
 ## Cross-Collection Entry Points
 
@@ -172,6 +172,16 @@ Detailed index entry. **Raw dataset columns, French names** (Titre, Titre altern
 
 ## Phase 4: Triangulation Tools
 
+### get_temporal_distribution *(v0.9.0+)*
+Counts of matching items per year (or month) — one call replaces paging through search results for any trend question. Also useful in Phase 1 to scope a topic's timeline before searching.
+- `subset` (optional, validated): articles (default) | publications | references | documents | audiovisual
+- `granularity` (optional, validated): year (default) | month — items dated only to a year keep a bare-year key even at month granularity
+- `keyword` (optional): ONE substring over the subset's text fields (same semantics as the subset's search tool)
+- `country` / `newspaper` / `subject` / `date_from` / `date_to` (optional): same semantics as the subset's search tool
+- `group_by` (optional, validated): country | newspaper — returns `distribution_by_group` (one map per group) instead of `distribution`
+- Returns `total_matches`, `dated_count`, `undated_count` (undated items are counted, never silently dropped), and the `distribution` map sorted by year
+- **Tip:** `get_temporal_distribution(keyword="hadj", group_by="country")` charts six decades of hajj coverage per country in a single ~1k-token call.
+
 ### get_sentiment_distribution
 Aggregated Gemini sentiment counts.
 - `country` (optional, exact name), `newspaper` (optional), `subject` (optional)
@@ -213,5 +223,6 @@ See `search_references` above (12 values, with counts).
 - Triage with `with_description=true` (limit ≤ 10) instead of calling `get_article` on everything; read full OCR only for the 2-3 finalists (Brief) / 6-8 (Extended)
 - A `search_articles` page of 20 ≈ 2.5k tokens; `get_article` ≈ 1-7k tokens; capped `get_publication_fulltext` ≤ ~7k tokens (+ ~1.6k when the issue has a TOC)
 - Use stats/distribution tools for overviews before fetching individual items; when `total_matches` exceeds ~50, analyze metadata rather than reading items
+- For "how did coverage evolve" questions, one `get_temporal_distribution` call (~1k tokens) replaces paging through result envelopes year by year
 - Combine filters (country + subject/keyword + date range) to narrow before reading
 - For temporal filtering: articles take `YYYY-MM-DD` or `YYYY`; publications/references take years

.github/workflows/ci.yml

@@ -0,0 +1,84 @@
+name: CI
+
+# Two layers:
+#  - test: hermetic, runs on every push/PR — typecheck, build, unit tests, and
+#    the offline fixture MCP round-trip (synthetic parquet, IWAC_OFFLINE=1).
+#  - live-smoke: weekly (and on demand) — the full smoke test against the real
+#    Hugging Face dataset. Its pinned counts (smoke-test.mjs EXPECTED block) are
+#    the dataset-drift alarm, so a red weekly run usually means the dataset
+#    changed, not the code.
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+  schedule:
+    - cron: "17 5 * * 1" # Mondays 05:17 UTC — weekly live smoke
+  workflow_dispatch:
+
+jobs:
+  test:
+    if: github.event_name == 'push' || github.event_name == 'pull_request'
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: mcpb
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v7
+
+      - name: Set up Node
+        uses: actions/setup-node@v6
+        with:
+          node-version: 24
+          cache: npm
+          cache-dependency-path: mcpb/package-lock.json
+
+      - name: Install dependencies
+        run: npm ci
+
+      - name: Typecheck
+        run: npm run typecheck
+
+      - name: Build server bundle
+        run: npm run build
+
+      - name: Unit tests
+        run: npm run test:unit
+
+      - name: Fixture MCP round-trip (offline)
+        run: npm run test:fixture
+
+  live-smoke:
+    if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: mcpb
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v7
+
+      - name: Set up Node
+        uses: actions/setup-node@v6
+        with:
+          node-version: 24
+          cache: npm
+          cache-dependency-path: mcpb/package-lock.json
+
+      - name: Restore HF parquet cache (~250 MB)
+        uses: actions/cache@v6
+        with:
+          path: ~/.iwac-mcp/cache
+          key: iwac-hf-parquet-${{ github.run_id }}
+          restore-keys: |
+            iwac-hf-parquet-
+
+      - name: Install dependencies
+        run: npm ci
+
+      - name: Build server bundle
+        run: npm run build
+
+      - name: Live smoke test (real dataset)
+        run: node smoke-test.mjs

.github/workflows/docker-publish.yml

@@ -6,7 +6,8 @@ name: Publish container image to GHCR
 #
 # Triggers on a version tag (v*), or manually. Version tags also publish the
 # Claude Desktop / stdio .mcpb bundles and the .agents research skill zip as
-# GitHub release assets.
+# GitHub release assets, then publish the server to the official MCP Registry
+# (registry.modelcontextprotocol.io) as io.github.fmadore/iwac-mcp-server.
 
 on:
   push:
@@ -24,6 +25,7 @@ jobs:
     permissions:
       contents: write
       packages: write
+      id-token: write # MCP Registry publish via GitHub OIDC (no secret needed)
     steps:
       - name: Checkout
         uses: actions/checkout@v4
@@ -79,3 +81,26 @@ jobs:
             mcpb/iwac-mcp-server-macos.mcpb \
             iwac-mcp-skill.zip \
             --clobber
+
+      # MCP Registry publish. server.json is generated HERE, in the same job
+      # that packed and uploaded the .mcpb assets, so the embedded fileSha256
+      # hashes always match the exact uploaded bytes. Registry versions are
+      # immutable: re-running this workflow for an already-published tag fails
+      # at this step by design (re-packed zips hash differently) — bump the
+      # version instead of re-publishing.
+      - name: Generate server.json
+        if: startsWith(github.ref, 'refs/tags/v')
+        env:
+          TAG: ${{ github.ref_name }}
+        run: node mcpb/scripts/make-server-json.mjs "$TAG"
+
+      - name: Install mcp-publisher
+        if: startsWith(github.ref, 'refs/tags/v')
+        run: |
+          curl -L "https://github.com/modelcontextprotocol/registry/releases/latest/download/mcp-publisher_$(uname -s | tr '[:upper:]' '[:lower:]')_$(uname -m | sed 's/x86_64/amd64/;s/aarch64/arm64/').tar.gz" | tar xz mcp-publisher
+
+      - name: Publish to MCP Registry
+        if: startsWith(github.ref, 'refs/tags/v')
+        run: |
+          ./mcp-publisher login github-oidc
+          ./mcp-publisher publish

.gitignore

@@ -9,6 +9,12 @@ mcpb/server/
 mcpb/*.mcpb
 iwac-mcp-skill.zip
 .duckdb-binding-stash/
+# Generated per-release by mcpb/scripts/make-server-json.mjs (embeds artifact
+# hashes) — regenerate, never hand-edit or commit.
+/server.json
+
+# Generated synthetic test fixtures (scripts/make-fixtures.mjs)
+mcpb/test/fixtures/
 
 # Local cache / secrets
 .cache/

README.md

@@ -68,18 +68,20 @@ Download `iwac-mcp-skill.zip` from the same release, then:
 
 ## What it gives Claude
 
-26 possible read-only tools across six IWAC subsets. **24 work out of the box**;
+27 possible read-only tools across six IWAC subsets. **25 work out of the box**;
 the 2 `semantic_search_*` tools are optional and require a free Google/Gemini API
 key (disabled by default). All keyword and filter matching is accent- and
-case-insensitive.
+case-insensitive. The unified `search`/`fetch` pair and the stats tools also
+return MCP structured content (`outputSchema` + `structuredContent`), which the
+ChatGPT connector contract requires.
 
 | Group        | Tools                                                                                       |
 | ------------ | ------------------------------------------------------------------------------------------- |
 | Cross-subset | `search`, `fetch`                                                                           |
 | Articles     | `search_articles`, `get_article`, `semantic_search_articles`                                |
 | Sentiment    | `search_by_sentiment`, `get_sentiment_distribution`                                         |
 | Index        | `search_index`, `get_index_entry`, `list_subjects`, `list_locations`, `list_persons`        |
-| Stats        | `get_collection_stats`, `get_newspaper_stats`, `get_country_comparison`                     |
+| Stats        | `get_collection_stats`, `get_newspaper_stats`, `get_country_comparison`, `get_temporal_distribution` |
 | Publications | `search_publications`, `list_periodicals`, `get_publication_fulltext`, `semantic_search_publications` |
 | References   | `search_references`, `get_reference`                                                        |
 | Other        | `search_documents`, `get_document`, `search_audiovisual`, `list_audiovisual`, `get_audiovisual` |
@@ -115,9 +117,14 @@ npm install
 node scripts/install-duckdb-bindings.mjs
 npm run typecheck   # tsc --noEmit
 npm run build       # esbuild -> single server/index.js
-node smoke-test.mjs
+npm test            # unit tests + offline fixture MCP round-trip (no network)
+npm run test:live   # full smoke test against the real HF dataset (~250 MB)
 ```
 
+CI runs the typecheck, build, unit tests, and the offline fixture test on every
+push; the live smoke test runs weekly (its pinned counts are the dataset-drift
+alarm).
+
 ## Roadmap
 
 See [TODO.md](TODO.md) — near-term: submit to the Anthropic extension directory,

TODO.md

@@ -9,6 +9,13 @@
 
 ## Distribution & Roadmap
 
+- [x] **Publish to the official MCP Registry** — automated since v0.9.0: the
+  tag workflow generates `server.json` (`mcpb/scripts/make-server-json.mjs`)
+  and publishes `io.github.fmadore/iwac-mcp-server` (2 `.mcpb` packages + the
+  `islam.zmo.de/mcp` remote) via `mcp-publisher` GitHub OIDC. Versions are
+  immutable — fixing an entry means bumping and re-tagging. The Anthropic
+  directory below is a separate, manual submission.
+
 - [ ] **Submit to the Anthropic extension directory**
   Fill out the interest form:
   <https://docs.google.com/forms/d/e/1FAIpQLScHtjkiCNjpqnWtFLIQStChXlvVcvX8NPXkMfjtYPDPymgang/viewform>
@@ -39,6 +46,16 @@
 - [ ] **Add `screenshots/`** showing a research query in Claude Desktop — the
   directory listing surfaces these.
 
+- [ ] **Migrate to MCP TypeScript SDK v2** (after its stable release alongside
+  the 2026-07-28 spec — verified July 2026: `2.0.0-beta.2` current, v1.x is
+  maintenance-only). Breaking for this server: scoped packages
+  (`@modelcontextprotocol/server`), raw Zod shapes replaced by Standard Schema
+  objects (`z.object(...)`), `serveStdio()` / `createMcpHandler()` replacing the
+  v1 stdio + StreamableHTTP transports (stateless core). Start with the official
+  codemod (`npx @modelcontextprotocol/codemod@beta v1-to-v2 .`), then re-run the
+  fixture + live test suites. Re-check the final 2026-07-28 changelog first —
+  details were RC-stage when noted.
+
 ## Data Enrichment (Track 2 — runs in the IWAC-Hugging-Face pipeline, not here)
 
 > Governing rule: all AI enrichment is precomputed offline as HF columns and

docs/connecting.md

@@ -64,9 +64,9 @@ Download `iwac-mcp-skill.zip` from the same release, then in Claude Desktop open
 
 ### Optional: semantic search
 
-Twenty tools work out of the box. Two extra tools find articles and publications
-by *meaning* rather than keywords; they need a free Google/Gemini API key and are
-**off by default**. To enable them, turn on **Enable semantic search** in the
+Twenty-five tools work out of the box. Two extra tools find articles and
+publications by *meaning* rather than keywords; they need a free Google/Gemini
+API key and are **off by default**. To enable them, turn on **Enable semantic search** in the
 extension's settings and paste a key from
 [Google AI Studio](https://aistudio.google.com/apikey). Most users don't need this.
 

mcpb/.mcpbignore

@@ -5,10 +5,11 @@ tsconfig.json
 *.ts
 *.map
 
-# Local test scripts
+# Local test scripts & fixtures
 smoke-test.mjs
 probe-cols.mjs
 download-all.mjs
+test/
 
 # Built bundles (never pack a bundle inside another bundle)
 *.mcpb