docs(spec): make invariants normative and tuned constants advisory

[JarbasAl]

🤖 Auto-generated by Claude Sonnet 5 (claude-sonnet-5), amended by Claude Fable 5.1 (claude-fable-5-1) via Claude Code — NOT human-reviewed. Verify before acting.

This closes six design-review findings against docs/SPECIFICATION.md, all worded prose only, no behavior changes. The first cut of this rewrite introduced an imprecision of its own — it described the label_set coverage gate as a per-league floor documented in docs/leagues.md, which does not exist — and this amendment corrects it.

R5b named the exact tuned constants (MAX_AUTO_WEIGHT_PER_PAIR, the 0.25 auto-vote weight) as normative text. The invariant — a symmetric per-pair cap, proportional scaling that preserves the observed win rate, dataset size never a lever, deterministic replay — is what R5b actually requires; the tuned numbers now live where they're tuned, arena/assembler.py and docs/methodology.md, and the spec points at them instead of restating them.

R1 said a battle pairs two predictions for the same sample and dataset, but didn't say what happens once a dataset publishes a sample-set manifest (sample_sets/<lang>.json, docs/runner.md). It now states that both predictions must fall inside the managed set, matching what arena.predictions.group_rows already does.

R12 read as fetching every vote issue and replaying "from scratch" on every run, which reads as contradicting §6's append-only committed vote record. It's restated as: fetch to detect anything not yet recorded, append it once, then rebuild every leaderboard by replaying the full committed record — which is the actual behavior and matches what assemble/tally/verify-replay already do.

§3.1 left two things unstated: whether trained_on covers a corpus republished under a different dataset id (it doesn't — the registrant has to list every id), and what the runner does with a pretrained fighter's declared label_set. The spec now states the gate as the code implements it: runner/intent_bench.py intersects the loaded model's class list with the corpus labels and refuses to write rows when the overlap is zero; any overlap scores, and each scored row records the overlap measured. The check is global and binary, identical in every league.

The dataset lang field description now states that a bare ISO 639-3 primary subtag is accepted only from registry/schemas.py's closed allow-list (BARE_PRIMARY_SUBTAGS), and that adding a language to it is an owner ruling recorded in the schema file, matching the code comment already there.

Only one of the six findings in the brief — restating the runner's process model (fork/child process/watchdog) as behavior rather than mechanism — turned out not to apply: docs/SPECIFICATION.md §3/§4/§7 don't name fork, child process or watchdog as a requirement anywhere (the process-model detail lives entirely in docs/runner.md, which is unchanged). I left that one alone rather than inventing text to rewrite.

Known separate defect, not fixed here

§7 item 4 describes the TTS modality as human-vote only — no objective metric, no benchmark board, no ELO seed, the ELO board accruing purely from blind A/B listening votes. That contradicts §4 R14 in the same document, where runner/tts_bench.py MUST score every clip with UTMOS, arena/metrics.py score_tts aggregates it into a benchmark board, and arena/assembler.py seed_elo seeds ELO from it. The code agrees with R14, not with §7. It is pre-existing and outside this diff; it needs its own PR.

Verification

• runner/intent_bench.py lines 718-726 on origin/dev: label_overlap = model_label_overlap(...); if label_overlap == 0: log and write no rows. No threshold, no per-league parameter. docs/leagues.md lines 66-73 describes the same binary rule and no threshold table.
• MAX_AUTO_WEIGHT_PER_PAIR = 5.0 (arena/assembler.py:61) and BARE_PRIMARY_SUBTAGS = frozenset({"arb", "kab"}) (registry/schemas.py:52) checked against the current source and match what the spec says about them.
• tests/test_spec_coverage.py: 3 passed (throwaway venv, editable install of this branch).
• ste_lint.py docs/SPECIFICATION.md — PASS.
• The style grep (as of|currently|now|recently|...) returns the same 5 hits as origin/dev, no new ones introduced.
• pr_hygiene_lint.py --base origin/dev reports a clean single-commit diff touching only docs/SPECIFICATION.md.

Docs only — no code touched.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

I've completed the automated review of your changes. 📑

I've aggregated the results of the automated checks for this PR below.

🔎 Type Check

Here's the lowdown on the latest automated check. 📉

✅ mypy: no type errors found

🔍 Lint

Here's the latest update on this check. 🗞️

❌ ruff: issues found — see job log

🔨 Build Tests

Testing the load-bearing capacity of your changes. 🏋️

✅ All versions pass

Python	Build	Install	Tests
3.10	✅	✅	✅
3.11	✅	✅	✅
3.12	✅	✅	✅
3.13	✅	✅	✅
3.14	✅	✅	✅

---

Final report submitted. Over and out. 📻