Skip to content

Commit 52a6a72

Browse files
committed
handoff: the resume point is publication now -- gate, article, email, arXiv
Every surface serves the same card, so the next session's work is the writeup as a Hub community article, the email after it, and arXiv beside both. The header states the 2026-09-05 state (lock, D074 review, D075 review pass feeding no published number, the D074 collision and how to avoid the next one), the gate lists the four digests that must agree plus the deploy-vs-commit check the digest cannot see, and each step names what a session may do and where it must stop. The Wave 2 'next action' section is retitled as historical so nobody resumes from it.
1 parent f5ef78a commit 52a6a72

1 file changed

Lines changed: 121 additions & 2 deletions

File tree

docs/HANDOFF.md

Lines changed: 121 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,126 @@
33
The resume point. A fresh session should be able to continue from this file without
44
re-deriving anything.
55

6-
**Last updated: 2026-09-04 — (D071/D072) H2 is measured and closed.** The raw-corpus adapter
6+
**Last updated: 2026-09-05 — delivery is current on every surface; what is left is
7+
publication.** Main is green (a `constraints.txt` lock pins CI's install; the Space's stats
8+
snapshot no longer fails on a test count or a float ulp). `MEASUREMENT_CARD.json`
9+
carries 18 claims and one unmet item (Result 2), verdict `NOT_VERIFIED`; the Hub dataset,
10+
the Hub model and the Space all serve that exact card (digest `3aacbe46…`), and the dataset's
11+
`results/` now ships both `h2_design_effect` files the H2 claim cites. D074 is the independent
12+
review of D071/D072 `WAVES.md` required: the verdict stands, every number reproduced from the
13+
response records, and the pre-registration turned out to state H2's threshold two ways (design
14+
effect ≥ 2, and "twice the width", which is 4 under its own definition; the result clears
15+
neither). D075 opened a third, salted, blind labelling pass over the 18 rater/judge
16+
disagreements plus an equal control arm; it is a separate `review` pass and **feeds no published
17+
number** — folding it needs its own entry with before and after. Two sessions wrote a D074
18+
within minutes of each other on 2026-09-05; the later one is now D075. That will recur while two
19+
sessions share one append-only file: check `grep -c '^## D0' docs/DECISIONS.md` against the
20+
last number before appending.
21+
22+
**Still open, in order:** publication (the section below); the D075 fold decision, once the
23+
review pass is complete and `scripts/review_labels_cli.py report` says whether the controls
24+
moved; a second rater on `G200-ego` (`RED-TEAM.md` A1, still the first weakness on the front
25+
page, ~35 minutes of someone else's time); `judge_weights_rev` recorded into the result JSON on
26+
the next real judge run (D074 finding 4); `REVIEW.md` R2, the pseudo-cluster proxy on Build AI's
27+
own frames, now calibratable against real `worker_id`s (`RED-TEAM.md` A13); Result 2, blocked
28+
on two reasons the adapter did not touch.
29+
30+
## Next session: publication
31+
32+
Three steps. One is the critical path, one follows it, one runs beside it. `docs/private/`
33+
holds the outreach sequence and the email draft; it is gitignored and is never quoted here or in
34+
anything public (`AGENTS.md` rule 5).
35+
36+
### Gate, before anything goes public
37+
38+
Run all of it; publish only if every line is clean.
39+
40+
```
41+
git status --short && git rev-list --left-right --count origin/main...main # clean, 0 0
42+
gh run list --limit 1 # ✓ on main
43+
make validate # tests, mypy, lint
44+
make card && git diff --exit-code MEASUREMENT_CARD.json # committed card == fresh
45+
python3 -c "import json;print(json.load(open('MEASUREMENT_CARD.json'))['content_digest'])"
46+
python3 -c "import json;print(json.load(open('space/public/data/stats.json'))['generated_from']['card_digest'])"
47+
curl -sL https://caiotheodoro-vernier.static.hf.space/data/stats.json | python3 -c "import sys,json;print(json.load(sys.stdin)['generated_from']['card_digest'])"
48+
curl -sL https://huggingface.co/datasets/caiotheodoro/vernier/raw/main/MEASUREMENT_CARD.json | python3 -c "import sys,json;print(json.load(sys.stdin)['content_digest'])"
49+
```
50+
51+
The four digests must be identical. The digest does not see UI-only changes, so also compare
52+
`git log -1 --format='%h %ci' -- space/` with the Space repo's `lastModified`
53+
(`curl -s https://huggingface.co/api/spaces/caiotheodoro/vernier`); as of this header the last
54+
Space commit (f5ef78a, 23:31 local) is four minutes newer than the last upload. If either is
55+
behind, `make space-data && make space`
56+
then `hf upload caiotheodoro/vernier space/dist . --type space`; if the dataset's is behind,
57+
`make hf-dataset` then `hf upload caiotheodoro/vernier hf/dataset . --type dataset` (both need
58+
`HF_TOKEN` from `.env`; `hf` is the 1.30 CLI at `~/.local/bin/hf`). If the other session has
59+
folded D075 into the primary labels in the meantime, every number downstream moves: run
60+
`make agreement`, `make distil`, `make card`, `make space-data`, republish, and only then
61+
continue — a writeup that cites numbers a later commit restates is the exact failure this
62+
project audits in its subject.
63+
64+
Then the writeup's own rule: every figure in `docs/WRITEUP.md` is copied from the card or a
65+
`data/*.json` file, and the card wins on disagreement. Check the ones that have moved most
66+
recently by hand: the H2 table (1.25 / 1.62 / 1.27 and 1.31 / 1.66 / 1.29), the 2,144-vs-2,153
67+
worker count, the decision range (D001–D075), and the two cost figures ($17.63 from the two E2
68+
result files; ~$7.50 from D072, the one number in the writeup that cites a decision rather than a
69+
data file, because no result file carries it).
70+
71+
### 1. The writeup, as a Hugging Face community article — critical path
72+
73+
There is no CLI for this; it is the web editor at https://huggingface.co/new-blog, published
74+
under `caiotheodoro`. Body is `docs/WRITEUP.md` with two edits: drop the italic *"Draft for the
75+
Hugging Face community blog"* paragraph at the top (keep the sentence about where the numbers
76+
come from, reworded as a statement, not a draft note), and turn the bare repository paths
77+
(`docs/PRE-REGISTRATION.md`, `data/e2_full_n10000.json#H1`, …) into links to
78+
https://github.com/caiotheodoro/vernier at `main`, since on the Hub they resolve to nothing. The
79+
H1 title is the article title. Tags: `egocentric`, `dataset-audit`, `llm-as-judge`,
80+
`evaluation`, `build-ai`. Link the collection
81+
(https://huggingface.co/collections/caiotheodoro/vernier-same-judge-same-6pp-on-2-hands-both-releases-6a9b1bc9fac182a7a2d9c997)
82+
in the "Where everything is" list; if the editor offers to attach datasets and models, attach
83+
`caiotheodoro/vernier`, `caiotheodoro/vernier-rung1-probe`, and the two `builddotai/*-Evaluation`
84+
datasets so the article surfaces on their pages, which is the whole point of publishing on the
85+
Hub rather than anywhere else.
86+
87+
After it is live: a `DECISIONS.md` entry (D076 unless the other session has taken it) recording
88+
the URL and the date; the URL added to `README.md`'s map, `llms.txt`, the dataset card
89+
(`scripts/export_hf_dataset.py`, then `make hf-dataset` and re-upload), and the collection.
90+
Nothing in the article is edited after that without a dated note in the same entry — it is a
91+
published result now, and the project's own standard for Build AI's card applies to it.
92+
93+
### 3. The email — after 1, never before
94+
95+
`docs/private/OUTREACH.md` is the sequence and the reasons; `docs/private/EMAIL-DRAFT.md` is the
96+
text. Five lines, link first. What a session can do: check every link in the draft resolves,
97+
check every figure in it against the card, confirm the article is live, and stop. Sending is the
98+
user's act, from the user's account, and is not automated. Two things the draft must reflect as
99+
of now: the article URL (it is the first link), and `docs/ETHICS.md` §4 as rewritten in D073 —
100+
24 downscaled stills are republished in the Space and come down on request; that is the one
101+
sentence in the email that is a courtesy to them rather than a result. D009's exception (private
102+
disclosure first, only for a live privacy or security exposure) has not been triggered; nothing
103+
found in this project is one. Record the send as a one-line `DECISIONS.md` entry with the date
104+
and nothing else — the content stays private.
105+
106+
### 2. arXiv — beside the other two, gating nothing
107+
108+
cs.CV, single author, no affiliation ("Independent"). An unaffiliated first-time submitter needs
109+
an endorser; ask with the article already public, since the ask is then "read this" rather than
110+
"trust me" (`docs/private/OUTREACH.md` says why in more words). Do not name a candidate in any
111+
public file. Build the submission from the writeup, not from a rewrite:
112+
113+
```
114+
pandoc -s docs/WRITEUP.md -o arxiv/vernier.tex # TeX source, which arXiv requires
115+
# for anything TeX-produced
116+
```
117+
118+
Then a title, an abstract (the first two paragraphs of the writeup, tightened), CC BY 4.0, and
119+
cross-list `stat.AP` if it is offered. The arXiv version cites the article URL and the Git
120+
commit it was built from. Submit after 1 is live so v1 links to it; submit before 3 only if the
121+
endorsement arrives first — the email should carry the arXiv link if one exists, and does not
122+
wait for it.
123+
124+
125+
**Prior header — (D071/D072) H2 is measured and closed.** The raw-corpus adapter
7126
D065 declined to size turned out to be small: ffmpeg's `subfile` protocol extracts one frame
8127
from an h265 mp4 inside a tar over HTTP range requests, so no new dependency was needed, and a
9128
tar header is 512 bytes at a computable offset, so `scripts/build_corpus_manifest.py` indexed
@@ -252,7 +371,7 @@ A live E2 or E5 run now also leaves per-frame records behind, next to its checkp
252371
per-file negation. Forward-only -- the committed aggregates predate this and have no jsonl.
253372
`docs/DECISIONS.md` D069.
254373

255-
## The next action
374+
## The next action, as it stood in Wave 2 (historical; the current one is "Next session: publication" above)
256375

257376
**The Qwen3-VL judge is deployed and fully smoke-tested live end to end, image + logprobs
258377
included — real correct output.** `cloud/modal_qwen3vl.py` is live on Modal

0 commit comments

Comments
 (0)