[feat] Score finished sessions in the background #1179

Parent: #1137

Goal

#1178 scores a session when a member asks. Pass rates need a steady sample without anyone remembering to ask. Let a member opt in to having their finished sessions sampled and scored on their machine, within a daily limit, and published with their usage report.

Rules

• The member opts in. scoring.sessions: true in the member's local config turns it on, read from the config of the scope the session's capture belongs to (~/.teamai/config.yaml for user scope). It is off by default, so no tokens are spent without the member, as auto_start is off in Discussion: Workflow Runtime Integration #818. scoring.daily_limit caps automatic judge launches per local day for that scope's data home: a non-negative integer, default 10, where 0 launches none.
• The scorer opts in. A scorer applies to sessions only with sessions: true in its file. agents keeps selecting workflow steps by definition, and an ordinary session has no definition. This issue adds the key to the scorer schema [feat] Add workflow scorers and recorded quality labels #1085 defines, default false.
• Candidates are sessions with a capture ([feat] Score a finished session on demand #1178) on this machine, finished because nothing has happened in them for 6 hours. Activity is the latest dashboard event for the session, or the session's updatedAt from AgentSession.list when there is no event.
• Sampling uses [feat] Add workflow scorers and recorded quality labels #1085's rule over the session subject: hash(scorer, subject) mod 10000 < samplingRate × 100. Every machine makes the same choice.
• Each key is tried once. Before launching a judge, write an attempt marker for (scorer, rubric.sourceHash, subject) and take the day's slot, both under the worker's lock. A failed judge or a crash does not make the key a candidate again. A label from teamai session save --scorer ([feat] Score a finished session on demand #1178) also counts as tried, and is still the way to score again by hand. A session resumed after it was scored is not scored again automatically.
• The limit counts every launch, failed ones included, in <dataHome>/scores/daily.json. Candidates over the limit wait for a session start on a later day, oldest first. Nothing runs on a timer.
• When. After the background pull that a session start runs (pullHandler), a detached process picks candidates and runs the judges. The hook never waits for it. Only one such process runs per data home at a time.
• Publishing. Labels go to [feat] Score a finished session on demand #1178's pending outbox and are published by the next pull's usage report, under the same opt-out (isUsageReportEnabled). Pending labels count as work for that report: today it returns early when there are no usage deltas or votes (team-push.ts:1093-1102), which would leave them pending.
• Sessions that TeamAI launched never have a capture ([feat] Score a finished session on demand #1178), so workflow steps and judges are never candidates.

Scope

• The scoring keys in the local config schema (types.ts), next to workflows.
• The sessions key in the scorer schema.
• Candidate selection, sampling, attempt markers, the daily limit and the detached run.
• Publishing pending labels from the usage report path (reportUsageToTeam, team-push.ts).
• docs/guide/ and docs/guide/zh-CN/: a section on background scoring in the scorer guide, with the cost it implies and how to turn it off.

Acceptance criteria

1. With scoring.sessions unset, starting sessions launches no judge and writes no label.
2. With scoring.sessions: true, a scorer with sessions: true and samplingRate: 100, and a captured session last active 7 hours ago, the next session start judges it, and the next pull publishes the label to scores/<member>/<YYYY-MM>.jsonl.
3. A scorer without sessions: true is never used on a session.
4. samplingRate: 0 never scores a session in the background, while teamai session save --scorer still does.
5. With daily_limit: 2 and five candidates, two judges run that day and the rest run on session starts on later days. A failed judge counts toward the limit and is not retried. daily_limit: 0 launches none.
6. A worker killed after taking a slot and before the judge answers does not retry that key, and the slot stays used.
7. The session-start hook returns before the judge finishes.
8. Two session starts at the same time run one scoring process.
9. A pull with no usage deltas and no votes still publishes pending labels.

Test plan

• Unit tests for candidates, sampling, attempt markers, the daily counter and the single-process lock, with a fake clock.
• E2E with fixture captures and a fake judge for criteria 1 to 6 and 9.
• One real-CLI run with Claude for criterion 2, recorded in the PR.

Watch out for

• The daily counter is local and counts attempts, not tokens. Spending caps beyond it are out of scope.
• The detached process must not hold the hook's stdio or the run lock.

Out of scope

Retrying failed judges, scoring other machines' sessions, spending budgets in tokens or money, scheduled scoring without a session start, and scoring subagents.

Dependencies

• Blocked by: [feat] Score a finished session on demand #1178
• Unblocks: none