Skip to content

Commit 6e4cc2b

Browse files
committed
Take evidence recording off the editor reply path
Two in three spoken replies still waited on a tool round, and nearly every one was the interviewer calling record_framework_evidence for Repeat, Example or Algorithm: steps the prompt told it to leave alone and the phase judge records anyway. Each call holds the reply back by a second model pass. Neither the prompt nor leaving those steps out of the tool's schema stopped the calls, so at a typed editor the tool is no longer declared. The prompt says the platform records each step, and that the editor is not to be read to learn whether one was: without the tool's replies the model did that instead, a tool round all the same. Test events there no longer ask for an evidence call either. The phase judge now covers what only the tool recorded there. It records Test from a hand trace while nothing can run the written code, judged with the code in view and only from speech, and the STAR steps, quoted only from inside the behavioral round and not from a story the interviewer told first; it reads the round as new once it holds something, and again when its start moves back to the interviewer turn in flight. Repeat, Example and Algorithm stay open after Coding while a step about the code is, and a coding-round step may be quoted from a comment the candidate typed, as one whose speech was not recognized is asked to, recorded as an editor snapshot. It can report a declined probe, and the platform writes the end-of-session skips for a started round that the wrap-up used to ask for, except where the probe was declined. The coding round's routine judgments leave 8 of the 48 for a behavioral round still to come; the last judgment and a round transition may spend them. A whiteboard keeps the tool, its wording and the judge as they were, since the judge is not shown the board.
1 parent 67866dd commit 6e4cc2b

14 files changed

Lines changed: 967 additions & 314 deletions

File tree

‎docs/interview-contract-versions.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ Bundle 36: live prompt 27, report prompt 20, rubric 1, report schema 3.
1212

1313
| Bundle | Introduced |
1414
|---|---|
15-
| 36 | The live `read_editor` guidance says that before any editor snapshot the editor holds only the starter, which needs no read, so the opening turns do not wait on a tool round trip to see it. The `earlierSteps` reminder asks for an unrecorded earlier step with the next evidence call rather than before speaking, so it no longer adds a second tool round ahead of the reply. A turn that answered the candidate with tool calls alone and said nothing is followed by one note that asks for the reply if one is due; Jim may still answer it with silence, but not with nothing, since the note is owed a completed turn. In coding interviews, until Coding is recorded, the interviewer leaves Repeat, Example and Algorithm to the phase judge, which records them from the candidate's own words, and is told not to call `record_framework_evidence` for those three, so the reply does not wait on that tool round. A coding silence nudge says whether the candidate has written code, instead of offering branches for an empty editor and for code that leave the interviewer reading the editor to choose between them; it says nothing is written only when the editor holds just the starter or nothing, asks them to begin coding once understanding, example and algorithm are all recorded, and leaves the choice to the editor for edits too small to count or too long to compare. The `read_editor` guidance adds that a language named aloud needs no read, since choosing it on the tabs sends its own event. The report prompt, rubric and report schema are unchanged. |
15+
| 36 | The live `read_editor` guidance says that before any editor snapshot the editor holds only the starter, which needs no read, so the opening turns do not wait on a tool round trip to see it. The `earlierSteps` reminder asks for an unrecorded earlier step with the next evidence call rather than before speaking, so it no longer adds a second tool round ahead of the reply. A turn that answered the candidate with tool calls alone and said nothing is followed by one note that asks for the reply if one is due; Jim may still answer it with silence, but not with nothing, since the note is owed a completed turn. At a typed editor the interviewer is no longer given `record_framework_evidence`, each call of which held its reply back by a second model pass, and is told that the platform records every step and says so in a system event. There, test runs no longer carry the `earlierSteps` reminder, and a rewrite after the complexity was given asks only whether it still holds. The phase judge records Repeat, Example, Algorithm, Coding and Optimizations from the candidate's words and editor as before, keeping the first three open after Coding while a step about the code is, Test from their hand trace while nothing can run the written code, and the four STAR steps from what they say inside the behavioral round; its instruction defines those steps, lets it quote a coding-round step from a comment the candidate typed to explain it, and lets it report a declined probe. The platform writes the end-of-session skips for STAR steps a started round left without evidence, which the wrap-up used to ask the interviewer for, except where the judge found the probe declined, and there the coding round leaves 8 of the 48 judgments for a behavioral round still to come. A whiteboard interviewer keeps the tool and its wording, since the judge is not shown the board. A coding silence nudge says whether the candidate has written code, instead of offering branches for an empty editor and for code that leave the interviewer reading the editor to choose between them; it says nothing is written only when the editor holds just the starter or nothing, asks them to begin coding once understanding, example and algorithm are all recorded, and leaves the choice to the editor for edits too small to count or too long to compare. The `read_editor` guidance adds that a language named aloud needs no read, since choosing it on the tabs sends its own event. The report prompt, rubric and report schema are unchanged. |
1616
| 35 | The live interviewer receives the same worked examples shown on the candidate's screen, generated from the problem bank. When the candidate hides examples, their contents are omitted from the live prompt. Live prompt 26; the report prompt, rubric and report schema are unchanged. |
1717
| 34 | Optional English accent directions are appended by the live prompt builder without changing the original interview instructions. Default adds no text. Voice and accent Random choices are resolved independently before token issuance and retained for the interview and reconnects. Live prompt 25; report prompt 20, rubric 1 and report schema 3 are unchanged from bundle 33. The server stamps the resolved `interviewerVoice` and `interviewerAccent` after report generation, including incomplete reports. Default voice records the configured Gemini voice; Default accent records Default. These optional provenance fields are normalized at the browser sanitizer, following bundle 7, without changing scores or existing field semantics. Reports without them remain readable without inventing missing interviewer settings. |
1818
| 33 | The live clock policy explicitly forbids reading the TIMER label or its sentence aloud. The last platform reading remains authoritative; requested time answers and the five-minute warning retain their existing rules. Tool-assisted totals remain spend counters and never seed the comparable context peak. A total below the last completed single-generation prompt can establish a decrease; all observations in the tool turn are considered before deciding. Provisional usage received before a tool call is retracted from the comparable peak. Live instructions shorten repeated speech, evidence and interview-step guidance while retaining the three-sentence limit, safety boundaries, evidence requirements and mode-specific behavior. A paged `read_conversation` tool retrieves recorded candidate and interviewer speech omitted from local recovery tails, with substring search, byte cursors, bounded serialized pages and no platform instructions or private review notes. Recovery directs missing earlier details to that tool. Report brief 20 removes the rolling-assessment direction to report embedded instructions. Editor and board material require an exact candidate excerpt before reporting an attempted instruction; fences, headings and platform closing events do not establish one, and feedback does not describe prompt handling. Recovery removes the quoted owed event's superseded final platform clock. Additional anti-script wording was tested and withdrawn: it increased prompt tokens without preventing spoken timer leakage. Configured continuity inputs move the event clock to the final position instead of duplicating it. Configured editor, board and hint tool replies also move their redundant result clock to the authoritative continuity clock. Evidence-recording continuity omits the clock, matching ordinary evidence replies and retaining platform/read-tool time sources. A tool-response batch carries continuity and the final framework checklist once each in their final eligible responses, preserving every result and ordered state change. The final eligible coding evidence response can carry a changed, parsed editor excerpt once so speech-deferred reviews do not leave a stale editor for the tool continuation. A successful changed-code response, including a default editor read, requests the proactive review once using the final batch state, prioritizing the candidate's latest question; a default read that stops at its page limit neither carries the review nor marks the buffer shown, and only a read showing the whole buffer marks it shown, so later-line pages and requested hints whose editor page stopped at its limit do not; only a delivered review consumes an unreviewed parsed substantive revision, so explicit later-line reads, failed writes and later edits still arm the watcher. Timed replies send their private clock as non-generating ordered context, followed by the action on the same ordered channel; while the provider owes or is producing a reply, the action goes as realtime text instead, since a completed client turn would interrupt that reply even before its first output. The socket counts as owing one from detected candidate speech, its transcription, an interruption, model output or a reply-requesting send until the turn completes, and provider evidence lapses after 10 seconds while a requested reply stays owed through the longest configurable reply deadline, which the reply watchdog enforces first. Each timed reply counts as one input. Silent context and untimed input retain their transport. Timer-like candidate source text stays intact. Provider speech activity, while audio is attached and frames are not paused, defers idle prompts, checkpoint refreshes and idle assessment calls until speech ends, without treating activity as transcript evidence. Advisory transport restarts wait through the quiet transcription tail before resuming a stale checkpoint. During ongoing speech the periodic room watch checks held advisories without waiting for another provider event or stall; they finalize the partial transcript before replacing the socket, cancel its unheard reply, and restore the full quoted prefix silently before queued audio continues. The silent speech handoff omits its clock; requested-time reads retain their clock sources. A bounded 40-second microphone queue retains frames while the room loop reconnects, replacing the dependency default of 100 ms. Only a post-cancellation checkpoint can resume that handoff; a resumed socket receives the listening note alone and keeps its context baseline, and a cold one the full cold briefing before it. A platform request the replaced socket still owed is named in the note and stays owed. Events the replaced socket emits while the prefix is finalized are counted in the log. Failed writes retain local briefing debt. Coding-only Live tool declarations offer only the six REACTO phases; behavioral sessions retain all ten phase choices. Tool descriptions remove procedural rules already present in the system instructions while retaining unique paging and snapshot semantics. Whiteboard compression checkpoints omit editor and test data and request the current drawing through `read_board`; a behavioral checkpoint does not return to the drawing. Report repairs compact only whitespace outside JSON strings in complete bounded responses, preserving values and existing malformed or oversized fallback behavior. Whiteboard initial and recovery instructions describe drawing and evidence rather than editing, implementing a function or passing executed tests; coding instructions remain unchanged. Editor and requested-hint reads distinguish a stale or unknown browser-report source from current code, independently of retained Test credit; current and no-report reads remain unchanged. Editor-update outcomes, combined-mode progress and recovery also use exact displayed-report provenance while retaining edit-tolerant Test credit. Recovery briefings add only the owed-question rule and any opening or latest candidate line their transcript tail no longer holds whole, rather than repeating their own no-restart rules or asking a silent checkpoint to continue. Tool continuity follows the same plausibility gate and is omitted while the floor is held. Configured compression carries bounded continuity beside platform inputs after the opening while a cut is plausible (before any measurement, after a detected cut, or while the last comparable prompt count is in the upper half of the band between target and trigger), including the active round, evidenced steps, pending request and a bounded prior candidate opening in coding, so the first response after a context cut can continue without replaying the greeting or full problem restatement. Provider-default sessions retain their existing input path. Default editor reads can collapse identical consecutive leading comments while keeping every distinct note, original line numbers and explicit line access. Tool-assisted usage sums remain spend counters but are excluded from context-size comparisons to avoid false checkpoints. Completed coding retains the first silence check and 30-second attempt throttle, then doubles the spacing of successful same-episode nudges up to 480 seconds. Meaningful candidate activity resets the episode; failed writes do not advance its successful-send cadence. Suppressing a nudge still permits significant editor reviews, and unfinished coding, behavioral support, required replies and trusted timers retain their behavior. Active provider speech detection plus recent voiced microphone frames defers reply-timeout, generation-stall and deferred advisory restarts across watch, event and playout paths. Speech end or acoustic quiet rebases outstanding reply clocks once; stale activity edges alone cannot renew them. A provider speech start with no microphone or provider evidence for 10 seconds is treated as ended. Paused frames do not count as acoustic evidence, context resets discard the acoustic history, and the absolute interview deadline remains authoritative. During real incoming speech, a parsed substantive editor change settled for ten seconds can be sent as non-generating context. Successful delivery has its own language-qualified snapshot and becomes the baseline later editor updates diff from, without consuming optional review or reply debt; silent delivery itself diffs from the code already shown by any route and sends nothing when the model already holds the current code; failed writes remain pending and socket recovery resets or initializes that snapshot. Ordered local writes do not guarantee consumption order against realtime audio. Coding transitions are conditional on implementation not having begun; otherwise review current work. Recovery, review, silence and timer prompts can carry a bounded edits-presence note, until Coding is evidenced, without treating edits as a completed or correct implementation. Silence nudges omit the empty-editor branch when the editor holds candidate edits, and coding-only sessions omit the behavioral-question ban from nudges and reviews, which their system instruction already carries. Silence policy still selects its existing setup-error and next-step branches before that note. Existing final checklist tool responses can expose candidateEditsPresent only in the coding editor round, using the unchanged written-content heuristic; a comment or stub may qualify. Tool continuity wording is shorter with identical latest-utterance selection, trust fences, JSON data and timer. Cold and checkpoint recovery envelopes are shorter while retaining archive lookup before earlier-speech answers, reconciliation with the current local record, historical-data trust boundaries and the active-session and no-restart rules. Completed Example walkthroughs are not reopened without a relevant implementation change or a concrete unresolved concern; initial ordinary and boundary examples and execution-mode Test requirements are preserved. Outside candidate speech, judged from the microphone's own voice level or provider speech detection, each Live socket receives one camera presence frame; frames during speech keep the five-second interval, and board images are unaffected. |

‎docs/provider-cost-and-degradation.md‎

Lines changed: 15 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -195,21 +195,26 @@ rate. `CODETRIAL_MAX_INTERIM_REVIEWS` defaults to 6, accepts `0` to disable
195195
the reviews, and is capped at 72. This is a quota guard, not a completeness
196196
limit: the final report still receives the complete transcript and editor state.
197197

198-
The phase judge, which records the conversational REACTO steps from the
199-
candidate's own words, uses the same report model and keys. Every judgment
200-
needs a candidate turn or editor it has not read, or the window of a call that
201-
failed, and a step it records still open: Repeat, Example and Algorithm until
202-
Coding is recorded, Coding and Optimizations while there is code. On the watch
198+
The phase judge, which records the interview's steps from the candidate's own
199+
words and editor, uses the same report model and keys. At a typed editor it is
200+
their only recorder, since the interviewer has no evidence tool there. Every
201+
judgment needs a candidate turn or editor it has not read, or the window of a
202+
call that failed, and a step it records still open: Repeat, Example and
203+
Algorithm until Coding is recorded, and at a typed editor after it while a step
204+
about the code is, Coding and Optimizations while there is code, and at a typed
205+
editor Test while nothing can run the written code and the STAR steps once the
206+
behavioral round has opened, read from where it opened. On the watch
203207
tick one starts once the candidate has been quiet and not typing for 2 seconds,
204208
no more often than every 6 seconds, or every 30 seconds when only the editor
205209
changed; typing pauses every few seconds while Coding is open, so editor-only
206210
judgments also take at most 12 of the calls. The end and a held round
207211
transition may start one sooner: the end waits at most 5 seconds in all for its
208212
judgments, and a round transition is held at most 5 seconds while the room
209213
keeps running, or until the resume when the interview is paused. Both share the
210-
ceiling of 48 calls per interview. Each reads at most 6 KiB of transcript tail
211-
and 6 KiB of editor. A rate limit on a judgment does not cool the key, so the
212-
judge cannot leave the final report without one.
214+
ceiling of 48 calls per interview, of which a coding round at a typed editor
215+
leaves 8 for a behavioral round still to come. Each reads at most 6 KiB of
216+
transcript tail and 6 KiB of editor. A rate limit on a judgment does not cool
217+
the key, so the judge cannot leave the final report without one.
213218

214219
## What bounds concurrency
215220

@@ -408,8 +413,8 @@ and prohibit returning to it merely to recover earlier work. They also omit
408413
coding-run progress cues. Coding checkpoint output is unchanged by these mode
409414
guards; no token reduction is claimed for this correctness fix.
410415

411-
Only while a pair is configured do `read_editor`, `log_hint` and
412-
`record_framework_evidence` answers also carry the latest unanswered candidate
416+
Only while a pair is configured do `read_editor`, `log_hint` and, at a
417+
whiteboard, `record_framework_evidence` answers also carry the latest unanswered candidate
413418
utterance, as quoted data, so a reply owed across a cut is not lost, and only
414419
while a cut is plausible (as below) and the floor is not held. Without a
415420
pair these application-level additions are omitted. Provider-default sliding

0 commit comments

Comments
 (0)