A broken Bulk-Only transport is recovered by the class's own sequence, and one that stays broken is taken offline

[Japabu]

The T14's stick hung four boots the same way, and run 55's black box carried
the shape. This branch makes the mass-storage transport recover on a ladder —
the class's own Reset Recovery, then a port reset that keeps the disk, then
offline behind one last reset — each rung ended by the device's own answer and
each on a bound of its own; and since round 7, a disk whose device a reset of
this driver's moved to another port takes its number back when the same device
binds there, so the volume on it carries on. Since round 11, every device found
connected at bring-up is reset before this kernel asks it anything, and the
command a rung that took sends again is bounded by the call, not by an
operation budget the break already spent.

Not armed. Round 6's ladder met the owner's ruling on the T14 for the first
time (run 79: no replug). Round 7 survived a staged break twice there (runs 80, 81); Round 8 fixes the wait that redded PR CI. See Round 11 first: T14 run 84, the first real failure, and what it does and does not show.

Builds on #447, which is open and not merged: its head 78929f12 is in this
branch as 7fd707c3. origin/main at 2fbfdf9d (the governance cut, #475) is
merged in as 7f4155b1: two conflicts, both main's rewrite — src/tiers.rs
taken as main's (its RELEGATED is gone; the branch's two names stay Nightly by
their registration word in tests/toyos.rs, where a comment citing the deleted
validate_ci_profile is cut), and tests/test-durations taken in main's
column-less format with the branch's one row
(usb_reset_records_the_phase_it_cut 64260) added in it.

Round 12: the review of 0a6b7da5 answered

Commits e45c56f5 (the NOTEs and the REMOVE) and cec68125 (two issues).
origin/main is cf715c49, already in this branch; the merge was a no-op.
Nothing a bench line shows changed: the boot scan's decision, its log line
and the reset it writes are what they were at 0a6b7da5, and the only edits
are where a question is asked and who may ask it.

The round's BLOCKER is not answered here: the inherited reset is
bench-unproven, the T14 boot that judges it is the orchestrator's, and this
head is what it is booting. No image is staged by this round, and Round 11's
T14 subsection describes what was requested at 0a6b7da5, not a run.

The NOTEs

• init_device asked port::reset_needed a second time only to choose a log
  string. It asks once now: port::inherited_reset takes that answer instead
  of the port, so the log line and the reset kind come from one read. No
  caller sees a different value, and the trained-link case is still the only
  one the two disagree on.
• With that, reset_needed's header is true as written — it is the one place
  the question is answered, and inherited_reset only says what its None
  costs at bring-up.
• Portsc::reset_changed is private: nothing outside portsc.rs asks it.

The REMOVE

The four T14 run 86 / run 84 citations in toyos-xhci/src/port.rs are
gone, and so is the sentence in Round 11's defect 1 that promised a T14 run
"below" this body does not carry. The test's words are unchanged — the same
four PORTSC values, asserted the same way — and what they mean is now said
without a run number.

The two CI reds nobody held

log_reserve_window ("Init process crashed during boot") and log_nested_emit
("the record-making child would not start: other error"), both on attempt 1 of
run 35760805511 at 0a6b7da5, both green on their ALONE re-run, neither
quarantined, neither on main's CI at cf715c49. They are one shape said twice:
a process that would not start. Shard 9's guest had test-runner and
soundd fault on the first instruction of their own images with the PML4 entry
for the user half absent, on two CPUs in the same millisecond, each with its
own CR3 and PCID; shard 10's log gate got Err from spawning
/system/bin/echo, printed as other error — the kind our std gives an errno
it cannot name.

Re-measured rather than argued, on the dev host, 2026-09-22:

arm	sessions	log_reserve_window	log_nested_emit
this head	4 × cargo test (fast tier, 347 names)	4/4 PASS	4/4 PASS
cf715c49, whole change reverted	4 × cargo test --test toyos-build	4/4 PASS	4/4 PASS

The revert was the whole change as a checked patch (52 files, 467 insertions,
8,781 deletions), applied under a trap that restores it; with it applied
git diff cf715c49 was empty, and the tree read clean after. The base arm ran
the guest binary alone because a patched revert leaves
kernel/src/log/recovery.rs tracked and absent, which reds sourcegate's walk
in cargo test --lib; the guest suite is the same binary either way.

Neither arm reproduced either name, so neither arm acquits this branch:
0/4 against 0/4 is an instrument that does not reach a failure a KVM shard
produced with eight guest vCPUs on a four-CPU runner. Filed, not attributed and
not fixed:
issues/kernel/two-processes-started-at-an-entry-point-nothing-had-mapped.md
and issues/kernel/a-spawn-of-echo-was-refused-with-an-error-nothing-names.md,
each carrying its capture, both arms, and what a next sighting must capture. No
quarantine row is proposed — that is the orchestrator's call and needs a says
fragment. For the record, the other reds those eight sessions produced were
blocked_dump (the panic
issues/kernel/the-blocked-task-dump-panics-when-a-cpu-is-inside-inbox-submit.md
carries), a quarantined console_locale_detect stall, i8042_undecoded_bytes
and — on the base arm — metal_device_probe.

Gates at cec68125 (tree clean before and after)

command	exit
cargo test -p toyos-xhci	0 (111 passed)
cargo test --test toyos-build -- --nightly usb_transport_break	0 (1 passed)
… --nightly usb_	0: 12 passed, 1 quarantined and green: usb_disk_index_stable
… --nightly xhci_	0 (16 passed)
… --nightly blackbox_	0 (7 passed)
cargo test (the fast tier)	0: 347 passed, 8 quarantined and green … 347 total (179.5s), 104 held back for the nightly tier
cargo test -p toyos-build --lib	0
cargo test --workspace --exclude toyos-build	0
cargo run -- --clippy	0

Unsure of

• Whether either CI red is this branch's. Four sessions each way say only that
  the dev host does not reach it.
• Everything Round 11 was unsure of still stands, defect 1's bench proof first.

Round 11: T14 run 84, the first failure nobody staged

Commits b7fd5929 (the change), a710dc21 and d85d03df (a staging for
defect 1, reworked, then deleted: see below), 4959c4ef and a07f021d
(issues). origin/main at cf715c49 is merged in as 72e0a648, no conflicts.

Run 84 (read by run 85's page): port 13 link already trained, no reset needed at 0.595 s; the bind's TEST UNIT READY, INQUIRY, READ CAPACITY and
serial string all completed in step; the boot's first READ(10) had no answer in the command phase in 2000 ms; the class reset's TEST UNIT READY babbled;
the port reset took and TEST UNIT READY answered under its own tag; the READ
was then not issued: 2000ms because the first wait had spent the operation;
the next READ's data did not come in 2 s; offline; no boot partition; panic at
rootfs.rs:71; the firmware and Linux could not use the stick until it was
replugged.

Defect 1: a device the firmware left is reset before anything is asked of it

Change. The boot scan resets every device it finds connected
(toyos_xhci::port::inherited_reset): a trained USB3 link is warm reset
before enumeration and says so (xHCI: port N link already trained before this kernel ran; warm resetting it before its device is asked anything). USB2 ports
were already reset. Hot-plug is unchanged: a link this kernel watched train
came up from power-on (reset_needed, the sim's
a_trained_superspeed_link_is_not_reset).

Why, from the specifications (recalled, not re-read: no spec text is in
reach of this worktree). xHCI 1.2 §5.4.1: HCRST drives no USB reset on the
root ports, so the controller's reset tells the device nothing. USB 2.0 §9.4.6:
SET_ADDRESS to a device in the Configured state is not specified, and that is
what an Address Device on an unreset, firmware-configured device sends. USB 2.0
§9.1.2: enumeration resets the port before it addresses the device. Warm rather
than hot because the link's state is as unknown as the device's, and a warm
reset is a hot one plus a retrain (xHCI 1.2 §4.19.5.1). Mature kernels do the
same: Linux's hub driver resets every port before it addresses the device on
it, a port it finds enabled when it loads included.

The hypothesis against run 84's own records: not borne out. At 0.596 s the
stick completed four tag-checked round trips, so its Bulk-Only state was in
step when the first READ went out, which a device still inside a firmware's
data-in would not be. And the recovery's own warm reset of port 13 took at
2.707 s, TEST UNIT READY answered, and the next READ's data still did not come.
What the records do show is a stick whose protocol answered every reset while
its media stopped at the first command that needed it: the reading
issues/kernel/a-stick-that-answers-late-is-broken-by-the-two-second-abandon.md
already carried, and run 84 is now its third record. This fix removes an
unspecified request and does what the specifications and Linux do; it is not
shown to cure run 84, and it is bench-unproven at this head: no T14 boot has
run against it, and this body carries none. The orchestrator's boot of this
head is what closes that.

Cost. Under QEMU, nothing measurable: three boots of xhci_superspeed_ports
each way, head and reverted, all read port 1 connected and port 1 enabled, speed=4 within 1 ms of each other and Boot: peripherals ready at 116–117 ms
(QEMU completes a reset at once). On the T14, an estimate from run 84's own
warm resets of port 13: 105 ms each (2.602→2.707, 4.758→4.863 s), each
including the Stop Endpoints before it, so an upper bound per inherited USB3
device, not a boot measured with this change.

Not stageable under QEMU. QEMU 11.1.0's xhci_address_slot
(hw/usb/hcd-xhci.c) calls usb_device_reset before it delivers SET_ADDRESS,
"Make sure device is in USB_STATE_DEFAULT state": under QEMU every enumeration
resets the device whatever the driver did first. Two stagings measured exactly
that. With this kernel playing the firmware (a READ(10) whose data nothing read,
then the device forgotten) and the fix reverted, the scan's first command was
still answered; a probe TEST UNIT READY sent to the same slot before the
re-enumeration was stalled, so the staging did leave the device inside the
data-in, and the Address Device took it out. Both are deleted (d85d03df).

Negative control NOINHERIT (init_device back to reset_needed, no reset
for a trained link): --nightly xhci_superspeed_ports EXIT=1,
"xHCI: port 1 link already trained, no reset needed" does not read "link already trained before this kernel ran; warm resetting it before its device is asked anything". That is the reset's own record, not a harm: no harm is
observable in QEMU, for the reason above. Independent oracle: the
specifications above and the T14. Pure:
a_device_found_at_bring_up_is_never_enumerated_without_a_reset.

Defect 2: the command a rung that took sends again is the call's

toyos_xhci::call::AfterBreak::issue(now, operation_ends): the first command
after a rung took, or after a held disk came back, is the one the break was
in, and only the call's window decides it; every other command starts
something new and needs the operation's budget too. AFTER_BREAK and
CALL_AFTER_BREAK are unchanged (4.75 s < tlb::ACK_TIMEOUT 5 s). The
command sent again gets at least what the rung that took left of its own
window: asserted at every took in every_path_through_a_call_ends_inside_its_bounds.

• Run 84's timeline, pure
  (a_read_whose_wait_spent_the_operation_goes_out_again_after_the_port_reset_took):
  operation ends 2.600 s, wait began 0.602 s, broke 2.602 s, class rung out of
  step at once, port rung took at 2.757 s: the READ goes out again with its
  whole 2 s, the next new command is refused by the operation's budget, and a
  break of the READ finds the last rung whole inside 5.352 s.
• Staged (usb-first-wait-spent, judged in usb_transport_break): a READ
  whose wait is unanswered for its whole 2 s (the staging now opens the call
  from its own wait, not an earlier transfer's), a class reset out of step, a
  port reset that takes: a READ whose wait spent 2.002 s went out again after the port reset took, and returned the host's bytes.
• Negative control REISSUEOP (again forced false, the base's order):
  cargo test -p toyos-xhci EXIT=101 (the two new tests), --nightly usb_transport_break EXIT=1, read refused=true matched=false.
• The re-send after a held disk came back is exempt the same way, bounded by
  the call; SilentReturn still ends at 4.250 s.

Defect 3: the end state, and the panic

What the stick last received. From the offline rung, a warm reset of port
13 with nothing after it (reset=true and nothing sent after it, Reset Device=true); the READ it was owed data-in for was ended by that reset. Then
the panic's reboot, 60 s later: every connected port reset, the controller
halted and reset, its ports unpowered, no Bulk-Only command was open, so this reset cuts none. So every path this kernel drove ended with the device reset
and inside no command, and the stick was still lost: its media had stopped
before the first READ, and nothing a host does on this machine clears that (the
T14's ports do not drop VBUS). Recorded in
issues/kernel/a-cut-bulk-only-command-does-not-brick-the-benchs-stick.md; no
code change.

The panic path quiesces USB already: acpi::reboot calls
stop::before_reset on every reset, the panic's included, and run 84's page
carries its account (usb-quiesce: xHCI 00:14.0 5/5 connected port(s) reset … 16/16 port(s) unpowered). The panic at rootfs.rs:71 stays: a boot with no
root is over.

Not stageable under QEMU either: its usb-storage returns to its reset
state on every port reset and every Address Device, so a second boot finding
the disk could not red. No staging was built.

Found and filed

issues/kernel/a-disk-operation-can-spin-past-the-tlb-ack-tripwire-before-its-break.md:
CALL_AFTER_BREAK bounds the spin from the wait that broke, and an operation's
earlier batches spin before it under the same lock: 1.99 + 4.75 s is past the
5 s tripwire. Arithmetic on the constants; not seen.

Run 86: the reset was judged before it finished (fixed in 0a6b7da5)

The plain boot of the first round-11 head regressed on the T14: port 13's warm
reset was judged in the millisecond it was asked for, on
PORTSC 0x002212b1 — PR still set, link RxDetect — because the PRC beside it
was a flag nothing had cleared, left by whatever reset the port before this
kernel ran. The scan called the link untrainable, skipped the port, found no
disk, and panicked in rootfs.rs. The stick itself was fine afterwards.

• Portsc::reset_finished is the completion rule — PR back to '0' and PRC
  or WRC set (xHCI 1.2 §4.19.5, §4.19.5.1) — and every wait for a reset asks
  it: the boot scan's, the port machine's, the ladder's port rung's. The bound
  is unchanged (port::RESET_DEADLINE_NS, 2 s, the driver's transfer budget).
• port::reset_write clears the reset-finished flags it saw, so the flag that
  comes up next is this reset's; the ladder's hand-written acknowledge beside
  its own reset write is now that one write.
• Pure: a_warm_reset_is_finished_only_once_the_controller_has_cleared_pr
  walks run 86's word and run 84's completion word on the same port.
• Negative control STALEPRC (reset_finished back to the flag alone and
  reset_write without its acknowledge, the whole of this fix):
  cargo test -p toyos-xhci EXIT=101 on that test. --nightly xhci_ EXIT=0
  with it applied, and that is the finding: QEMU never leaves a stale PRC and
  clears PR with the completion in the same word, so no guest boot in this
  suite can red on this. The reading is the specification's rule and run 86.
• Gates re-run whole at 0a6b7da5; the table below is that run.

Gates at 0a6b7da5 (tree clean before and after)

command	exit
cargo test -p toyos-xhci	0 (111)
cargo test --test toyos-build -- --nightly usb_transport_break	0
… --nightly usb_reset_records_the_phase_it_cut	0
… --nightly usb_	0: 12 passed, 1 quarantined and green: usb_disk_index_stable
… --nightly xhci_	0 (16)
… --nightly blackbox_	0 (7)
cargo test (the fast tier)	0: 347 passed, 8 quarantined and green … 347 total (158.8s), 104 held back for the nightly tier
cargo test -p toyos-build --lib	0
cargo test --workspace --exclude toyos-build	0
cargo run -- --clippy	0

Each arm — NOINHERIT, REISSUEOP (both measured at d85d03df) and STALEPRC — was
applied as a checked patch, run, reverted, and the tree read clean after it.

Unsure of

• Whether defect 1's fix changes run 84's outcome at all (above).
• A device whose warm reset fails at bring-up is now skipped where it used to
  be enumerated untouched.
• The T14 cost per inherited device is an estimate from run 84, not a boot
  measured with this change; run 86 measured only a reset judged too early.
• Only the pure crate can red on run 86's defect: QEMU never leaves a stale
  reset-finished flag, and clears PR with the completion in the same word.

T14

Two images of this head, requested: usbbreak (the
usual usb-transport-break arm) and jobcase (no staged break). Both must
show port 13 link already trained before this kernel ran; warm resetting it
and then port 13 enabled, speed=4 before slot N enabled — which is what run
86 did not do.

Round 10: the review of ea1a44a6 answered

Commits 02cf8114 (the change; its title says "round 3" and means this
review, round 2 of the review comments), b0ea0007 and ee81dd4a (the two
moved-stick stagings boot four CPUs; the debt is named first). origin/main
at 845f3e20 (#461) is merged in as 696a8633, no conflicts. Nothing a
bench line shows changed: the kernel's production change is the flush-debt
bookkeeping (the two-hop case, which no T14 run has reached: run 83 broke on
the first write, no flush owed) and AFTER_BREAK's declaration moving, its
values unchanged; everything else is behind boot-actuators parameters no
metal row names. No T14 image is staged.

BLOCKER 1: the re-issue after a hold is timed from the break

usb-return-silent stalls the returning stick's bind 1.5 s (so it arrives
late in the held call, and still inside it) and leaves every transfer of the
operation the held call sends again on it unanswered, each wait spun to its
end (msc::return_silent, checked in wait_transfer). Moved::SilentReturn
requires, off the kernel's stamps: the held call on the break's CPU is the
one that sent the operation again (… went out again on it: it failed), the
device went offline on the last rung, the call ended inside
AFTER_BREAK.whole() (4.75 s) of the break, and not before its waits could
reach where the last rung begins (4.25 s) — so the staging really spent what
the call had left. Head: 4.250–4.251 s in each of six runs (three at b0ea0007, three at
ee81dd4a including both gate runs).

Negative control REISSUEFRESH (the review's diff, ctrl.after_break = if back { AfterBreak::CLOSED } else { call };, applied to ee81dd4a and built):
see the table. At 02cf8114 it was red SilentReturn: the call its transport broke in ended 5.114 s after the break, past the 4.75 s.

Why four CPUs. The held call sees the stick back only if a CPU in no call
on the held disk binds it. At two CPUs one REISSUEFRESH run put both in calls
on disk 0 until the call's bound (the held-disk issue's shape), so nothing
was sent again and the staging measured nothing; FlushedStick took that shape
in each of its six unmutated runs at two CPUs (the commit message of
ee81dd4a says five), and in REISSUEFRESH's alone run at b0ea0007 the job
reset before the stick bound, which the judge's order refused. Both
now boot smp: 4; the other moved-stick boots keep two, where the tolerant
judge already covers both shapes.

The judge reads the call that broke, not the first matching line. A call
on another CPU that found the disk held ends its own, and FlushedStick's first
run measured one such at 14 s from the break. The held call is now the first
… went out again on it / … is still held … line on the break's own CPU.

BLOCKER 2: a carried debt is owed again

The debt is toyos_xhci::flush::Debt, pure: wrote, flush (a carried debt
fails the flush once, by name, without going to the device), flushed,
adopted(owed), and owed(no_cache) = carried || (unflushed && !no_cache).
The kernel's MscDevice holds one; unflushed and flush_lost are gone.
a_debt_carried_is_carried_again_until_a_flush_reports_it is the two-hop
case: write acknowledged on instance 1, adopted as 2 owing it, 2 leaves before
any flush and owes it, adopted as 3, whose next flush is Lost, once.
Negative control CARRYDROP (owed without carried): red in that test
and in a_device_with_no_cache_owes_only_what_it_carries.

BLOCKER 3: both edges of the debt

Pure: a_write_is_owed_until_a_flush_that_succeeded_and_not_after and
a_write_on_a_carried_debt_outlives_the_flush_that_reports_the_loss. Staged:
usb-transport-break-flushed breaks the first WRITE(10) after a write was
reported complete and a SYNCHRONIZE CACHE then succeeded, with no write since
— kept in the actuator itself, not read off the driver's debt, so it stages
what it says whatever the debt holds. Moved::FlushedStick requires that
line before the break, the stick taken back as disk 0 with no left owing a flush and no flush failed, and the job run to its reset. At ee81dd4a the held call
saw it bound 0.112–0.113 s after the break in each of three runs. Negative control NEVERCLEAR2 (flushed
leaves unflushed set): red in both pure tests and in usb_transport_break;
the FlushedStick judge now reads the debt lines before any other, so the red
names the debt.

NOTEs

• AFTER_BREAK is declared once, toyos_xhci::call::AFTER_BREAK; the kernel
  aliases it and asserts AFTER_BREAK.wait == USB_TIMEOUT_NS; the test's
  bound is call_after_break_secs(), read from it.
• kick_cpu is refused only before apic::init (main.rs), which runs
  before smp::boot_aps: no other CPU exists to be halted then. Said in
  ports_wanted's doc. The T14 boots x2APIC (run 83: LAPIC: x2APIC enabled (ID 0)).
• read_block_retrying's spin across BudgetExpired calls (up to
  block::DEADMAN) is added to the held-disk issue; not fixed here.
• REMOVE: the two T14 run 79 citations in tests/common/usb.rs are gone.

Gates at ee81dd4a (tree clean before and after)

command	exit
cargo test -p toyos-xhci	0 (107)
cargo test --test toyos-build -- --nightly usb_transport_break	0; also 0 in the three HEAD runs at b0ea0007
… --nightly usb_reset_records_the_phase_it_cut	0
… --nightly usb_	0: 12 passed, 1 quarantined and green: usb_disk_index_stable
… --nightly xhci_	0 (16)
… --nightly blackbox_	0 (7)
cargo test (the fast tier)	0: 347 passed, 8 quarantined and green … 347 total (171.0s), 104 held back for the nightly tier
cargo test -p toyos-build --lib	0
cargo test --workspace --exclude toyos-build	0
cargo run -- --clippy	0

Mutations (each applied as a checked patch, built, the kernel rebuilt for the
kernel arms, and the tree restored):

arm	tree	command	exit and first red
REISSUEFRESH (the review's diff)	ee81dd4a	--nightly usb_transport_break	1: SilentReturn: the call its transport broke in ended 5.115 s after the break, past the 4.75 s a call may spin for (5.114 s at 02cf8114, 5.112 s at b0ea0007)
NEVERCLEAR2 (self.unflushed &= core::hint::black_box(true) in Debt::flushed)	ee81dd4a	--nightly usb_transport_break	1: FlushedStick: "… disk 0 came back on port 3 slot 1 as the same device (…); its volume carries on, and it left owing a flush …" of a stick that came back as itself
NEVERCLEAR2	02cf8114	cargo test -p toyos-xhci	101: a_write_is_owed_until_a_flush_that_succeeded_and_not_after, a_write_on_a_carried_debt_outlives_the_flush_that_reports_the_loss
CARRYDROP (owed without carried)	02cf8114	cargo test -p toyos-xhci	101: a_debt_carried_is_carried_again_until_a_flush_reports_it, a_device_with_no_cache_owes_only_what_it_carries
NOOWED (Debt::adopted(owed && false) in the bind)	02cf8114	--nightly usb_transport_break	1: OwedFlush: after the break, no line reads "usb-storage: disk 0 flush failed: …"

The harness re-runs a red alone; REISSUEFRESH's alone run at ee81dd4a was red
on another boot: test-runner SEGFAULT execute unmapped address at _start+0x0, [qemu] Init process crashed during boot — the unattributed
shape Round 8 records under log_flush_retry, n=1 here, in a mutated tree; no
gate run at the head showed it.

Unsure of: SilentReturn and FlushedStick lean on four CPUs keeping one out
of every call on the held disk; a boot that puts three in calls at once would
miss the shape and say so by name, not pass. The 1.5 s stall leaves the held call
about 0.4 s of its 2 s operation budget to see the stick back: an estimate
from the constants (2 s, 1.5 s, the 0.1 s connect debounce), not measured.

Round 9: the review of 8dd2994c answered

Commits d60cd335 (the change), 99a81a8c (late's test bounds its loop),
f0695038 (two defects the moved-stick boots then showed), 9b577295 (an
issue file), ea1a44a6 (clippy). origin/main had not moved since 909f34b4; nothing merged.
Round 8's image booted as T14 run 82 (sha256 3b4a5bdc…84697): break at
1.638 s, disk 0 left port 13 … it is held 1.756 s, the same serial bound on
port 13 on cpu0 at 2.786 s, is back … it completed on cpu4 at 2.803 s.

BLOCKER 1: the whole call inside its bound, and the bind never on its CPU

• One call, one bound. AfterBreak is carried by served across every
  command of the operation, the hold and the command sent again, and closed
  there; the bind closes each of its own commands; scsi no longer closes it
  per command. A later command of the same operation spends what the break
  left it.
• The hold is part of the call (toyos_xhci::call::AfterBreak::hold): it
  opens the call where nothing broke, ends where the last rung begins, and the
  command sent again is clipped there, as after a port rung that took, so a
  break of it climbs on what is left. By construction:
  every_path_through_a_call_ends_inside_its_bounds walks every sequence of 6
  steps (rung entered, rung taken, hold, spin to the window's end) from three
  starts (a wait that broke at once, one that ran its 2 s, a call that found
  its disk held) and asserts every path ends inside the bounds' sum;
  a_command_sent_again_after_a_hold_leaves_the_last_rung_its_bound has the
  last rung whole after a hold. The kernel's const assertion keeps that sum
  (4.75 s) under tlb::ACK_TIMEOUT (5 s).
• The hold only waits for the verdict. It looks with try_lock (a bind can
  hold the lock for seconds), marks PORT_WORK_AT due and kicks every other
  CPU once (the connect's interrupt may be its own CPU's, which takes none),
  and takes the lock again for the re-issue only before the hold's end. Binding
  runs where hotplug binds: the port machine, on a CPU that reaches a pass.
• Staged under QEMU (Moved::SlowStick, usb-slow-return: the returning
  stick's bind stalls 2.5 s): the break's call ends is still held when this call may wait no longer … asked again 2.000 s after the break on cpu0, the
  stalled bind is on cpu1, the disk is still taken back and the job runs to
  its reset. Every moved boot that comes back asserts the held call ended
  within 4.75 s of the break (from the kernel's stamps) and that a bind while
  it held ran on another CPU. Six runs at f0695038: SameStick 0.112–0.113 s,
  bound on cpu1; SlowStick 2.000–2.001 s.

Found while measuring, and fixed (f0695038): with both vCPUs inside
calls on the held disk (a write and a read), no CPU took the pass, port 3 was
first enumerated at 2.455 s after both holds ended at 2.354 s, and the window
had passed: disk 0 lost. The window is the device's to arrive inside: a held
disk is not forgotten while a port reads connected and untaken, as it already
was not while an enumeration is under way. That shape still answers
BudgetExpired to the calls that waited, which is filed:
issues/kernel/a-held-disk-waits-for-a-pass-no-cpu-takes-when-every-cpu-is-in-a-call-on-it.md.
And the host moved the stick on the HELD record, which reaches the console only
when klogd runs; with the rung spinning, the cue could arrive after the
rung's 1.5 s (a device that left too late: offline). The staging writes its cue
to the console itself.

BLOCKER 2: a power cycle is not a move

MscDevice::unflushed is set by a WRITE the device reported complete and
cleared by a SYNCHRONIZE CACHE that succeeded. A disk held while
owes_a_flush (unflushed, and not no_write_cache) is taken back with , and it left owing a flush of writes it had reported complete, so its next flush fails, and that flush fails once by name (flush failed: its device came back from leaving its port owing a flush …). How QEMU's usb-storage reports a
cache: it answers SYNCHRONIZE CACHE with GOOD (its scsi-disk flushes the
backing), so this driver never latches no_write_cache for it; the driver does
not read the caching mode page. Arms under QEMU: Moved::SameStick (the break
is the boot's first write, nothing acknowledged: taken back clean, no failed
flush); Moved::OwedFlush (usb-transport-break-owed: the first write that
goes out while an earlier one is acknowledged and not flushed: taken back
owing, next flush fails by name). The T14 stick answers does not implement SYNCHRONIZE CACHE, so it never owes one.

BLOCKER 3: Late is pure

toyos_xhci::late::Late, with
a_ring_that_keeps_producing_is_given_up_on_one_ring_past_the_deadline (and
three more). The kernel wraps it with its clock and RING_SIZE.

BLOCKER 4: usb_reset_records_the_phase_it_cut

Not this branch's ladder, and not the driver: main's 5213b6e7 ("The loader
files the record's tail instead of scrolling it") moved the black box's ring
tail off the console into loader.log. The wedge's own records
(usb-wedge: stopping every CPU at the, wedge: staged) are tail; the
account naming the phase is head and was on the console in every red run
(usb-quiesce: a Bulk-Only command was open in its data (unqueued) phase …).
The QEMU half now boots a kept image and reads the wedge's records off
loader.log, as boot_deadline_ends_a_wedge does since that commit. Green at
this head, all four arms.

NOTEs

• SERIALSHORT: usb-serial-short asks each disk for its serial string in
  8 bytes; usb_transport_break's first boot requires every disk to say
  serial number named and not read. The reviewer's mutation now reds there
  (serial number "TOY.............").
• msc.rs:271: toyos_xhci::ladder::left(phase, data_out); a
  command-phase break of a command with a Data-Out phase is owed data and enters
  at the port rung. Test: a_break_before_a_data_out_phase_is_done_leaves_the_device_owed_it.
• msc.rs:2307: after an operation that could not find its disk, or that
  saw it go held, a disk found here is back.
• msc.rs:1991: a disk refused not ready (Bind::NotReady) or for want of
  a pool block while a disk on its controller is held is enumerated again
  (AfterSlot::Again), not left refused; ports that read empty are serviced
  first, so a teardown frees its block before an enumeration elsewhere claims
  one; and a held disk is not forgotten while its device's enumeration runs, so
  its own bind (its whole READY_BUDGET) never makes it late. The block arm is
  not staged.
• Not required, stated: the port rung's in-place recovery (Reset Device,
  Address Device, SET_CONFIGURATION, TEST UNIT READY on the same slot) has no
  hardware reading: runs 79–82 all read nothing is connected. The stale-event
  ordering (the old slot's Disable Slot before the new Enable Slot) is not
  stated in a test.

REMOVE

Deleted: the Round 7 sentence on pinning a CPU, Round 8's "not yet booted
there" and "Head image staged and requested", the "not this diff, #447's test"
clauses, identity.rs's run-79 provenance, and the T14 run NN citations in
ladder.rs and call.rs.

The two checks

Negative controls (mut-usbtransport-9.sh: each anchor matched once, the
diff non-empty, the kernel artifact newer than the mutation for guest arms, the
tree clean after; exits in mut-exits-usbtransport-9.txt and
whole-exits-usbtransport-9.txt):

arm	what it is	build / test	reds on
WHOLE	kernel/src/drivers/xhci and toyos-xhci/src restored to 8dd2994c (0 diff lines in those paths but late.rs, left unreferenced), tests kept, the three new actuators kept referenced	0 / 1	the first judge it reaches: serial number "TOYOS0BOOTSTICK1" read from a short answer. The per-blocker arms below reach the rest
LATECAP (B3)	Late never gives up past the deadline	0 / 101	late::tests::a_ring_that_keeps_producing_…
HOLDFRESH (B1)	the hold opens a fresh call	0 / 101	every_path_through_a_call_ends_inside_its_bounds, a_command_sent_again_after_a_hold_…
BINDHERE (B1)	the hold steps the ports itself (recheck_ports), as 8dd2994c did	0 / 1	SameStick: the stick was bound on cpu0 while the call held for it spun there with IF clear
OWEDNOT (B2)	adoption ignores the owed flush	0 / 1	OwedFlush: … no line reads ", and it left owing a flush …"
B4REVERT (B4)	tests/common/power.rs restored to 8dd2994c	— / 1	3 of 4 arm(s) unmet, the review's words
LEFTDATA (NOTE)	a command-phase break is not owed data	0 / 101	ladder::tests::a_break_before_a_data_out_phase_…
SERIALSHORT (NOTE)	the review's mutation	0 / 1	serial number "TOY............."
ARRIVED	a held disk is forgotten though a device has arrived	0 / 0	survives: the every-CPU-held interleaving comes at a rate (1 in 6 runs) and is not staged; filed

The pure arms ran at 99a81a8c (the pure crate is unchanged since); the guest
arms at f0695038.

Independent oracle: QEMU's own xHCI and usb-storage for the move, the
slow bind and a device that answers SYNCHRONIZE CACHE; the kernel's own stamps
for the bound; T14 run 82 for the move on hardware (the bind on another CPU
than the held call, cpu0 against cpu4); main's 5213b6e7 for B4's
premise.

Gates at 9b577295

Suites at 9b577295 (tree clean before and after, gates-exits-usbtransport-9.txt); ea1a44a6
changes one clone in the harness, after which clippy and usb_transport_break ran again.

command	exit
cargo test -p toyos-xhci	0 (103)
cargo test --test toyos-build -- --nightly usb_transport_break	0 at ea1a44a6; 6 of 6 at f0695038
… --nightly usb_	0: 12 passed, 1 quarantined and green: usb_disk_index_stable; usb_reset_records_the_phase_it_cut among the passed
… --nightly xhci_	0 (16)
… --nightly blackbox_	0 (7)
cargo test (the whole fast tier)	0: 347 passed, 8 quarantined and green … 347 total (173.9s), 104 held back for the nightly tier
cargo test -p toyos-build --lib	0
cargo test --workspace --exclude toyos-build	0 (139 test result: ok, none failed)
cargo run -- --clippy	1 at 9b577295 (redundant_clone, tests/common/usb.rs), 0 at ea1a44a6

Unsure of

• The bound runs from the start of the wait that broke, or from the hold. A
  call's commands that completed before the break are outside it, as on
  main; so are the retries above the block layer, which spin with IF clear
  across fresh budgets (issues/kernel/syscall-preemption-is-incidental.md,
  issues/kernel/every-wait-in-this-kernel-is-a-spin.md).
• A command sent again late in the call and broken finds its lower rungs spent
  and goes offline behind the last reset.
• A device that arrives just after its disk's window is adopted if no look ran
  in between; the window is judged at the next look, not at the connect.
• usb_transport_break's second boot counted a boot-stick break
  (no answer in the data phase in 2000 ms, 0 TRB(s) unreached and no stop event) in 1 of 4 runs at 99a81a8c; the same line is in main-based logs of
  other branches this session and in
  issues/build/a-loaded-suite-reds-a-volume-checker-on-both-arms.md.

Requested: t14-request-usbtransport-9.txt, the head image once, sha256 d777c6ae2d0c38e9b717d6628213c69301029b3977d0812d015e658a1d349903, armed usb-transport-break alone.

---

Round 8, head da873d72a6f2a4df178e04c23cd6d7d463de9dac: a wait reads the ring before it believes its deadline

origin/main at 0773ae87 (#476, #477) merged in as cd0a6b1a, no conflicts.
Then a0926caa (the fix) and da873d72 (an issue file for an unrelated red).

The cause

PR CI run 35661087102 (head 98f58bdf) redded four shards: boots whose programs
died at _start (execute unmapped address, PML4E 0) and host-side other errors. The one capture carrying the whole console (guest 10,
screen_console_panic):

[2.856] transport broke on SCSI 0x35: no answer in the status phase in 2000 ms; break 1 of 3 running
[4.031] Stop Endpoint timed out after 1175 ms with USBSTS=0x00000008 and CRCR.CRR=0
[4.031] the class reset was not answered; break 2 of 3 running
        Stop Endpoint / Reset Device not issued: the controller left an earlier command of this call unanswered
[4.081] is offline … every operation on it is refused from here
        read of 1 blocks at 9076 failed on disk 0; fault: … a file byte … that the device would not read

The Stop Endpoint's wait returned 425 ms past the class rung's 750 ms window,
which a loop reading the clock every iteration cannot do while its CPU runs; and
the endpoint read Stopped right after, so the command had been answered. QEMU
runs an xHCI command inside the vCPU's doorbell write, and a Stop Endpoint on a
Bulk-In whose packet belongs to an in-flight SCSI request cancels it
synchronously (scsi_req_cancel → blk_aio_cancel, recalled from QEMU's source
and measured below), waiting out the host's flush. The runner's first flush took
~3.2 s by the two waits' sum: 2 s broke the SYNCHRONIZE CACHE's status wait, the
rest held the vCPU in the doorbell write. wait_command checked its deadline
before the ring, so it returned "unanswered" with the answer in the ring; the
call then refused every command and the boot stick went offline. The ordering is
main's too; what made it fatal here is round 6's per-rung bound (class reset
0.75 s), where the old single 4 s budget left the stall room.

Reproduced on this host under TCG by delaying the boot stick's first flush
3.2 s (QEMU blkdebug inject-error with event=none,iotype=flush,errno=0, once=on,delay-ns=3200000000 on the harness's -drive, an uncommitted patch):
the head reds with CI's lines (Stop Endpoint timed out after 1202 ms, class reset was not answered, offline, SEGFAULT … _start+0x0).

The fix (kernel/src/drivers/xhci/wait/mod.rs, Late)

wait_command and wait_transfer read the ring before they believe the
deadline; past it they still read, giving up at the first empty read or after
one ring's worth (RING_SIZE), so a ring that keeps producing still cannot hold
the bound open. With it, the same staged flush reads: the Stop Endpoint
answered; the class rung's next command not issued: the bound on this part of the call is spent (the stall ate the rung); the port reset took … TEST UNIT READY under its own tag; the SYNCHRONIZE CACHE refused as ask-again on its
operation budget; SCSI 0x28 completed after 2 break(s) running; the transport came back. main (0773ae87, the whole change reverted in this worktree under
the same flush) breaks the same way and survives on the full 2 s Stop Endpoint
wait (endpoint 3 … recovering 2.369 s, next line 3.574 s).

The T14. The change touches the recovery path's waits and no rung, bound or
step. On hardware a CPU is not held inside a register write, so a wait's
deadline passes with the ring empty exactly as before: no bench line of runs
80/81 changes.

The two checks

Negative control (24 names CI redded, each alone, one filter per run, under
the staged 3.2 s flush; names-*-usbtransport-8.txt): the fix reverted (the
head before it) 24 of 24 red; with the fix 24 of 24 green; main 2 of 2
green. The fix is one file, so reverting it is the whole change.

Independent oracle: QEMU's own xHCI and usb-storage against a host flush
delayed by QEMU's blkdebug, and CI run 35661087102's record, whose lines the
staged arm reproduces one for one.

Gates at da873d72, tree clean (gates-exits-usbtransport-8.txt)

command	exit
cargo test (the whole fast tier)	0: 345 passed, 8 quarantined and green … 345 total (153.6s), 103 held back for the nightly tier. The run before it on a0926caa exited 1 on 90_stdio_buffering alone (a 10000-byte line captured as 7168 + 2832; green alone); filed as issues/build/stdio-buffering-saw-one-long-line-arrive-as-two.md, base not measured
cargo test --test toyos-build -- --nightly usb_	1: usb_reset_records_the_phase_it_cut alone, 3 of 4 arm(s) unmet; on the tree before the fix (cd0a6b1a) 1, the same line. Every other usb_* passed, usb_transport_break among them
… --nightly xhci_	0 (16)
… --nightly blackbox_	0 (7)
cargo test -p toyos-build --lib	0
cargo test -p toyos-xhci	0
cargo test --workspace --exclude toyos-build	0 (139 test result: ok)
cargo clippy -p toyos-xhci --all-targets -- -D warnings	0
cargo run -- --clippy	0
wall_clock_rtc_unstable alone on cd0a6b1a, before the fix and without the staged flush	0: CI's reds do not reproduce under TCG without it

Unsure of

• A slow SYNCHRONIZE CACHE still breaks the transport on CI (main too): 2 s is
  the status wait's bound for every command, and a hosted runner's first flush
  exceeds it. Now it costs a port reset, not the disk. Filed already as the
  screen_fatal_halt quarantine's issue.
• The class rung is skipped when a stall eats its window. The ladder reads that
  as a rung that did not verify, which is true of the call and not of the device.

---

Round 7, head c819b02cf9760022a8ea214640955c264c637ee6: the same device, come back, is the same disk

Four commits after the merge: 3c8785b0 (the change), 59099fd7 (the moved-stick
judge takes run 74's shape too), dcf5a4bd (the issue file), c819b02c
(clippy).

What the bench read (run 79, round 6's image 77fbcc71…)

Break at 1.842 s, entered at the port rung (correct per round 6). port 1 reset while recovering (hot on a USB2 port) … reset, and the port is enabled at
1.897 s; Address Device (after the port reset) failed: code 4 at 1.948 s — the
stick had left the USB2 half; the last rung found port 1 in RxDetect. write of 1 blocks at 11399 failed on disk 0. At 2.876 s port 13 connected, link already trained, speed 4, the same 0781:5581, and disk 1 ready on slot 6, 7507812 blocks at 2.878 s — 0.98 s after the reset. The boot parked to its deadline and
read WEDGED because disk 0, the root volume, was offline. No replug; the stick
enumerated at once under Ubuntu and the log partition checked clean.

What changed

Identity (toyos-xhci/src/identity.rs, pure, 6 tests). same(left, arrived) requires every field to agree: the device descriptor's idVendor,
idProduct and bcdDevice (USB 2.0 §9.6.1), the string its iSerialNumber names
(§9.6.1, §9.6.7), INQUIRY bytes 8–36 — vendor, product, revision — and READ
CAPACITY's sector count and size. Speed, port, slot and endpoint descriptors
are not identity: the move changes all of them. The bind reads the serial
(string descriptor 0 for the first LANGID, then iSerialNumber, into the disk's
own data buffer) and logs it: usb-storage: slot N serial number "…".

Why a different stick plugged in during the window cannot be adopted.
Every field but the serial number is the model's: a second SanDisk Ultra of the
same size answers the same USB ids, bcdDevice, INQUIRY and capacity. The serial
number is the one field a unit carries of its own — the Bulk-Only class
specification requires a mass-storage device to publish a unique one (BOT 1.0
§4.1.1, recalled, not read this round). So a device whose serial is absent or
unreadable is never adopted, as the disk that left or as the device that
arrived (Differs::LeftUnnamed / ArrivedUnnamed): identity cannot be
proven there, and it is not assumed. What this cannot tell apart is two units
sold with one serial number, which some cheap sticks are; the capacity and
INQUIRY checks then still bind it to the model, and no more.

The companion port, and why the driver does not need it. xHCI 1.2
§4.19.7: "The mapping of xHCI Root Hub Ports to the physical USB connectors of
a system is defined by platform implementations and outside the scope of this
specification." The Supported Protocol capability (§7.2) says which port
numbers speak USB2 and which USB3 (on the T14 port 1 is USB2 and 13 USB3) and
never which two share a connector; Appendix D gives that to ACPI _UPC/_PLD
group tokens, which this kernel does not parse. Read off
scratchpad/r2-xhci-spec.txt this round. A companion link could only say
where to look — and the port machine already enumerates whatever connects
anywhere — and could never say the device is the same one. Identity answers
both, so no pairing was built and the port is not checked.

The bound. RETURN_WINDOW = 2 s from the reset this driver made:
twice run 79's 0.98 s (reset to bound on port 13), and a const assertion
holds it at no more than the port rung's and last rung's bounds together (1.5 +
0.5 s) — what the ladder already gave a reset device to answer, which a device
that left has made moot. identity::returns_by(reset_at, now): a disk is held
only if this driver reset its port and it is seen gone inside the window; a
device that leaves with no reset of ours was unplugged, as before.

Where the wait is, and why it cannot starve a rung. A disk is held at one
of two points: its port read empty inside a call (the port rung's reset found
nothing connected, the last rung's did, or a transfer's wait saw the port go),
or its port's teardown ran within the window. It leaves its pool block (which
stays claimed by the old port until that teardown) for a per-controller list;
the poll forgets it when the window passes, saying disk N did not come back within 2000 ms of its port reset; it is offline, and wakes for that. An
operation on a held disk (storage_read/write/flush → served) waits after
the ladder has ended — the last rung has run, or found its port empty — so it
spends nothing any rung is owed. It waits with the controller lock given back,
stepping the ports itself (recheck_ports, 1 ms apart), because its CPU is
the one it holds; the enumeration and bind of whatever arrives happen inside the
wait. It ends one of three ways: the disk is back (the operation is issued again,
whole); the window passed (Device, as any offline disk's operation); or the
caller's own block::OPERATION passed (BudgetExpired — nothing was issued
while it waited, and every caller asks again above its locks).

What a filesystem sees; a write in flight when the device vanished. A write
reports complete only on its own CSW with status 0. A device that left inside
a write sent none, so that write was never reported complete; the reset that
moved it ends the command at the device (USB 2.0 §9.1.1 — every state returns
to Default). The operation is issued again whole, from the caller's own
buffer: every CDB here is idempotent, so blocks the device took before it
left are written again with the same bytes, and a block it took part of is
written whole. The re-issued command is the one that broke: the operation is
serialized by the disk's block-layer lock, so the call that broke is the one
that waits and re-issues (disk 0 is back, and the operation it was asked went out again on it: it completed). If the window passes first the write fails as
today (Device) and the medium at that block is whatever the stick committed —
as today. The old slot's rings are torn down (Disable Slot) before its pool
block is reused, so nothing of the old command can land later.

A new disk, not the old one. A device that differs is refused by name (the device on port P is not disk N come back: <field>) and bound under a new
number; the old disk stays held and is lost when its window ends.

Tests

usb_transport_break boots seven times now; two are new.

• The move, same stick (Moved::SameStick): the jobcase boot off the boot
  stick with usb-transport-break and a new actuator, usb-reset-moves, which
  holds the port rung's first reset (within the rung's bound) until the port
  reads empty. QEMU cannot move a device on a reset, so the host does it:
  device_del of the stick, then blockdev-add of the same backing file
  (locking: off — the unplugged drive may still hold it) and device_add on
  port 3 with the same serial=. Judged in order: the break, the entry at the
  port rung, the hold, disk 0 left port 1 (…) after this driver reset it; it is held, port 3 connected, the same serial read, disk 0 came back on port 3 … its volume carries on, disk 0 is back, and the operation it was asked went out again on it: it completed, and the job's Rebooting.; Boot: complete;
  nothing failed on disk 0, no budget refusal on disk 0, no did not come back, no disk 1 ready; a clean console. The host's move can land after the
  rung's bound (measured once in four boots): the reset then verifies, the
  stick leaves at the next look and is held by its port's teardown — run 74's
  shape — and the judge accepts it and names which shape it saw; only the first
  shape proves the write that broke is the one re-issued.
• The negative control, another stick (Moved::AnotherStick): the same
  backing under another serial number — everything a second unit of the model
  shares with the first. Judged in order: held, the device on port 3 is not disk 0 come back: its serial number differs, disk 1 ready on slot, disk 0 did not come back within 2000 ms of its port reset; it is offline; never
  disk 0 came back nor disk 0 is back.
• The boot stick now carries serial=TOYOS0BOOTSTICK1: QEMU's default serial
  is built from the port path, so the same stick on another port would read as
  another unit.
• The pcap judge told a rung's SET_CONFIGURATION from an enumeration's by the
  request before it being a GET_DESCRIPTOR; the bind's (and OVMF's) string
  reads broke that. It now asks whether a GET_DESCRIPTOR(Configuration) came
  since the last SET_CONFIGURATION (is_a_rungs_configuration).
• The metal judge (transport_break_on_metal) accepts the moved shape: held,
  as the same device, went out again … it completed, no did not come back, then the DONE chain.

The two checks (device driver; a device's identity across a trust boundary)

Negative controls (mut-whole-usbtransport-7.sh, exits in
mut-exits-usbtransport-7.txt; each under a restoring trap, tree clean after):

arm	what it is	build / test	reds on
WHOLE	kernel/src/drivers/xhci/{device,mod,wait/boot,wait/msc}.rs and toyos-xhci/src/lib.rs restored to the merge base 7f4155b1, identity.rs deleted (635 lines out); only the staging kept (the usb-reset-moves actuator and its 19-line hold) and every test	0 / 1	after the break, no line reads "usb-storage: disk 0 left port 1 (its port read empty) after this driver reset it; it is held " — the first four boots pass, the moved boot reds; the head's kernel prints that line, so the arm's was the one booted
SERIAL	same() ignores the serial number	101 (pure) / 1	identity::tests::a_device_that_differs_in_any_field_is_a_new_disk; the negative control boot: disk 0 came back on port 3 … serial number "TOYOS0OTHERSTICK" … its volume carries on, and no is not disk 0 come back: its serial number differs

Independent oracle: the specifications (xHCI 1.2 §4.19.7, §7.2 and Appendix
D, read; USB 2.0 §9.1.1, §9.6.1, §9.6.7 and BOT 1.0 §4.1.1, recalled; SPC-4
§6.4.2 for INQUIRY's fields, recalled); run 79's record, which is the failure
this round is built from and whose numbers (1.897 s, 1.948 s, 2.878 s) are a
pure test; QEMU's own usb-storage as the device; and the requested T14 boot.

Gates at c819b02c / 59099fd7 (gates-exits-usbtransport-7.txt)

The guest suites ran at 59099fd7; dcf5a4bd is an issue file and c819b02c
two clippy attributes in the pure crate, no behaviour.

command	exit
cargo test --test toyos-build usb_ -- --nightly (12 + 1 quarantined)	1: usb_reset_records_the_phase_it_cut alone, 3 of 4 arm(s) unmet; every other usb_* passed, usb_transport_break among them (26 s); usb_disk_index_stable quarantined and green
… xhci_ -- --nightly (16)	0
… blackbox_ -- --nightly (7)	0
usb_reset_records_the_phase_it_cut on the base 7f4155b1 (tree restored to it, 0 diff lines)	1, the same 3 of 4 arm(s) unmet
log_flush_retry head / base 7f4155b1	0 / 1 (the retried flushes left the log volume breaking the format) — one run each, not a rate; tracked in issues/boot-media/log-flush-retry-reds-two-ways-at-two-in-five.md
cargo test -p toyos-xhci	0 (96)
cargo test -p toyos-build --lib	0
cargo test --workspace --exclude toyos-build	0 (139 test result: ok)
cargo run -- --clippy (the five shapes of src/clippy.rs)	0 at c819b02c (1 at dcf5a4bd: large_enum_variant and type_complexity in identity.rs, fixed by c819b02c)

Unsure of

• Whether the T14's stick publishes a serial that reads the same on both
  halves. SanDisk sticks publish one and Linux shows one serial for the device;
  that the USB2 and USB3 enumerations give the same string is expected, not
  measured. If not, the requested boot reds by name (is not disk 0 come back: its serial number differs) and ends as run 79 did.
• The window against a slow bind. 2 s is twice one measurement. The bind's
  own TEST UNIT READY may take READY_BUDGET (0.5 s) on a stick spinning up.
• The broken call waits only inside its own operation budget. If the stick
  comes back after the write's 2 s block::OPERATION, the write answers
  ask-again, and logd gives up its volume on an append that is refused
  (filed: issues/boot-media/logd-ends-the-boots-log-on-one-refused-create-and-nothing-durable-says-so.md).
• MSC_BLOCKS is 2 per controller: the old port's block is freed only at its
  teardown, so on a controller already serving two disks the returning device
  could be refused for want of a block until that teardown. Not staged.
• A disk held when its window passes gives no second chance: a device that
  comes back at 2.1 s is a new disk, as before this round.
• Not this branch's: why the stick starts on the USB2 half
  (issues/hardware/the-t14s-superspeed-stick-enumerates-at-high-speed-under-toyos.md),
  the FAT long-name orphan, logd's give-up — all filed.
• issues/kernel/a-disk-taken-offline-is-never-brought-back.md now says the
  moved half is held by identity, QEMU-judged and not yet T14-judged; its second
  half (offline for the connection's life) stands.

Requested (t14-request-usbtransport-7.txt): the head image once, sha256
1e9e6a0c714ce2103419d67db2857f843c9c12f42e5d86f8b9605543a60e486b, armed
usb-transport-break alone.

---

Everything below is round 6 and earlier as they were written.

Round 6, head acefbc1592c32510b012921829f6789f7ac0f6d5: a ladder with a bound per rung, and a device owed data is reset at its port first

One commit for the change, b684a622, and two after it: 7b312d6e (the port rung's 50 ms and the offline line's wording), acefbc15 (the whole-change control's needle).

What the bench read (run 77, read out by run 78, 2026-09-21)

Round 5's image, stick on slot 1 / port 1 (the USB2 half, speed 3):

[1.843] transport broke on SCSI 0x2a: a staged break skipped the data phase wait; break 1 of 3 running
[1.843] bulk pair Stopped: endpoint 3 … 0 B unmoved; endpoint 4 with 1 TRB(s) unreached and a stop event of code 26 (Stopped), 4096 B unmoved
[1.843] bulk pair dropped and added …
[3.843] transport broke on the recovery's TEST UNIT READY: no answer in the status phase in 2000 ms; break 2 of 3 running
[3.844] bulk pair Stopped: endpoint 3 with 1 TRB(s) unreached … 13 B unmoved; endpoint 4 with 0 TRB(s) unreached …
[5.843] transport broke on the recovery's TEST UNIT READY: this call's budget after a break ran out during the command phase; break 3 of 3 running
[5.843] Stop Endpoint not issued: this call's budget after a break is spent
[5.843] is offline: both bulk endpoints Stopped=false, port 1 reset=false (hot on a USB2 port, PORTSC 0x00000e03 then 0x00000f91, link Other(12), speed 3), its slot is kept until the unplug

No is a status behind line and no Reset Recovery took. Linux afterwards:
usb 3-1: device descriptor read/64, error -71, device not accepting address, until a replug.

A. What the stick does at a class reset, settled against the records and the texts

Refuted: round 5's account, that the stick keeps the abandoned WRITE's
status and hands it to the next reader. Run 77 read the Bulk-In after the
class reset for 2000 ms and nothing came.

Supported: the stick answers the Bulk-Only reset and both clears and does not
leave the WRITE it was given. BOT 1.0 §3.1 has a device that answers the
reset "ready … for the next CBW", and §5.1 has a CSW echo its own CBW's tag.
Runs 74 and 75 (round 4): the first status after the class reset carried
0x58a, the abandoned WRITE's tag, not the re-issued WRITE's 0x58b. Run 77: the
TEST UNIT READY's 31 bytes were taken (endpoint 4 with 0 TRB(s) unreached)
and no status came for it (endpoint 3 … 13 B unmoved); the second one's 31
bytes were not taken in 2000 ms. A device that had left the WRITE would have
answered 0x58b, and would have answered a TEST UNIT READY.

Refuted: that it takes whatever arrives next on the Bulk-Out as that WRITE's
data, byte for byte, and so that round 4's retry wrote a block whose first 31
bytes are a CBW. The saved volumes say otherwise. t14-run76/sda3-before-flash.img
is the stick after run 75: sector 1080 of the log partition (the write in
question, 11399 × 8 = 90112 + 1080) begins TOYOS-LOG at offset 0 with the
new long-name entry at 0x60, and grep -a -b -o USBC over the partition finds
nothing. The re-issued WRITE's 4096 bytes are on the medium, aligned, under the
abandoned tag. t14-run78/sda3-before-flash.img is the stick after run 77: no
USBC, and sector 1080 without the new entry — the TEST UNIT READY's 31 bytes
were not committed either. What the stick did with those command blocks — took
them as command blocks while still owing the first WRITE's status, or discarded
them — no record says.

What no path of this driver can now do is put bytes on the Bulk-Out of a
device in that state: a break that leaves a device owed Data-Out bytes enters
the ladder at the port reset, and the first thing on its Bulk-Out afterwards is
a TEST UNIT READY that follows a reset and a SET_CONFIGURATION (judged on
QEMU's wire capture, below). A break anywhere else still gets the class reset
and its TEST UNIT READY: 31 bytes, dCBWDataTransferLength 0. At a device that
has wrongly stayed in a Data-Out phase that is less than one logical block of
any size, the next rung resets the port before anything else is sent, and both
saved volumes show this stick committing none of a command block.

Why the port reset first, and not the class reset. The bench has the other
half of the experiment already:
issues/kernel/a-cut-bulk-only-command-does-not-brick-the-benchs-stick.md
records six boots (runs 39, 43 ×3, 45, 47) in which this stick was left holding
a CBW with no data, a queued data phase, an unread status, or a write in
flight, and had its port reset with no class reset before it: it enumerated
0 s after the machine came back, every time. The three boots of this branch
that cost a replug (66, 75, 77) all ran the class reset at a device inside a
command first; the one that did not (74) is the one whose WRITE the stick had
been let finish. That is six to three, on one stick, and it is what Linux's
usb_stor_invoke_transport does too: port reset first, the class reset only if
that fails (recalled, not read this round).

Run 77's bus reset did happen, and did not bring the stick back. PORTSC … then 0x00000f91 has PR set: round 5 wrote the reset with nothing left to wait
on, so nobody saw it finish, acknowledged it, or addressed the device. Unlike
run 74 no port 1 disconnected followed. So fact 4 stands only for a stick
that had been let finish its WRITE; for one that had been given a class reset
and two command blocks inside it, a bus reset was not enough. Whether it is
enough with no class reset before it is what the requested boot asks, and the
six boots above are why it is asked that way.

Texts: BOT 1.0 §3.1, §5.1, §5.3.3–5.3.4, §6.6.1, §6.7.3 and USB 2.0 §7.1.7.5,
§9.1.1, §9.2.6.2, §9.4.5, §9.4.7 are recalled; no copy is on this machine.
xHCI 1.2 §4.3.1 (PRC is the reset's completion), §4.6.5 (Address Device with
BSR clear is defined from Default), §4.6.8–4.6.11, §4.19.5, §5.4.8 (PR, WPR)
were read off scratchpad/r2-xhci-spec.txt this round.

B. The ladder (toyos-xhci/src/ladder.rs, pure; msc.rs takes the steps)

1. Class reset. Round 3's quiesce, Drop-and-Add, Bulk-Only reset and both
   clears, ended by TEST UNIT READY under its own tag. Anything else — another
   tag, a stall, silence, a request not answered — is the next break.
2. Port reset, the most the port has (warm on USB3, hot on USB2), waited
   for on PRC (§4.3.1), its flags consumed as an enumeration's are; 50 ms
   (USB 2.0's 10 ms reset recovery, and the 40 more Linux gives); Reset Device (§4.6.11: slot to Default, address
   0, every endpoint but EP0 Disabled); Address Device with BSR clear (§4.6.5),
   whose SET_ADDRESS is the first thing the reset device answers;
   SET_CONFIGURATION with the bind's value; Configure Endpoint adding the pair
   as the bind described it; TEST UNIT READY. On the slot, the pool block and
   the disk number the device already has, so the caller's command is
   re-issued and a mount carries on: under QEMU the boot stick's own staged
   break comes back this way with the root volume on it. The rung's input
   contexts are a page of the disk's own, not the controller's shared one,
   which an enumeration outstanding on another port may still be read from.
3. Offline: the endpoints stopped (or found Disabled), the port reset and
   waited for, whichever step the device stopped answering at, Reset Device,
   the slot back from the poll. Nothing is sent after the reset. A device that
   left its port is its port's teardown's, as a round trip whose port read gone
   already was.

A rung is climbed once per run of breaks: a rung that verified and whose
command broke again is not asked again, the next break climbs on, and only a
completed round trip ends the run. MAX_TRANSPORT_BREAKS is the number of
rungs.

The bounds (toyos-xhci/src/call.rs). Fact 3's defect was one budget for
everything after the break. There is now one per part — the wait that broke and
re-issued commands 2 s, class reset 0.75 s, port reset 1.5 s, offline 0.5 s —
and every wait is clipped to where the rungs still ahead of it begin, so a
rung is entered with the whole of its own bound whatever was spent before it,
and the call still ends at their sum, 4.75 s, asserted under the 5 s TLB-ack
tripwire the caller's IF-clear spin answers to. A controller that left a
command unanswered is sent no other in the call, in any rung, and its port is
still reset: that is a register write.

Gone with round 5's account: bot::whose and the reading-on past an abandoned
status; and PortState::taken_offline / AfterSlot::Revive, the port's
enumerate-again-once after offline, which the port rung now does in place and
with the disk kept.

C. The end state, whichever rung ends the climb

ends on	what the device was last sent	under QEMU	for the bench
class reset verified	its own status for TEST UNIT READY, then the caller's command	the faults gate: three runs	not reached by the requested boot
port reset verified	the same, after a reset, an address and a configuration	the staged WRITE break on the data disk and on the boot stick; four runs of the faults gate	the port reset took
offline	a port reset, seen to complete, and nothing after it	usb-transport-offline; the faults gate's third break; a bind's; usb-reset-break	port N reset=true, the last thing it was sent
the device left its port	the reset that moved it; the port it arrives on enumerates it	not reachable: QEMU's ports are not paired	nothing is connected, then port 13 connected

What QEMU cannot show is the thing the ruling is about: that this stick's
firmware is left enumerable. Its usb-storage comes back from any reset. The
bench's the boot stick enumerated N s after the machine answered is the
judge, and the requested boot reads it.

Tests

usb_transport_break boots five times.

• The staged WRITE break (data disk): the driver says the device is owed
  data and enters at the port reset; on the wire (QEMU's pcap) no Bulk-Only
  reset and no command block stands between the abandoned WRITE's CBW and the
  rung's SET_CONFIGURATION, and the first block after it is a TEST UNIT READY;
  every block verifies host-side.
• The faults gate: four runs on one disk, each climbing the class reset and
  then the port reset (4 Bulk-Only resets each followed by both clears and a
  TEST UNIT READY; 4 rung SET_CONFIGURATIONs each followed by one), one class
  reset staged out of step, the fourth run offline behind its last reset with
  the slot back while still plugged in; then a disk whose INQUIRY breaks the
  budget over inside its bind climbs the same ladder.
• usb-transport-offline, which now stages run 77's starvation: every
  rung's TEST UNIT READY is unsent and unanswered for the rung's whole bound.
  The port rung really spends 1.5 s (read off the kernel's own timestamps), and
  the last rung must still stop both endpoints, see its reset complete, tell
  the controller and give the slot back, with no not issued line.
• The boot stick's own chain: comes back by the port reset, root mounted,
  and the sealed section carries the port reset took.
• log_flush_retry's hung device (usb-reset-break, now a whole climb that
  answers nothing) goes offline through the port rung.

The two checks (device driver; concurrency with the event ring)

Negative controls, mut-usbtransport-6.sh on the clean head: exact-once
replacements under a restoring trap, each tree shown to build, exits in
mut-exits-usbtransport-6.txt.

arm	what it is	build / test	reds on
WHOLE	kernel/src/drivers/xhci, usb_gate.rs, actuator.rs and toyos-xhci restored to 37d30495 (0 diff lines against it, and 0 against round 5's driver 907fbe37: the driver run 77 measured); this round's tests kept	0 / 1	the driver never said "… is owed the data of the command that broke, so nothing can be asked of it on the Bulk-Out: its port is reset with no class reset before it" (mut2-exits)
ENTRY	a device owed Data-Out bytes enters at the class reset	0 / 101, 0 / 1	ladder::tests::a_device_owed_data_out_is_never_given_the_class_reset; the same line as WHOLE (mut2-exits)
R1-UNVERIFIED	the class reset is believed whatever its TEST UNIT READY came to	0 / 1	the faults gate's out-of-step read refused=true matched=false
R2-SKIPPED	the port rung is not climbed	0 / 1	" was not answered; break ": a rung refused without a device refusing it
R2-UNVERIFIED	the port reset is believed whatever its TEST UNIT READY came to	0 / 1	no transport broke on the port reset's TEST UNIT READY: … break 2 of 3 under usb-transport-offline
R3-NORESET	the last rung takes the disk offline without resetting its port	0 / 1	the last rung's reset was not seen to complete
BUDGET	no rung opens a bound of its own (round 5's one budget)	0 / 101, 0 / 1	four of call's tests; is offline: … Stopped=false, port 2 reset=true …, Reset Device=false, its slot is kept until the unplug — run 77's shape — and the test wants Stopped=true

Independent oracle: the texts above; QEMU's own capture for the wire; the
saved volumes of runs 75 and 77 for what the stick committed; and the bench
records of runs 39–47 and 74–78, which this round is built from and the
requested boot judges it against.

Gates at acefbc1592c32510b012921829f6789f7ac0f6d5, tree clean before and after (gates-exits-b-usbtransport-6.txt)

command	exit
cargo test -p toyos-xhci (90 tests); cargo test in toyos-xhci/sim	0; 0
cargo test -p toyos-blackbox	0
cargo test -p toyos-build --lib	0
cargo test --workspace --exclude toyos-build	0
the five cargo clippy shapes of src/clippy.rs	0 each
usb_transport_break, blackbox_done_chain, blackbox_panic_chain, boot_deadline_ends_a_wedge, hard_lockup_ends_a_deaf_cpu, usb_reset_hands_devices_back, blackbox_foreign_record, blackbox_early_panic_sealed, blackbox_fault_sealed, machine_reboot, usb_storage_gate, usb_storage_shapes, usb_short_read, usb_boot_stick_pulled, xhci_hotplug, usb_disk_index_stable, usb_refused_disk_first, usb_storage_write_error, usb_flush_optional, usb_pool_exhausted, xhci_hid_break, xhci_flap, each -- --nightly	0 each
usb_reset_records_the_phase_it_cut	1, the same 3 of 4 arm(s) unmet as on the base 37d30495 this session (ab-base-exits-usbtransport-6.txt: base 1) and as rounds 4 and 5 measured; #447's test, not on --known-red's list
log_flush_retry	1, and not by this diff: the base 37d30495 this session 3 red of 3 (ab-base-exits), on the format checker and the deadman, in its first two boots, which arm no transport break. Its third boot (the hung device) reaches this round's port rung and goes offline through it in the one run that got there (guest-a-log_flush_retry: the port reset was not answered; break 2 of 3, is offline: … reset=true …). Tracked: issues/boot-media/log-flush-retry-reds-two-ways-at-two-in-five.md

The gates ran twice, at b684a622 (gates-exits-a) and at 7b312d6e (gates-exits-b), the same exits both times; acefbc15 changes one test's needle and was judged by the WHOLE and ENTRY arms above and the test build.

Chosen between, and unsure of

• Entering at the port reset for a device owed Data-Out bytes is a reading of
  nine boots of one stick, not a specification. BOT §5.3.4 makes Reset
  Recovery the host's answer to every error, and a compliant device honours it
  anywhere. The texts permit a port reset at any time, so it costs a compliant
  device an enumeration and nothing else.
• A bus reset on the USB2 half may move this stick to port 13 (run 74).
  Then the rung reads the port empty, the disk is its teardown's, and the stick
  arrives under a new number: alive and enumerable, which is the ruling, and
  with the root volume lost, which is
  issues/kernel/a-disk-taken-offline-is-never-brought-back.md.
• Whether a warm reset brings this stick back on the USB3 half is not
  measured. Run 75's was hot, after a class reset.
• Offline leaves the device in Default on an enabled port and does not disable
  the port: §4.19.5 asks for one or the other "immediately", portsc::Write
  offers no PED by design, and nothing measured says which the stick prefers.
• The bounds are chosen, not measured on hardware: 56 ms is the one bus reset
  the bench timed (run 74).
• Filed: issues/hardware/the-t14s-superspeed-stick-enumerates-at-high-speed-under-toyos.md
  (fact 5, with the readings, not chased).
• .claude/agents/implementer.md is not on main; read from the job scratchpad.

Requested then (t14-request-usbtransport-6.txt, booted as run 79): the head image once, sha256
77fbcc71fcce8526a47c76f9d1477986bb5f7d672193e7555019d8052082a25a.

---

Everything below is rounds 1 to 4 as they were written. Where it says a
recovery whose every step answered says Reset Recovery took, that a break is
always answered with the class reset, that the give-up writes PORTSC.PR on
one budget with the recovery, and that a disk given up on is left with its port
believing something is attached, round 6 above is what the branch now does.

What the record says

Run 55's page, as the next boot's loader printed it (the page had dropped its
227 oldest records, the first break among them):

[3.471 cpu5] usb-storage: 00:14.0 slot 5 transport broke on SCSI 0x28: status phase completion code 3
[3.471 cpu5] xHCI: 00:14.0 slot 5 endpoint 3 is Halted, recovering
[3.471 cpu5] xHCI: 00:14.0 slot 5 endpoint 4 is Running, recovering
[3.478 cpu5] usb-storage: 00:14.0 slot 5 transport broke on SCSI 0x28: command phase completion code 4
[3.478 cpu5] xHCI: 00:14.0 slot 5 endpoint 3 is Stopped, recovering
[3.478 cpu5] xHCI: 00:14.0 slot 5 endpoint 4 is Halted, recovering
…
[5.556 cpu0] xHCI: Set TR Dequeue timed out
[5.556 cpu0] usb-storage: 00:14.0 slot 5 reset recovery failed; disk is offline

22 transport broke lines in the 85 ms from 3.471 s to 3.556 s, then a command
the controller never answered. Code 3 is Babble Detected Error and 4 is USB
Transaction Error (xHCI 1.2 Table 6-90); endpoint 3 is the Bulk-In, 4 the
Bulk-Out. Run 24 shows how one starts: no answer in the status phase in 2000 ms with both endpoints Running — a CSW two seconds late, abandoned.

Why the two codes alternate is not established. The first round's body said
each recovery reset one pipe at both ends and the other at the host only. That
is not what the base did: the pipe that was not halted got Stop Endpoint and
Set TR Dequeue and no ClearFeature, and neither command zeroes a toggle or a
sequence number (§4.6.9, §4.6.10), so both ends of that pipe were left
agreeing. Nothing in either record reads a sequence number or the device's
state. What the record does establish is below.

Where the base was wrong, against the texts

1. Reset Recovery cleared the halt on one pipe. BOT 1.0 §5.3.4 is a
   Bulk-Only Mass Storage Reset, then ClearFeature(ENDPOINT_HALT) on the
   Bulk-In, then on the Bulk-Out — not conditional on a halt. The base sent a
   clear only for the pipe its controller reported Halted. And a clear cannot
   be sent alone: it reinitialises the device's data toggle whether the endpoint
   was halted or not (USB 2.0 §9.4.5), so the host's own has to be zeroed to
   match — Reset Endpoint does that from Halted (xHCI 1.2 §4.6.8), and a
   Configure Endpoint with Drop and Add set does it from Stopped (§4.6.8's
   note).
2. The break count was per call, and the give-up left the disk online.
   Three breaks returned Scsi::Broken → BlockError::Device, which
   file_backing::read_block_retrying returns at once (it asks again on
   BudgetExpired alone). The repeats in the record — root: read of block 3 failed at 3.502, 3.517, 3.525 and 3.540 s — are new callers, each given three fresh attempts at a
   disk nothing had marked failed.
3. The give-up left the device where the last break found it: inside a
   command, one endpoint halted. Nothing past §5.3.4 was taken; the bus reset
   came with the platform reset at 252 s, and by the first round's account of
   runs 55 and 56 the stick then needed a physical replug before it answered.

The change

The recovery (toyos-xhci/src/reset_recovery.rs, pure; msc.rs takes the
steps). Both endpoints to Stopped — Reset Endpoint from Halted, Stop Endpoint
from Running, Set TR Dequeue Pointer from Error (§4.6.10), nothing from
Stopped — then one Configure Endpoint dropping and adding both on fresh rings,
then the class's three requests.

An event outranks the Endpoint State field. §4.8.3's note has the field's
update deferred past an error and past a command's completion, and tells
software to keep its own image from events. Quiescing is that image for one
quiesce: the transfer event that ended the round trip is where a pipe starts
(Stall, Babble, USB Transaction and Split Transaction Error are Halted, TRB
Error is Error — Broke::Code carries its pipe so scsi can hand it over); a
command that succeeded leaves its pipe Stopped whatever the field goes on
reading; and a Context State Error rules out, for that pipe, the state the
command was chosen from. A field that moved is believed; one that still reads
the ruled-out state is taken to lag, and the plan goes on to Halted and then
Error, the two states §4.6.9's note says a Running endpoint reaches by itself.
Round 2 re-read the same field with no delay: on a controller that defers the
write it spent its five looks in microseconds and took a healthy disk offline.
Seven looks at most now — three commands per pipe and the look that finds both
Stopped — asserted to be the worst case and not a margin, over all sixteen
pairs, every move under the look, with a field that is current and one that is
never written again. What a Context State Error means, what any other code
means and what silence means are the crate's decisions; the driver's loop
issues what it is told and ends where the plan ends.

The input context is the pure crate's (toyos-xhci/src/configure.rs). Every
dword the Configure Endpoint reads — Drop flags, Add flags, slot context, both
endpoint contexts — is decided by configure::bulk_pair and handed to the
driver as a list it writes without looking inside, so the driver has no flag
left to omit. The recovery's slot context is the specification's: §6.2.2.2
asks for Context Entries and the Hub fields, and Table 6-7 has USB Device
Address and Slot State initialised to 0 as input. Round 2 copied all four
dwords out of the output context, both of those non-zero, and cited §6.2.2.2
for it; the copy and the sentence are deleted.

One budget for the whole call (toyos-xhci/src/call.rs,
CALL_AFTER_BREAK). Round 2 sent a controller that had left one command
unanswered another from take_offline and then waited on a port reset: up to
four 2 s spins in one call, under XHCI with preemption off, against a TLB-ack
tripwire of 5 s. The budget opens at the start of the bulk wait that broke and
is 4 s, twice the driver's one timeout. Every later wait of the call is clipped
to it; a step reached with nothing left is not taken and says so; a command
that got no answer ends further commands to that controller in that call, by
name; a command is not re-issued into a spent budget. A const assertion holds
the budget under tlb::ACK_TIMEOUT. Measured by hand on af1dd433 plus an
uncommitted probe patch (one gate read with the bulk endpoint's doorbell and
the command ring's both withheld for the whole call; TCG, dev host): read began
0.200 s, break 1 of 3 at 2.200 s, Stop Endpoint timed out after 1999 ms at
4.200 s, Stop Endpoint not issued: the controller left an earlier command of this call unanswered, offline with Stopped=false … its slot is kept until the unplug, read returned 4.201 s — 4.001 s, and 4.002 s on the harness's
re-run. The probe's first run, one commit earlier, panicked the kernel on a
wrapped subtraction in the settle; af1dd433 fixes it and moves the arithmetic
into the crate with its test.
Round 2's hand probe, quoted in 2527bfff's message, ran on the uncommitted
tree before that commit (a 96-line dirty diff in wait/mod.rs), which that
message does not say and history is not rewritten to add.

Take-offline is one path, inside a bind too. Both endpoints to Stopped
first, because Disable Slot is defined only over endpoints that are Stopped or
idle (§4.6.4's note); then PORTSC.PR (§4.19.5); then the slot, and where it
goes is one decision (reset_recovery::slot_after_offline). A bound disk's goes
back from the poll. bind used to answer false and device::refuse issued
Disable Slot whatever the quiesce had said; it now answers
Bind::Refused(SlotGoes), and a pair that could not be Stopped leaves its slot
with the port, whose teardown gives it back. A TEST UNIT READY whose recovery
fails goes offline by the same path instead of falling into refuse.

The INQUIRY faults are staged by the bind that takes them, one more than the
budget, so the bind's own disarm has one to take back: the test asserts
bound under 4 staged INQUIRY fault(s): untaken=1.

The log tells what happened. A completed round trip that ends a run of
breaks says so where the count is cleared; a recovery whose every step answered
says Reset Recovery took; the reconfigure prints the two states the
controller's output context holds after it; a command's timeout prints the
milliseconds it really waited.

Unchanged on purpose: the single-pipe stall paths (BOT §6.7.2, §5.3.3).

Wiring that no committed test judges, only reading: that scsi passes
broke.event() and not None; that device::bind acts on
SlotGoes::WithTheUnplug; that the two break sites open the budget. The
decisions they feed are tested in toyos-xhci. QEMU models no lagging field,
no silent controller and no quiesce that fails; the hand probe is the one run
that reached the last two.

What the tests judge, and what QEMU cannot

usb_transport_break boots three times; the third is round 4's, the T14's own
boot under QEMU — jobcase off the boot stick with the arm, judged by the metal
judge and then on the page's recovery section. The second boot's gate stages, on a disk
that answers every well-formed command: two breaks (one short of the budget,
which the test reads off the gate's own line) by a bad CBW signature, which
QEMU's usb-storage answers with a STALL of the Bulk-Out — Bulk-In Running,
Bulk-Out Halted; two by a withheld CBW, answered with a STALL of the Bulk-In —
the pair the other way round; then three, the give-up; then three on the
INQUIRY of a disk plugged in over QMP, the give-up inside a bind. The first
boot gains a read whose port reads gone, asserting the Quiet::Gone arm: the
line, and no command, reset or offline after it.

• The three requests are judged on the wire. BootOptions::usb_pcap has
  QEMU capture the disk's own traffic, and the test requires every Bulk-Only
  reset in it to be followed, with nothing between, by a clear to the IN
  endpoint and then the OUT one: 6 resets, each with both.
• The Drop and Add reach no wire, and QEMU cannot tell them from an Add
  alone: it re-creates an endpoint on an Add whatever the Drop says. Their
  judge is the pure crate, which decides every dword of the input context and
  whose tests read the flags. The output context calling both endpoints Running
  with no doorbell rung is still asserted, and says only that an Add happened;
  the Bulk-In found Running and not Stopped at the next break is what the
  partial fix (M2) reds on.
• QEMU models no data toggle, no sequence number and no device a phase ahead
  of its host. That the clears resynchronise a device, and that a real stick
  comes back at all, is judged nowhere in QEMU. BOT §6.6.1's mandatory answer
  to an invalid CBW is a STALL of the Bulk-In; QEMU does only the optional
  Bulk-Out half. A Stopped pair at a break is not reachable in this driver
  without a lying look (every recovery leaves both Running, every break follows
  a doorbell) and is covered by the pure model alone. The T14 boot below is
  the only judge of the device.

The two checks

Negative control — what was run. On the clean head a9adf0d7, a guarded
script (refuses a dirty tree or a moved head; applies under a trap that
restores; every replacement must match exactly once and the diff is asserted
non-empty; proves git status --porcelain empty after) applied each mutation
and ran the build, then the judge. QEMU arms: cargo test --test toyos-build --no-run, then usb_transport_break -- --nightly; this harness builds the
kernel under --quiet, so Compiling kernel is not printed, and the witness of
the kernel's own rebuild is its artifact's mtime against a marker touched after
the mutation (1 in every QEMU arm), with no cargo build failed line. Pure
arms: cargo test -p toyos-xhci --no-run, then cargo test -p toyos-xhci.

arm	what it is	build / test	reds on
green	the head	0 / 0	—
M1	the count never clears	0 / 1	2 withheld CBWs …: read refused=true matched=false untaken=1 healthy=false
M2	the partial fix: the base's per-endpoint recovery, count and offline path kept	0 / 1	break 2 of 3 running left the bulk pair ["Stopped", "Halted"]
M2B	only the clears made conditional on a halt	0 / 1	Bulk-Only reset 1 was followed by clears of [Some(02), None] — the wire
G1	give_back_offline_slots never called	0 / 1	"xHCI: slot 2 disabled" at Some(26078) and "xHCI: port 2 disconnected" at Some(26032)
R1	the recovery's Drop flags never written	0 / 101	configure::a_recovery_drops_and_adds_both_endpoints
C1	a Context State Error ends the quiesce like any refusal	0 / 101	four of reset_recovery's tests, a_context_state_error_is_followed_by_the_command_for_where_the_pipe_went first
R3	the slot context carries USB Device Address and Slot State as an output context holds them	0 / 101	configure::a_recoverys_slot_context_is_context_entries_and_zeroes
R3B	the slot context without Context Entries	0 / 101	that test and context_entries_is_the_higher_index_of_the_pair
EV	the transfer event's code is not taken; the plan reads the field	0 / 101	a_halting_completion_code_outranks_a_field_that_still_reads_running
LAG	a field that still reads the ruled-out state is believed again (round 2's rule)	0 / 101	three tests, the exhaustive walk among them
B1	a wait after a break is not clipped to the budget	0 / 101	three of call's tests
B2	a silent controller is sent the next command all the same	0 / 101	two of call's tests
S5	a slot goes back whatever the quiesce came to	0 / 101	a_slot_goes_back_only_over_a_stopped_pair
U6	the bind prints untaken=0 and never disarms	0 / 1	… bound under 4 staged INQUIRY fault(s): untaken=0, want untaken=1

G1, R1, C1 and R3 are the four the second review found surviving on
502400ba, each build 0 / test 0 there. U6 survived this round's first head
(6555dca5, 0 / 0) and is why the bind is asked for one fault more than it
takes. The whole-change revert (round 2's NC) was not re-run: it is a
hand-written behavioural revert, not a revert of the whole change, and the
review said what it keeps.

Independent oracle. The specification, read and not recalled: xHCI 1.2
§6.2.2.2 and Table 6-7 (the slot context), §4.8.3's note and halt list and
§4.6.9's note (the image and the inference), §4.6.11 (the issue's missing
rung), off the copy the second review fetched; the pure crate's controller
model is written from §4.8.3's state machine and not from the driver. And
QEMU's own capture of the disk's traffic, a second implementation's record of
the wire, for the three requests. Real hardware is requested below and has not
run.

Gates, each with its own command's exit

Dev host, TCG, head a9adf0d7, tree clean before and after.

command	exit
cargo test in toyos-xhci/, and cargo test -p toyos-xhci	0 (79), 0
cargo test --lib	101 — redlist::tests::every_row_can_say_what_it_claims: four known-red rows measured 2026-08-20/21 are "more than 31 days ago" as of 2026-09-21. src/redlist.rs is byte-identical to origin/main's; 310 others pass. Main's, by the calendar; not adjudicated here.
cargo test --workspace --exclude toyos-build	0 (139 test result: ok, 0 FAILED)
the five cargo clippy shapes of src/clippy.rs, run directly	0 each, 0 warnings
cargo test --test toyos-build usb_transport_break -- --nightly	0
usb_storage_gate, usb_short_read, usb_boot_stick_pulled, xhci_hotplug, usb_reset_hands_devices_back, usb_disk_index_stable, usb_refused_disk_first, usb_storage_write_error, usb_flush_optional, usb_pool_exhausted, xhci_hid_break, xhci_flap, each -- --nightly	0 each

CI. Run 35360971260 on 502400ba: guest (2) and guest-suite red on
blocked_dump — PANIC: … src/sched/dump.rs:134:5: the blocked-task dump ran under a lock: preempt depth 2, ALONE: GREEN. That is main's scheduler defect
(issues/kernel/the-blocked-task-dump-panics-when-a-cpu-is-inside-inbox-submit.md),
being fixed on #471; this branch touches nothing under kernel/src/sched.
cargo run -- --known-red blocked_dump says NOT KNOWN-RED, so the pull request
cannot enter the queue until #471 lands. No CI run exists for a9adf0d7 as this
is written.

log_flush_retry arms usb-transport-break with usb-reset-break, so it
runs this path, and is red at a rate on both trees. As the second review took
it, same host and session: 502400ba 1,1,0,1,1,0,0,0 — 4 red of 8;
78929f12 detached 1,1,1,1,1 — 5 of 5. The branch does not raise the rate;
n is too small to say it lowers it. One of the head's reds is a shape in none of
the base's five runs and named by no issue: test-runner SEGFAULT, execute unmapped address at _start+0x0, [qemu] Init process crashed during boot, no
transport line in the excerpt — n=1, unattributed. Not re-measured at
a9adf0d7. --known-red log_flush_retry: NOT ON THE LIST. Nightly tier.

The price. usb_transport_break boots three times; src/tiers.rs' ci_ms: 5_382
and tests/test-durations' 5408 ms are the hosted price of its first boot
alone, and the row says so. No hosted run has priced them all: the name
is Nightly (Why::TimerAnchored), the pull-request lanes do not run it, and this
PR has no run of its own to take a figure from. Locally the three boots read
PASS usb_transport_break (9s) in round 4's gate run; round 3 measured its two
at 16.9 s, and the difference is not explained here.

Round 4, head 95a090a2

Dev host, TCG; exits in all-exits-usbtransport-4.txt, mut2-exits-usbtransport-4.txt
and ab-*-exits-usbtransport-4.txt. Everything but the last two rows was run at
4e39301a; 95a090a2 changes one judge in tests/common/power.rs, and the
rows it can move were run again on it.

command	exit
cargo test -p toyos-blackbox	0 (33)
cargo test -p toyos-xhci	0
cargo test -p toyos-build --lib	0, and 0 again at 95a090a2
cargo test --workspace --exclude toyos-build	0
the five cargo clippy shapes of src/clippy.rs	0 each; the root shape 0 again at 95a090a2
usb_transport_break (three boots now), blackbox_done_chain, blackbox_panic_chain, boot_deadline_ends_a_wedge, hard_lockup_ends_a_deaf_cpu, usb_reset_hands_devices_back, blackbox_foreign_record, blackbox_fault_sealed, machine_reboot, usb_storage_gate, usb_storage_shapes, usb_short_read, usb_boot_stick_pulled, xhci_hotplug, usb_disk_index_stable, usb_refused_disk_first, usb_storage_write_error, usb_flush_optional, usb_pool_exhausted, xhci_hid_break, xhci_flap, each -- --nightly	0 each
blackbox_early_panic_sealed	1 at 4e39301a, by this round's diff: its judge took the line under the report's head for the tail's first record, and it is the recovery section's now. 95a090a2 fixes the judge; 0 there.
usb_reset_records_the_phase_it_cut	1, and not by this branch: the same three arms unmet with the same words ("usb-wedge: stopping every CPU at the" never reached the boot after the reset) on this round's base a54ce47f (2 of 2) and on #447's own head 78929f12 (1 of 1), each measured by restoring that tree into this worktree. The name does not exist on origin/main: it is #447's test, and #447 is open. --known-red says NOT ON THE LIST.

Negative controls, each a checked patch applied and restored by one script,
each tree shown to build:

mutation	judged by	exit
a break is any USB record (says_a_break loses its second clause)	cargo test -p toyos-blackbox	101, four tests
the newest that fit are kept, not the oldest	cargo test -p toyos-blackbox	101, the storm test
the kernel's walk sees no record (to clamped to 0)	usb_transport_break	1: no line of the page's recovery section says "transport broke on SCSI 0x2a…"
the section writes no line where nothing broke	blackbox_done_chain, boot_deadline_ends_a_wedge	1, 1

A first attempt at the third and fourth removed the kernel's call sites
instead and did not build (append_recovery is never used), so its three reds
measured nothing and are not counted.

Hardware

The T14 disagrees with QEMU, and this does not land on what it said. Three
boots of the usb-transport-break arm on the ThinkPad T14 (SanDisk Ultra
0781:5581 on the Intel xHC at 00:14.0), 2026-09-21, the images round 3 staged
from a9adf0d7. Each is one boot; none is a rate.

run	driver	what happened
65	base (78929f12), control ba034430…	usb-storage: read of 1 blocks at 10147 failed on disk 0, root: read of block 1187 failed, test-runner died at 3.68 s, the machine sat until the boot deadline ended it at 120117 ms of a 120000 ms bound. It answered ssh after 181 s and the stick enumerated at once. toyos-fat32-check then found the log partition corrupt: a long-name entry carrying checksum 0xAC of a short name whose checksum is 0xB0.
66	this branch, efc536e8…	The machine answered ssh only after 254 s and the stick was wedged for its next host: Linux enumerated 0781:5581, usb-storage bound, no SCSI command was answered, reset SuperSpeed USB device 21 s later, no /dev/sda until a physical replug. The base never did that under this arm. What the driver did is not known: the stick held a 29-line loader log and no kernel log, and the black box was cleared by the next image flashed, because a record belongs to the image that armed it.
69	this branch, the same image again	Clean. The job ran, Syncing filesystems..., Rebooting. at 5.808 s, DONE, back in 63 s, the stick fine, every port reset and unpowered in the reset's account. But the sealed record says /log holds this boot to 0 ms and 302 record(s) committed after that reached no volume, the stick carries no kernel log file at all, and the 16-record tail holds no transport line.

So on real silicon the new driver is intermittent: one clean survival the base
did not manage in its one boot, and one wedged stick the base did not cause in
its one boot. The owner's standing ruling is that ToyOS never leaves a USB
device in that state, so the branch stays unarmed. Three review rounds under
QEMU found neither outcome: its usb-storage has no flash to be inside when a
reset reaches it, and the QEMU boots of this arm that were looked at recovered in one to two
milliseconds.

Nothing about the wedge is explained yet, and nothing in the driver was changed
on a guess. This round is the instrument.

The instrument. Every sealed record — DONE, WEDGED, PANIC — now carries a
section of its own: every usb-storage: and xHCI: record from the boot's
first transport break on, oldest first, at most 4096 bytes, under a line that
counts what was kept of what the ring held; one line where no transport broke.
It is above the wedge's tail, so the census lines that filled the control's
page (186 records kept from 3.679 s on, 249 older ones dropped, no line of the
break among the kept) cannot push it off, and it never spends the 2048 bytes
kept for the reset's own account. Selection and placement are
toyos-blackbox's and host-tested; the kernel walks the ring for it with the
lock-free reader the panic path already uses. It lives outside the driver's
four paths, so the control image carries it too. One measuring line in the
driver: once a quiesce finds both pipes Stopped it says how many TRBs the
controller had not reached on each ring, which tells an abandoned transfer that
completed from one that was cut.

A smaller change would not have done it. Raising the tail's 16 records does not
help a wedge, whose tail is 110 s of census lines, and selecting out of the
rendered tail reads a 32 KiB window of the newest records that the same census
lines fill. This round's first commit message says the break was among run
65's 249 dropped records; what is known is only that it is not among the kept.

Run 69's missing log. The arm abandons the boot's first WRITE(10), and that
write is logd creating its file: under QEMU logd is spawned at 0.301 s, the
break is at 0.323 s and logd's first line comes after it. logd ends its
volume for the whole boot on one refused create, with no retry, and says so
only on its console, which the T14 does not have. In run 69 the disk stayed
online to the shutdown (a SYNCHRONIZE CACHE round trip answered at 5.808 s), so
the create was answered with an error by a disk that was still there; which
error is not known, and the section will say. Filed:
issues/boot-media/logd-ends-the-boots-log-on-one-refused-create-and-nothing-durable-says-so.md.
It is logd's policy and outside this branch's fence.

Run 66, as hypotheses. 254 s is the 120 s deadline, about 74 s, and the 60 s
runs 65 and 69 spent outside the kernel: the stick was already wedged when the
firmware went to load the loader off it. Ranked, each with the record that
would confirm it:

1. The class reset reaches a stick holding a cut data phase. On silicon the
   quiesce's Stop Endpoint can land inside the abandoned transfer; the one QEMU
   boot looked at read 0 TRBs unreached on both pipes. Confirmed by endpoint 4 is Stopped with 1 TRB(s) … not reached in a wedged boot against 0 TRB(s) in a clean one.
2. The 4 s call budget cuts a wait inside the recovery, leaving a control
   transfer abandoned on EP0, the later requests unsent and the port reset
   written with no settle. Confirmed by timed out after <N> ms with N under
   2000, would not take a Bulk-Only Reset, not issued: this call's budget after a break is spent, or an offline line reading Stopped=false or
   reset=false.
3. The recovery is sound and the wedge is made at the reset, with a command
   still open. Ranked last: six staged cuts of that kind (runs 39–47) never cost
   this stick. Confirmed by a section ending in completed after and
   usb-quiesce: a Bulk-Only command was open in its … phase.

Requested (t14-request-usbtransport-4.txt): an instrumented control and an
instrumented head, staged by one script from 4e39301a (the head, 95a090a2, differs from it by one judge in tests/common/power.rs).

	sha256	size
control	47e3783ca63ba5c29b61b04ac09387e929070b4c616407f8cede443b00ef54a9	82837504
head	0c7cab1fded271e317889b2bbc58ef3d38f3a27750b7aef14eb279150d90e943	82837504

Control twice, then the head up to five times, stopping once one wedged-stick
record and one clean record have both been read. A wedged stick's record is
still in DRAM: the owner replugs the stick and the same image is flashed again,
whose first loader pass prints the record and boots no kernel. Any other image
clears it.

What no boot of this arm can show is still the recorded failure: run 55's CSW
two seconds late, then Babble and USB Transaction Error alternating.

Q2 — why the first boot after a replug passes and later ones fail

The reboot path leaves the xHCI clean (stop.rs: PR on every connected port,
HCRST, PP clear, bus mastering off). The stick is what carries state: the
T14's ports never lose VBUS. On the record, a boot that hung left its stick
inside a command with one pipe halted from 5.556 s until the platform reset at
252 s, and the stick then needed a replug. That a bus reset did not clear what
that stick was left in is what the replug showed; what state that was is not
measured.

Q3 — the deadline that fired 132 s late

#462's file attributes the lateness to cpu0's recovery loop (timer=9 xhci=1729). That census is from 3.47 s; the loop ended at 5.556 s. My reading:
with logd, netd and the runner dead and init parked, every CPU planned
TimerPlan::Stop and halted, and deadline::poll runs from the timer entry
only — so the bound fired on the first interrupt anything raised after 120 s.
Outside this change's path, not fixed here.

Filed, not fixed

• issues/kernel/a-stick-that-answers-late-is-broken-by-the-two-second-abandon.md
  — the first break in both records is a CSW two seconds late. It quotes its
  evidence now, and states what it infers as inference.
• issues/kernel/a-disk-taken-offline-is-never-brought-back.md — the rung past
  offline, a device reset that keeps the disk.
• issues/boot-media/logd-ends-the-boots-log-on-one-refused-create-and-nothing-durable-says-so.md
  — run 69's missing log.
• issues/build/the-metal-loop-writes-the-readback-volume-into-a-directory-it-has-not-made.md
  — run 69's readback, lost after the boot was spent.

Fence: kernel, toyos-xhci, toyos-blackbox, the harness, the metal profile
and four issue files. No file under toyos-abi/, toyos/ or userland/libc/.

🤖 Generated with Claude Code

https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK

[Japabu]

Review at 8dd2994c, whole branch, against origin/main 909f34b4. Logs are usb466-review-* in the job scratchpad.

Readiness. PR CI at 8dd2994c: 20 passed, 3 skipped, none failed. At this head: cargo test -p toyos-xhci exited 0 (96 passed). usb_transport_break -- --nightly exited 0 (all seven boots, moved stick and another stick included). T14 runs 80, 81 and 82 read as the brief says, and run 82's sha256 matches the round-8 request (3b4a5bdc…84697). There is no bench reading of 8dd2994c itself. That head adds nothing to the driver beyond main's merge.

Earlier BLOCKERs. The round-2 review is not on the PR. Its four surviving mutations (G1, R1, C1, R3) are recorded as killed at a9adf0d7 in the body. I did not measure them again.

usb_reset_records_the_phase_it_cut at this head: still red. It exited 1 with 3 of 4 arm(s) unmet (usb-wedge-data-owed: "usb-wedge: stopping every CPU at the" never reached the boot after the reset), and it was red again when run alone. cargo run -- --known-red usb_reset_records_the_phase_it_cut says NO, not quarantined. The implementer's round-4 A/B (ab-447-exits-usbtransport-4.txt) has it red on #447's own tree 78929f12. #447 is open and not on main.

Mutations (usb466-review-mut.sh: each anchor matched exactly once, the diff was non-empty, the kernel artifact was newer than the mutation, and the tree was clean after every arm):

arm	build / test	result
LATECAP: Late::gives_up never gives up past the deadline (read_past == usize::MAX)	built / usb_transport_break 0, xhci_hotplug 0	survives
SERIALSHORT: read_serial passes the whole zero-padded 255-byte buffer, not &arrived[..delivered]	built / usb_transport_break 0	survives
HEREBACK: wait_for_return answers Back for any disk found here	built / 1 (boot timed out, 287k-line console)	killed
NOCALLHOLD: with_storage never holds a disk whose port read empty	built / 1 (no line reads "… disk 0 is back …")	killed
HOLDALL2: every disk that leaves is held, reset or not	built / usb_boot_stick_pulled 0, xhci_hotplug 1	killed
(HOLDALL, first spelling, did not build: returns_by never used. Not counted.)

BLOCKER

• kernel/src/drivers/xhci/wait/msc.rs:2265,2314: served/wait_for_return run outside the bound CALL_AFTER_BREAK < tlb::ACK_TIMEOUT puts on a disk call (kernel/src/drivers/xhci/mod.rs:319). The caller's CPU has IF clear on the syscall path (FMASK masks IF). On that CPU the wait (1) runs recheck_ports, and with it the whole bind of whatever device arrives: bring_up's TEST UNIT READY loop, the INQUIRY and READ CAPACITY ladders, and two string-descriptor control transfers. That is several seconds from a device that answers slowly or breaks, and any device plugged in during a hold qualifies. (2) After Back it re-issues the operation, which can open a second full 4.75 s AfterBreak up to 2 s after the operation began. Either can hold IF clear past 5 s, and a concurrent shootdown then panics another CPU on a device's behaviour. This is derived from the code and its constants, not staged. The QEMU moved-stick boot binds the returning stick on the same CPU as the waiting write (cpu0 throughout, usb466-review-mut-NOCALLHOLD-usb_transport_break.log:326-338). The bench bound it on cpu0 while the write waited on cpu4.
• kernel/src/drivers/xhci/mod.rs:1301,1335: adoption does not tell a stick that moved from a stick that was unplugged and plugged back in, a power cycle. The branch's own Moved::SameStick boot is exactly device_del + device_add, and it is adopted with its volume carries on. A device with a volatile write cache loses writes it acknowledged before it lost power. The adopted disk's next SYNCHRONIZE CACHE then returns Ok on the new instance, so a filesystem is told those writes are durable. The ruling is met without this: adopt only a held disk that owed no flush (no_write_cache, or no write acknowledged since its last successful flush). Otherwise make the adopted disk's first flush fail by name.
• kernel/src/drivers/xhci/wait/mod.rs:153-176: nothing tests the round-8 claim that a wait gives up after one ring's worth of reads past its deadline. LATECAP (no cap at all) builds and passes usb_transport_break and xhci_hotplug. This is a wait that can spin with IF clear on high-risk code. Late is a pure decision that sits in the kernel. Move it into toyos-xhci with a test that a ring which keeps producing is still given up on.
• tests/common/power.rs (usb_reset_records_the_phase_it_cut, which reaches main with this merge because it carries A reset records the Bulk-Only phase a device was left in, and the black box keeps that account readable #447): landing this PR puts onto main a nightly test that has been red at every head measured (78929f12, 7f4155b1, cd0a6b1a, 8dd2994c) and is not on --known-red. Before landing, fix it or put it on the redlist with an issue, at its owner (A reset records the Bulk-Only phase a device was left in, and the black box keeps that account readable #447) or here.

NOTE

• kernel/src/drivers/xhci/wait/msc.rs:2210: SERIALSHORT survives. No test covers the driver slicing the descriptor to what was delivered, so the claim that a serial read short is never adopted rests on the pure crate alone.
• kernel/src/drivers/xhci/wait/msc.rs:271: Broke::left treats a command-phase break of a Data-Out command as Elsewhere. A CBW whose handshake was lost after the device took it leaves the device owed data, and the class rung's TEST UNIT READY is then 31 bytes on its Bulk-Out, which is what ladder.rs says is never sent.
• kernel/src/drivers/xhci/wait/msc.rs:2307: on the first look, a disk that was adopted between with_disk returning and whereabouts reads Here with awaited == false, so the operation fails Device although the disk is back. Narrow; not staged.
• kernel/src/drivers/xhci/wait/msc.rs:1991: a returning stick whose bind is refused (READY_BUDGET 0.5 s, or no free block of MSC_BLOCKS while the old block waits on its teardown) leaves its port Refused and never enumerated again. That is a physical replug, the case the ruling forbids. It predates the branch, and the branch now makes it reachable after its own reset.
• In-place port-rung recovery (Reset Device → Address Device → SET_CONFIGURATION → TEST UNIT READY on the same slot) has never been seen to work on the T14. Runs 79–82 all read nothing is connected, so only the move path has a hardware reading.
• QEMU reuses the slot and the pool block across the move (slot 1, msc_block +0x10000 both times), so the old and new TRB addresses are identical. Stale-event safety rests on the old slot's Disable Slot completing before the new Enable Slot, which the command ring orders. No test states it.

REMOVE

• PR body, Round 7: "So no operation pins its CPU longer than round 6's bounds already allowed". False; see the first BLOCKER.
• PR body, top and Round 8: "Round 8 … is not yet booted there" and "Head image staged and requested". Stale: run 82 booted it.
• PR body, Round 6/7 gates: "not this diff, A reset records the Bulk-Only phase a device was left in, and the black box keeps that account readable #447's test". This PR is the merge that lands A reset records the Bulk-Only phase a device was left in, and the black box keeps that account readable #447's test.
• toyos-xhci/src/identity.rs:39-41: run 79's numbers as the constant's provenance. They belong in the commit.
• toyos-xhci/src/ladder.rs:144,233,258 and toyos-xhci/src/call.rs:194: T14 run NN citations in source. They are investigation history.

SEND BACK

[Japabu]

Round 2 review at ea1a44a6. It covers round 1's BLOCKERs and the delta 8dd2994c..ea1a44a6, which contains no merge. Logs are usb466-review2-* in the job scratchpad. The mutation script is usb466-review2-mut.sh. Each arm's anchors matched exactly once and the diff was non-empty. Every counted arm built, and every kernel arm rebuilt the kernel. The tree was clean after every arm. One arm, WEDGEGONE, was left applied when its runner was killed. It was restored by hand and run again under the trap.

Readiness. PR CI at ea1a44a6: 20 passed, 3 skipped, none failed. At this head, cargo test -p toyos-xhci exited 0. usb_transport_break --nightly exited 0, with SameStick bound on cpu1 while the call held on cpu0, and SlowStick's call ending 2.000 s after the break. usb_reset_records_the_phase_it_cut --nightly exited 0 (61.8 s). T14 run 83's image sha matches d777c6ae…. On the T14, the break came at 1.646 on cpu4, the stick was held at 1.764 and came back as the same device, bound on cpu0/cpu7. The re-issued write completed at 2.798 on cpu4.

Round 1 BLOCKERs

• B1: OPEN, narrowed to the re-issue. The bind no longer runs on the caller's CPU. BINDHERE (recheck_ports() put back into wait_for_return) turns usb_transport_break red with SameStick: the stick was bound on cpu0 while the call held for it spun there with IF clear. The pure accounting holds too: HOLDNOTOOK (hold without took(PortReset)) fails every_path_through_a_call_ends_inside_its_bounds and a_command_sent_again_after_a_hold_leaves_the_last_rung_its_bound. The kernel wiring that carries the call into the re-issue is untested. REISSUEFRESH gives the re-issue after Back a closed AfterBreak (ctrl.after_break = if back { AfterBreak::CLOSED } else { call };). It builds and passes usb_transport_break with exit 0. The run's held call ended 2.102 s after the break, under the test's 4.75 s. That is round 1's second half exactly: a re-issue opening a fresh 4.75 s budget up to 2 s into the call.
• B2: OPEN. NOOWED (flush_lost = owed && false) is killed: OwedFlush: … no line reads "usb-storage: disk 0 flush failed: …". Two problems remain, below.
• B3: CLOSED. LATECAP in toyos_xhci::late (the ring cap set to usize::MAX) fails a_ring_that_keeps_producing_is_given_up_on_one_ring_past_the_deadline.
• B4: CLOSED. The test is green at head. WEDGEGONE, which renames the kernel's usb-wedge: stopping every CPU at the record, is red with "usb-wedge: stopping every CPU at the" never reached the the loader's file, so the moved judge cannot pass with the wedge's lines absent.
• Round 1 NOTE SERIALSHORT is now killed: serial number "TOY............." : a serial number that arrived short was read.

BLOCKER

• kernel/src/drivers/xhci/wait/msc.rs:2357 — no test fails when the re-issue after a hold gets a fresh budget. REISSUEFRESH survives usb_transport_break — on high-risk code this is the claim that closes round 1's B1 (the whole call, re-issue included, inside CALL_AFTER_BREAK < ACK_TIMEOUT). A staging that stalls the re-issued command (a break on the re-issue, or a re-issue that spins its waits) and times it against the break's stamp would kill it.
• kernel/src/drivers/xhci/wait/msc.rs:180 and :2046 — owes_a_flush is unflushed && !no_write_cache and ignores flush_lost. Take a write acknowledged on instance 1, then a leave, so instance 2 is adopted with flush_lost = true. If instance 2 leaves again before any write or flush (for example when the held operation was a read and it breaks again), it is recorded with owed_flush = false. Instance 3's next flush then returns Ok for the write instance 1's power loss may have dropped. This is derived from the code, not staged. It is the B2 ruling's own case one hop later: a pending flush_lost is itself a flush owed.
• kernel/src/drivers/xhci/wait/msc.rs:850 — nothing tests that a successful flush clears the debt. NEVERCLEAR2 (dev.unflushed &= core::hint::black_box(true) in place of = false) builds and passes usb_transport_break with exit 0. Every staged break falls on a disk with no successful flush between the acknowledged write and the leave, so "cleared by exactly a successful flush" has no arm. On a filesystem-durability claim in a device driver, both edges are needed: owed after an acknowledged write, and not owed after the flush that followed it.

NOTE

• The every-CPU-in-a-call case (the filed issue) needs every CPU inside a call on the held disk. On the T14 that is 8 CPUs at once. It cannot cost a replug: forget_the_unreturned keeps the record while the port reads connected and untaken, so the first CPU that takes a pass binds the device. It costs BudgetExpired. One caveat is recorded in the issue itself: file_backing::read_block_retrying retries BudgetExpired for up to block::DEADMAN (120 s) without leaving the kernel. A demand fill on a held disk therefore spins across successive calls, each inside 4.75 s but not their sum. Not staged.
• kernel/src/drivers/xhci/mod.rs:663 — kick_cpu is one ICR write with no delivery wait and no lock, so it is safe with IF clear. Without x2APIC it is a no-op, and a halted CPU whose timer is stopped is then never woken to bind.
• kernel/src/drivers/xhci/wait/msc.rs:2379 — HOLDUNBOUND (the hold pushed 10 s out) survives, because until (block::OPERATION, 2 s) ends every hold first. The hold's own clip is never the binding end in any test.
• The refuse_for_now loop is bounded, measured by staging. NEVERREADY stages a returning stick that never answers TEST UNIT READY while a disk is held. Port 3 was enumerated again 4 times, 0.6 s apart, and disk 0 was still lost at 2.759, 0.43 s past its window. A device that never becomes ready does not keep a disk held.
• OwedFlush has no hardware reading. Run 83 broke on the first write, with no flush owed.
• tests/common/usb.rs:1836 — CALL_AFTER_BREAK_SECS = 4.75 is a second declaration of AFTER_BREAK.whole(), which the kernel owns. The test should read the one declaration.
• EMPTYREADER (an empty usb-bot+scsi-cd hot-added on port 3) never reached the guest, because port 3 never connected. It is not counted.

REMOVE

• tests/common/usb.rs:1814, :1838 — the T14 run 79 citations in source.

SEND BACK

[Japabu]

Round 3 review at 0a6b7da5. It covers round 2's three BLOCKERs and the branch's own delta since ea1a44a6 — 696a8633..ee81dd4a (round 10) and 72e0a648..0a6b7da5 (round 11); the two merges of origin/main in the range are not reviewed. Logs are usb466-review3-* in the job scratchpad; the mutation script is usb466-review3-mut.sh, pinned to this head. Every arm's anchors matched exactly once, every arm's diff was non-empty, every kernel arm rebuilt the kernel, and git status --porcelain was empty after each. Two STALEPRC attempts aborted on an ambiguous anchor before running; the trap restored the tree and the third ran.

Readiness. PR CI at 0a6b7da5: gh api .../check-runs reads 20 success, 3 skipped, 0 failure. At this head cargo test --test toyos-build -- --nightly usb_transport_break exited 0 and cargo test -p toyos-xhci exited 0. Hardware at this head exists for the branch: T14 run 89 (image sha matched, worktree clean at 0a6b7da5) broke at 1.844 on cpu4, held the disk 1894 ms, adopted the same device on port 13 at 2.923, and the WRITE went out again at 2.941 and completed — 1.097 s after the break, and past the operation's 2 s, which is AfterBreak::issue's exemption doing the thing it was added for.

Round 2 BLOCKERs

• B1 (the re-issue after a hold gets a fresh budget): CLOSED. REISSUEFRESH (ctrl.after_break = if back { AfterBreak::CLOSED } else { call };) is now killed: usb_transport_break --nightly EXIT=1, SilentReturn: the call its transport broke in ended 5.115 s after the break, past the 4.75 s a call may spin for (and 5.113 s alone). Round 10's usb-return-silent is the staging that made it redable.
• B2 (owes_a_flush ignores a carried debt): CLOSED. NOOWED (Debt::adopted(owed && black_box(false))) EXIT=1, OwedFlush: after the break, no line reads "usb-storage: disk 0 flush failed: …". The two-hop rule the ruling actually named is killed too: CARRIEDDROP (carried dropped from Debt::owed) → cargo test -p toyos-xhci EXIT=101 on a_debt_carried_is_carried_again_until_a_flush_reports_it and a_device_with_no_cache_owes_only_what_it_carries.
• B3 (nothing tests that a successful flush clears the debt): CLOSED. NEVERCLEAR2 (dev.debt.flushed() behind black_box(false)) EXIT=1, FlushedStick: "… its volume carries on, and it left owing a flush of writes it had reported complete …" of a stick that came back as itself. The actuator keeps its own WROTE/FLUSHED off the driver's debt, so the staging cannot be satisfied by the thing under test.
• Round 2's NOTEs are answered: CALL_AFTER_BREAK_SECS is gone and the test reads toyos_xhci::call::AFTER_BREAK.whole(); apic::init stores X2APIC_ENABLED unconditionally (kernel/src/arch/apic.rs:62), so the new ports_wanted comment is true; the demand-fill spin is in the issue. Round 2's REMOVE is honoured — the two T14 run 79 citations are out of tests/common/usb.rs.

Round 11's own negative controls re-measured here: PRCONLY (reset_finished back to the flag alone) and STALEPRC (that plus reset_write without its acknowledge) both EXIT=101 on a_warm_reset_is_finished_only_once_the_controller_has_cleared_pr.

BLOCKER

• kernel/src/drivers/xhci/wait/boot.rs:460, toyos-xhci/src/portsc.rs:135 — round 11's inherited reset and reset_finished have no hardware reading at this head, and the only T14 run that ever took them went red. grep -c "already trained" t14-run88/head.log = 0: run 88's stick enumerated on 00:14.0 port 1, a USB2 protocol register at speed=3, so the boot scan never reached inherited_reset's new branch. Run 89's two hits are both the hot-plug line xHCI: port 13 connected, link already trained at 2.921 s, which is reset_needed's path and explicitly unchanged. The one run that did take it is 86, at head a07f021d: port 13 link already trained before this kernel ran; warm resetting it at 0.595, then is SuperSpeed and its link would not train … (PORTSC 0x002212b1); skipping it in the same millisecond, no disk, rootfs.rs panic. 0a6b7da5 is the fix for that, and it has never run that path on the machine that produced it. The change converts a port that previously enumerated and answered four tag-checked round trips (run 84) into a hard dependency on a warm reset nobody has watched succeed at bring-up here, and the PR states the QEMU arm cannot red on it: "no guest boot in this suite can red on this". On device-driver code, a claim standing on a hand-written PORTSC table alone. One plain T14 boot with the stick on a USB3 receptacle closes it.

NOTE

• toyos-xhci/src/portsc.rs:127 — reset_changed is pub with exactly one caller, reset_finished, and no user outside the module; grep -rn reset_changed over the tree is three hits: the definition, that caller, and one assertion in its own test.
• kernel/src/drivers/xhci/wait/boot.rs:463 — init_device calls port::reset_needed only to choose a log string, after inherited_reset has already answered; the boot scan asks a question it no longer acts on.
• kernel/src/drivers/xhci/wait/boot.rs:470 — a port whose controller leaves PR set now costs the boot the whole USB_TIMEOUT_NS (2 s), and 4 s where a USB3 hot reset escalates, where a stale flag used to end the wait at once. Bounded and correct; unmeasured on hardware.
• CI at this head is attempt 2 of run 35760805511. Attempt 1's guest (9) and guest (10) were red on log_reserve_window ("Init process crashed during boot") and log_nested_emit ("the record-making child would not start: other error"). cargo run -- --known-red answers "NO, not quarantined — its failure fails the suite" for both, and no file under issues/ carries log_nested_emit or that message. Two reds re-run rather than adjudicated.
• toyos-xhci/src/port.rs:69 — reset_needed's header still says "The one place that question is answered" while two functions now answer it; the sentence is qualified in place, so this is a doctrine that reads thinner than it did, not a second path.
• toyos-xhci/src/flush.rs earns its place: it takes two bools off MscDevice, puts the carried-debt rule in one function, and both of the rule's edges are arms that fail.

REMOVE

• PR body, Round 11 defect 1 — "This fix removes an unspecified request and does what the specifications and Linux do; it is not shown to cure run 84. The T14 run below is the test." There is no such run anywhere below it; the body promises a reading it does not carry, and that text becomes main's record.
• toyos-xhci/src/port.rs:524, :527, :542, :547 — four new T14 run 86 / run 84 citations in source, the class round 2 had removed from tests/common/usb.rs.

SEND BACK

[Japabu]

Round 3's BLOCKER is closed by a bench boot, not by a change: T14 run 90, the head 0a6b7da5 this review ran on, plain jobcase image sha256 d9194368aceaafb71dedc390dc45e3b86fe3bdeda4b4743bf03e33ee63f0f2d5, with the stick trained on the USB3 half of the receptacle — the half that produced runs 84 and 86.

[0.596 cpu0] xHCI: port 13 connected
[0.596 cpu0] xHCI: port 13 link already trained before this kernel ran; warm resetting it before its device is asked anything
[0.701 cpu0] xHCI: port 13 enabled, speed=4
[0.702 cpu0] usb-storage: disk 0 ready on slot 5, 7507812 blocks of 512 B (29327 MiB), msc_block +0x30000
[1.268 cpu0] Boot: complete (1268ms)

The loop reports EXIT=0, the record reads DONE, the stick enumerated 0 s after the machine answered and toyos-fat32-check passes on the log partition. The warm reset of the inherited link took 105 ms, which is the cost this change adds to such a boot, measured rather than estimated.

The review is right that runs 88 and 89 did not take that path: their stick trained on the USB2 half at speed=3. Runs 88, 89 and 90 were three boots of the same image; which half the stick trains on is not under the harness's control, so it took three.

🤖 Generated with Claude Code

https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK