Skip to content

A reset records the Bulk-Only phase a device was left in, and the black box keeps that account readable - #447

Open
Japabu wants to merge 20 commits into
mainfrom
wt/toyos-usbhang
Open

Japabu wants to merge 20 commits into
mainfrom
wt/toyos-usbhang

Conversation

@Japabu

@Japabu Japabu commented Sep 13, 2026

Copy link
Copy Markdown
Collaborator

A reset says which Bulk-Only phase it found a device in; it does not finish the
command, because six T14 boots say the device does not need it and the ring it
would write is one a live driver may still be enqueuing on.

What the machine said, and what this branch claims because of it

Six T14 boots cut a Bulk-Only command on the bench's own SanDisk Ultra and it
came back from every one
, stick_secs 0, PASS, EXIT=0:

run tree what the device was holding what ended the machine stick
39 base a CBW, nothing queued for its data, after one 4 KiB write deadline, 120062 ms 0 s
43 base the same, after 4 MiB of writes deadline, 120065 ms 0 s
43 base a data TRB on the ring, doorbell never rung deadline, 120066 ms 0 s
43 base the data taken, nothing reading its CSW deadline, 120065 ms 0 s
45 base a write in flight — the sweep never stopped the hard-lockup detector, 60003 ms 0 s
47 this branch a write in flight, 236 TRBs pending deadline, 120066 ms 0 s

Run 45 is the base arm and its bound is the finding the first review caught:
that sweep held IF clear for the rest of the boot, so
toyos_tco::hard_lockup_bound_ms ended the machine half a bound early and the
page read a cpu locked up with interrupts off: cpu7 has taken no interrupt for 60003 ms — not the bound the judge demands. Run 47 is the same arm with the
stimulus fixed, and the lockup bound did not fire.

So the reset path publishes and records and does not act:

  1. toyos_xhci::bot::Phase — where one Bulk-Only round trip stands (BOT 1.0
    §5.1), published by the driver between every enqueue and its doorbell and at
    each gap between two phases, read by the reset out of an atomic because that
    path may take no lock.
  2. xhci::stop::settle_commands — gives the driver USB_TIMEOUT_NS to close
    the command, then writes which phase the device was left in and what the
    controller says about its data endpoint. Reported, never judged.
  3. toyos_blackbox::ACCOUNT_BYTES and Account — the account reaches the page
    at all, and says how far the reset got if the machine ends mid-way.

It does not finish the command. An earlier revision of this branch did, and
it is deleted: nothing on that path can hold the controller lock, so a TRB
written onto a ring rebuilt from published numbers can be a second write of a
block the device already has — and no machine has shown the finish necessary.
BOT §5.3.4 makes reset recovery the device's obligation and this bench's device
honours it.
issues/kernel/a-cut-bulk-only-command-does-not-brick-the-benchs-stick.md
carries the rows and what is not claimed.

What "236 TRB(s)" rests on

The number in run 47's account — and in the tracker and in this branch's commit
messages — was produced by dequeue.wrapping_sub(base) / 16 and a
% RING_SIZE, in the kernel, with no test and no refusal: a dequeue pointer
that is not on the ring wrapped into some value between 0 and 255 that the
account printed as the hardware's own word.

That computation is now toyos_xhci::Ring::pending, in the pure crate beside
EndpointState::decode, which reads the other half of the same endpoint
context:

pub fn pending(self, dequeue: u64) -> Result<u16, NotOnTheRing>
  • It refuses by name. NotOnTheRing::Dequeue for a pointer before the
    ring's first TRB or past its last — which is what a context the controller
    never wrote, or one read mid-publication, reads as — Unaligned for one on
    the ring and not at a TRB, Tail for a driver enqueue point past the ring's
    end. The account prints the refusal where it used to print a number, so no
    reader can take one for the other.
  • The arithmetic is unchanged for a pointer that is on the ring, which is
    what makes run 47's 236 still the same 236: same masking of the low four bits
    (xHCI 1.2 §6.2.3 — Dequeue Cycle State plus three RsvdZ), same wrapped
    distance from the controller's index to the driver's tail. What is new is that
    a pointer off the ring is now said rather than wrapped.
  • It is tested, including the wrap in both directions, both ends of the
    ring, the four low bits, an unaligned base, and a ring of no TRBs — the
    division by zero this path may not take.

What is still owed on it: run 47 was measured under the old, unrefusing
computation, so that its pointer was genuinely on the ring is inferred from the
count being in range and the endpoint being Running on a ring the driver had
just enqueued on — not witnessed. A boot of the usbload arm on this tree would
witness it; the bench's stick is dead awaiting a physical replug, so that boot is
not taken here.

The black box lost the account, and the reserve was not why

Measured off the four pages (awk over each loader.log, split at the first
usb-quiesce line):

page report bytes, of REPORT_BYTES = 14296 page bytes free account
run 43 usb-wedge-data-owed 13773 2571 absent
run 43 usb-wedge-in-data 13763 2581 absent
run 43 usb-wedge-before-status 14253 2091 798 B
run 45 usbload 14258 2086 930 B

No report exceeded its reserve and neither losing page was full. stop_all
writes its barrier line and then waits — 2000 ms settling, a port-reset bound,
100 ms of device recovery per controller — and the one statement that rewrites
the length and checksum covering everything written was only reached after all
of it. A machine that ends inside one of those waits leaves every byte already
written covered by nothing.

toyos_blackbox::Account is the fix, and it seals per line, not per
fragment
: one writeln! reaches a writer as the pieces either side of each
argument, so a writer that sealed on every one of them would cover a torn line
with a valid checksum — the same defect one layer down. Bytes past the last
newline stay on the page, outside the sealed length, invisible to recover.
It lives in the crate rather than the kernel because the previous writer was
constructible only in the kernel and no host test could drive it.

Both are recorded in
issues/diagnostics/a-wedged-boots-record-outgrows-both-channels-that-carry-it.md,
which this does not close: its loader.log half stands.

The two checks

Negative controls (host, run):

mutation what it restores cargo test -p toyos-blackbox
ACCOUNT_BYTES = 0 the reserve gone 101a report ran to 16296, leaving under 1024 B for the account
Account commits on every fragment the torn line sealed as a complete account 101 — the cut line reads back (…xHCI 00:14.0 halted=), and one line built from three fragments seals 9 times where 3 lines owe 3
Account never commits the account covered by nothing 101 — the page carries the report and none of the account ("[0000] why\n")

Restored after each: 0, 26 passed.

The second and third are the control on the commit-per-line half that the last
round did not have: the test named then drove Report::commit directly and
passed with the kernel's writer deleted. The current test asserts the cut line
is not readable and that one line built from three fragments seals exactly
once.

Independent oracle: USB Mass Storage Class Bulk-Only Transport 1.0 §5.1,
§5.3.4, §6.7.2–3 and xHCI 1.2 §4.7 / §4.11.1 / §5.4.8 / §4.19.5 / §6.2.3, cited
at the sites — and the T14's own boot.usbload.stick_secs, a physical device on
a laptop whose ports never drop VBUS, answering through a channel this kernel
does not write.

The stimulus, and the state it leaves its CPU in

usb-reset-under-load sweeps the last eighth of the stick from the shutdown
syscall so the reset lands on a controller that is moving bytes. It keeps its
CPU interruptible — preemption off, IF on, the one-shot armed — so the boot
deadline and not the hard-lockup detector ends the machine, and it now puts
all three back on every path out
: an Interruptible guard whose Drop
restores them, because the caller goes on to drain write-back, sync every
filesystem, flush every disk and wait for the log to be durable. Whether the
syscall arrived with IF masked is read before the sti rather than assumed.

What this does not have

  • One CI measurement. usb_reset_records_the_phase_it_cut is a new
    registration and every shortcut is refused by the tree: src/durations.rs
    refuses a committed UNMEASURED marker, src/tiers.rs refuses that marker on
    a Nightly row and refuses a Nightly row with no CI evidence, and the local
    measurement of 63 s can never be Tier::Fast against a 10 000 ms ceiling. It
    is bootstrapped Tier::Fast with the marker, which is the two-cycle path
    tests/CLAUDE.md states. durations reds until a second commit replaces
    the marker with the measured value and re-tiers the name Nightly.
    This is
    not landable before that.
  • A T14 boot on this tree. The bench's stick is dead awaiting a physical
    replug, so no run was requested. What a boot would witness is named above.
  • Step 3 of the original brief — catching the real hang — is not started. The
    three kills share the hang, not the reset, and the stick cannot carry evidence
    about its own death; that question moves to the network track.

Fence

Kernel, pure crates (toyos-blackbox, toyos-xhci) and the harness. No ABI:
git diff --name-only origin/main touches no file under toyos-abi/, toyos/
or userland/libc/.

Gates, each with its own command's exit status

command exit
cargo run -- --clippy 0 (5 invocations clean)
cargo run -- --build-only 0
cargo test --lib 0 (298 passed)
cargo test --workspace --exclude toyos-build 0
cargo test -p toyos-blackbox -p toyos-xhci 0 (26 and 53 passed)
cargo test --test toyos-build usb_reset_records_the_phase_it_cut -- --nightly 0, PASS (64 s)
cargo test --test toyos-build usb_reset_hands_devices_back -- --nightly 0, PASS (39 s)
mutation: Account commits per fragment 101
mutation: Account never commits 101
mutation: ACCOUNT_BYTES = 0 101
restored, cargo test -p toyos-blackbox 0 (26 passed)

The renamed guest arm was run twice. The first attempt exited 2 and is not
quoted as a result: test result: INVALID. 0 passed, 1 invalidated by a host suspend of 994s — the laptop slept mid-run, which tests/CLAUDE.md says is a
re-run and not a verdict. The row above is the re-run, which carries no
invalidation line. No T14 boot was taken: the bench's stick is dead awaiting a
physical replug.

🤖 Generated with Claude Code

https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK

Japabu and others added 13 commits September 13, 2026 14:13
…e TRBs it queued

The stop this kernel performs before every reset waited for `IN_FLIGHT`, a count
raised between a bulk TRB's enqueue and its completion. A Bulk-Only Transport
command is three transfers (BOT 1.0 §5.1: the 31-byte CBW out, an optional data
phase, the 13-byte CSW in), and between any two of them that count is zero while
the device is still holding the command. Every completed boot therefore wrote
`usb-quiesce: no bulk transfer was outstanding, so this reset cuts none` — true
of the TRBs and not of the protocol — and a boot ended by the deadline in one of
those gaps port-reset a device waiting for data it will never get. §6.7.2/§6.7.3
leave such a device waiting; the class's way out is §5.3.4's Reset Recovery,
three control transfers a wedged kernel cannot issue.

`toyos-xhci`'s new `bot` module is the decision: six phases the driver publishes
as it walks a round trip, and what a reset owes a device in each. A transfer
already queued is the controller's to finish, so re-queueing one is refused by
the decision rather than by the site — for an out data phase, that would be a
second write of the block.

`xhci::stop` publishes the one open command the way it already publishes its
controllers, because the CPU holding `XHCI` may be the wedged one the reset is
ending: the device's pool block, its doorbell window, the data phase the CBW
promised, and where each bulk ring's next enqueue point is. `settle_commands`
then gives the driver its own bound to close the command, and where that does
not happen it queues the data the device is owed and a read of the CSW, rings
the doorbells and polls the CSW's own signature out of DMA. Nothing reads the
event ring, which is the one structure the wedged CPU is still the consumer of.
`InFlight` is gone: the command's guard spans the whole round trip and subsumes
it. On a boot with nothing open the path costs one relaxed load, as before.

`usb-wedge-mid-write` is the stimulus: at the shutdown syscall, after the job
list, it rewrites block 0 of the boot stick with the bytes it has just read from
it and stops every CPU between that command's CBW and its data phase, leaving
only the boot deadline to end the machine. The write is byte-for-byte
idempotent, so the medium is what it was however much of it the reset completes
— which is what lets a control that deliberately cuts a write run against the
bench's own stick.

`cargo test --lib`: 297 passed, exit 0.
`cargo test --workspace --exclude toyos-build`: exit 0, 138 suites ok.
`cargo test` in toyos-xhci: 49 passed, exit 0.
`cargo clippy --workspace --exclude toyos-build --all-targets -- -D warnings`: exit 0.
`cargo clippy -p toyos-build --all-targets -- -D warnings`: exit 0.

The kernel is UNBUILT on this commit: `cargo run -- --build-only` refuses because
the machine-wide sysroot is held by /Users/jan/Dev/jan/toyos-aperture for an
unlanded `toyos-abi/src/boot.rs` change. No T14 boot has run, so
`boot.usbwedge.*` carries ceilings and no readings, and
`usb_reset_finishes_an_open_command` carries the `UNMEASURED` marker that buys
its one CI measurement.
…t a second spelling of it

`usb_reset_hands_devices_back` held its own copy of the needle — "bulk transfer"
— beside the one in `metaldevices::LOADER_RECORDS`. Renaming the kernel's line
moved one and left the other, so all four arms failed by name against a kernel
that settles strictly more than it used to:

  FAIL usb_reset_hands_devices_back: 4 of 4 reset path(s) unmet:
    the orderly reboot: the account says nothing about a bulk transfer, ...

A second spelling of a predicate is one that goes on passing after the kernel
stops writing the line, which is the failure this one only just avoided by
failing loudly instead. The gate now reads `metaldevices::QUIESCE_COMMAND`.

`cargo test --test toyos-build usb_reset_hands_devices_back -- --nightly`: exit 0,
1 passed, all four paths reporting `and the stick enumerated again`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK
…ts image

`metalprofile` refuses a boot whose list nobody has priced:

  [metal] the boot "usbwedge" runs 0 job(s) and list.usbwedge.job_ms prices none
  of them; a list nobody has priced cannot be sized to the runner's bound

It is `deadlinewedge`'s list — one job and a wedge — taken one write later, so
it carries that row's ceiling and says so.

With it, `cargo test --test toyos-build usb_reset_finishes_an_open_command --
--metal --metal-readback target/metal-usbhang` stages
target/metal-usbhang/usbwedge/image.img, armed with
root=8ca4ff257e3ea079676f438e6298489f,usb-wedge-mid-write,boot-deadline=120000
read out of the image itself, and exits 2 (Verdict::Staged — the machine was not
touched).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK
…taged on it

Reverted here and reverted back in the very next commit. This tree exists to
build one image and nothing else, and it is on the branch so that the image the
T14 refuses came from a committed tree rather than from a working copy nobody
can reproduce.

Everything the fix touched goes back to `origin/main`: `xhci::stop` settles
`IN_FLIGHT` again, `msc` publishes nothing, `normal_trb` and `Dma::from_addr`
are gone, and `toyos_xhci::bot` does not exist. What stays is the stimulus —
`usb-wedge-mid-write`, its `mid_write` module, and the one call between the CBW
phase and the data phase — so this boot wedges in exactly the state the green
arm wedges in and the machine answers about the reset alone.

`cargo test --lib` reds on this commit by construction:
`the_kernel_writes_the_quiesce_lines_the_host_reads` holds the host's needles to
a `stop.rs` that no longer writes them. That is the revert being complete, not a
second defect.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK
… wedge staged on it"

This reverts commit 3fb10e2.

The control image is built and kept
(target/metal-usbhang-control/usbwedge/image.img,
sha256 9e917340770a4d7f5a616b0e5a956fa387214deed32c0fb17006151bf693b00a), and the
tree it came from is in this branch's history where anybody can rebuild it. What
it measured on the way past, with the reverted kernel in the guest:

  FAIL usb_reset_finishes_an_open_command: "the device answered with a CSW of
  status" never reached the boot after the reset
  test result: FAILED. 0 passed, 1 failed, 1 total (226.7s)

— red twice, the same failure both times, against the same base the green arm
passed on.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK
… three — and the account now fits on the page

Run 39 flashed the control — the base's reset path with the wedge on it — and
the stick came back: `back_secs 225`, `stick_secs 0`, EXIT=0. The stimulus fired
and the deadline ended the machine at 120062 ms, so a SanDisk Ultra holding a
CBW with nothing queued for its data phase survives this platform's port reset.
The green arm was not run: a control that does not kill the stick measures
nothing about the fix.

The recorded kills (runs 25/26 ccorpus, run 33 metalcase) were sustained
multi-megabyte writes. So the stimulus now varies both things that could matter.
Every arm moves 4 MiB to the device first — 64-block pairs read and written back
byte for byte at the disk's own end, because the bench's stick is thirty
gigabytes and the image on it is eighty megabytes — and takes the wedge inside
the very next command, at one of three phases:

  usb-wedge-data-owed      the CBW is taken and no data is queued
  usb-wedge-in-data        a data TRB is on the ring and its doorbell unrung
  usb-wedge-before-status  the data is taken and nothing reads the CSW

A boot carries exactly one; `quiesce` refuses two, because which phase a device
is stopped in is the whole measurement.

`usb-wedge-in-data` is ground the first design could not reach, and it found a
hole: the driver publishes its phase *between* the enqueue and the doorbell, so
a machine stopped there leaves a TRB the controller was never told about, and
`bot::owed` treated "queued" as "the controller's to finish". The reset would
have waited out a data phase nothing was moving. `Owed::ring_data` is the fix.

And the page: run 39's `loader.log` carries `older records dropped to fit this
page: 138` and **no `usb-quiesce` lines at all**. `Report::tail` fills to
`TEXT_BYTES`, so `blackbox::append`'s `Report::reopened` wrote into zero bytes
and said nothing about having done so. The reset's account — the only evidence
there is about the reset — was silently absent on the boot it exists for.
`readbacks/deadlinewedge` and `readbacks/hardlockup` carry none either, so both
existing wedge arms have been losing it since they were written.
`toyos_blackbox::ACCOUNT_BYTES` now reserves an eighth of the page for it.

  cargo test in toyos-blackbox: 24 passed, exit 0
  cargo test in toyos-xhci: 50 passed, exit 0
  cargo test --lib: 297 passed, exit 0
  cargo test --test toyos-build usb_reset_finishes_an_open_command: exit 0,
    PASS (61s), all three phases:
      usb-wedge-data-owed: stopped in its data (unqueued) phase, and the reset read the CSW
      usb-wedge-in-data: stopped in its data phase, and the reset read the CSW
      usb-wedge-before-status: stopped in its status (unqueued) phase, and the reset read the CSW

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK
…phases staged on it

Reverted here and reverted back in the very next commit. This tree exists to
build three images and nothing else, and it is on the branch so that the images
the T14 refuses came from a committed tree rather than from a working copy
nobody can reproduce.

`xhci::stop` settles `IN_FLIGHT` again, `msc` publishes no phase, and
`normal_trb` and `Dma::from_addr` are gone: `git diff origin/main HEAD --
kernel/src/drivers/xhci/stop.rs kernel/src/mm/dma.rs` is empty.

Two things are deliberately NOT reverted, because neither is the behaviour under
test and both are what makes the control readable:

- `toyos-xhci/src/bot.rs` — a pure module with no caller left in this kernel.
  The actuator names its phase with `bot::Phase`, and a control that could not
  say which phase it staged would answer the question with nothing.
- `toyos-blackbox`'s `ACCOUNT_BYTES` reserve. Run 39's control lost the whole
  `usb-quiesce` account to a full page; without the reserve this one would too,
  and what the base's reset path did is exactly what these boots are for.

One difference from the green arm's kernel, and the device cannot see it: on
this tree `bulk` takes no phase, so the `Phase::Data` wedge is taken just before
the data transfer is queued rather than between its enqueue and its doorbell. A
TRB nothing has rung for is a TRB the device never sees, so both leave it in the
same state — holding the CBW with no data arriving.

`cargo test --lib` reds on this commit by construction:
`the_kernel_writes_the_quiesce_lines_the_host_reads` holds the host's needles to
a `stop.rs` that no longer writes them. That is the revert being complete.

`cargo run -- --build-only`: exit 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK
…e wedge phases staged on it"

This reverts commit 09a7064.

The three control images are built and kept under
target/metal-usbhang-control/, and the tree they came from is in this branch's
history where anybody can rebuild it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK
…arm never lets the bus go idle

Run 43 flashed all three phase controls — the base's reset path, one per phase,
4 MiB of writes behind each — and the stick came back from every one:
`stick_secs 0`, PASS, EXIT=0. Each page shows the stimulus firing at ~2.8 s, the
deadline at ~120065 ms, and (now that the page reserves the account's bytes) the
base's own `no bulk transfer was outstanding, so this reset cuts none` followed
by the port resets. By the rule in the request no green arm ran.

What that says is about the *stimulus*, not about the device. A wedge stops CPUs
and then waits two minutes for the deadline — and the controller is not a CPU. A
TRB with its doorbell rung completes in microseconds; one never rung never
starts; a CSW nobody reads sits in the device's buffer. Two minutes later there
is nothing in flight on any of the three phases, and this platform's SanDisk
survives a port reset in every one of those idle states.

The three kills (runs 25, 26, 33) were not idle. `usb-reset-under-load` stages
what they were: from the shutdown syscall it sweeps the last eighth of the stick
— reading each 256 KiB run and writing it back byte for byte — and never stops,
so the deadline resets the machine with the controller moving bytes and the
device programming flash. A sweep and not a rewrite, so no block is programmed
twice in a boot; refused by name on a disk with no room for one, which is every
guest here and is what the QEMU half of the arm establishes.

And the account now says what the reset found rather than only what the driver
had queued: `OPEN` carries the device's own context block, and `settle_commands`
reports the data endpoint's state and how many TRBs the controller had not
reached on the ring. That is reported and never judged — which states this
platform survives is the open question, and a predicate over it would be the
suite deciding the answer.

  cargo run -- --build-only: exit 0
  cargo test --lib: 297 passed, exit 0
  cargo test --test toyos-build usb_reset_finishes_an_open_command: exit 0, PASS (66s)
    usb-wedge-data-owed: stopped in its data (unqueued) phase, and the reset read the CSW
    usb-wedge-in-data: stopped in its data phase, and the reset read the CSW
    usb-wedge-before-status: stopped in its status (unqueued) phase, and the reset read the CSW
    usb-reset-under-load: refused by name on a disk with no room, and the boot ended

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK
…aged on it

Reverted here and reverted back in the very next commit. This tree exists to
build four images and nothing else, and it is on the branch so that the images
the T14 refuses came from a committed tree rather than from a working copy
nobody can reproduce.

`xhci::stop` settles `IN_FLIGHT` again, `msc` publishes no phase and no context,
and `normal_trb` and `Dma::from_addr` are gone: `git diff origin/main HEAD --
kernel/src/drivers/xhci/stop.rs kernel/src/mm/dma.rs` is empty.

Not reverted, because neither is the behaviour under test and both are what make
the control readable: `toyos-xhci/src/bot.rs` (the actuator names its phase with
`bot::Phase`) and `toyos-blackbox`'s `ACCOUNT_BYTES` reserve (without it the
control loses its own account to a full page, and what the base's reset did is
what these boots are for). The load arm needs neither.

One difference the device cannot see: on this tree `bulk` takes no phase, so the
`Phase::Data` wedge is taken just before the data transfer is queued rather than
between its enqueue and its doorbell. A TRB nothing has rung for is a TRB the
device never sees.

`cargo test --lib` reds on this commit by construction:
`the_kernel_writes_the_quiesce_lines_the_host_reads` holds the host's needles to
a `stop.rs` that no longer writes them. That is the revert being complete.

`cargo run -- --build-only`: exit 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK
… arms staged on it"

This reverts commit bf7d015.

The four control images are built and kept under target/metal-usbhang-control/,
and the tree they came from is in this branch's history where anybody can
rebuild it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK
…acker keeps the finding

Run 45 put the base's reset path through a write that was still in flight — the
sweep never stops, so the deadline reset the machine with the controller moving
bytes. The kernel's own account on that page reads

  usb-quiesce: 1 bulk transfer(s) were outstanding and 1 still is after 2000 ms,
  so this reset cuts them ... a device cut in its data phase may need a physical
  replug before its next host can enumerate it

followed by every port reset and power-off — and the stick enumerated on the next
host anyway. With runs 39 and 43 that is four states of a cut Bulk-Only command,
four clean controls, and the reading this branch started from does not hold: the
phase the reset finds the command in is not what loses this stick. The three
kills share the hang, not the reset, and what a hang leaves the controller and
the device in is not reachable by a deliberate wedge — nor knowable through the
stick, which is the channel and is what dies.

That finding is now
`issues/kernel/four-phases-of-a-cut-command-and-the-stick-survived-all-four.md`,
with the five rows and run 45's own words in it.

What the bench keeps is one boot, not four. `usbload` is the owner's ruling as a
standing check — a machine writing to the stick continuously, reset out from
under itself, and the device enumerable afterwards — and `boot.usbload.stick_secs`
is the verdict. The three phase arms measured the same idle bus under three
names and are answered; they stay as QEMU arms, where they cost seconds and are
what a mutation of the reset path is seen in, and their metal rows are gone.

`usb_load_chain` asserts what the boot staged rather than what the device did:
the sweep ran, nothing but the bound ended the machine, and the reset's account
reached the page — which is `toyos_blackbox::ACCOUNT_BYTES` under test on the
machine. What state the reset found the controller in is printed and not judged;
this bench has four clean answers to that and no theory, and a predicate over it
would be the suite deciding the question.

  cargo run -- --build-only: exit 0
  cargo test --lib: 297 passed, exit 0
  cargo test --test toyos-build usb_reset_finishes_an_open_command: exit 0, PASS (63s)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK
@Japabu Japabu changed the title A reset settles the Bulk-Only command a device is inside, not just the TRBs it queued A reset settles the command a device is inside, the black box keeps room to say so, and four phases of a cut command did not brick the bench's stick Sep 14, 2026
@Japabu
Japabu marked this pull request as ready for review September 14, 2026 08:29
…finish its command

The review sent the branch back with the machine's own pages against three of
its claims. Each is answered here.

THE FINISH IS DELETED. `settle_commands` rebuilt the driver's rings from numbers
`OpenCommand` published and enqueued TRBs onto them. Nothing on that path can
hold the controller lock — that is the whole reason `POINTS` and `OPEN` exist —
so on every path but the panic one the driver's own CPU may still be enqueuing
there, and for an out data phase that is a second write of the block. Five T14
controls said the finish buys nothing on this bench: a SanDisk Ultra cut while
holding a CBW, while a data TRB sat unrung, while a CSW went unread, and with a
write genuinely in flight all came back `stick_secs 0`. So `finish`, `bot::owed`,
`bot::Owed`, `normal_trb` and the `Open`/`Device` fields only they read are gone,
and what the path keeps is the publication and the account: which phase the
device was left in, and what the controller says about its data endpoint.

THE ACCOUNT WAS NOT LOST TO THE RESERVE. Run 43's `usb-wedge-data-owed` and
`usb-wedge-in-data` pages carry zero `usb-quiesce` lines with `ACCOUNT_BYTES` in
place. Measured off the pages, their reports ran to 13773 and 13763 bytes of the
14296 a report may spend, leaving 2571 and 2581 free; `usb-wedge-before-status`
and run 45 spent 14253 and 14258 and carried 798 and 930 bytes of account. The
page was never full. The loader recovered a valid WEDGED page from that DRAM
afterwards, so `append`'s own `recover` succeeded and the closure ran — and
`stop_all` writes its barrier line and then waits seconds before `Report::seal`,
the one statement that rewrites the length and checksum covering everything
written. A machine that ends inside one of those waits leaves every byte
invisible and `append` returns true. `Report::commit` and a per-line
seal-and-flush in `blackbox::append` fix it; a page now says how far the reset
got. The reserve stays as the other half, and its test no longer asserts over
the constant it tests — `ACCOUNT_BYTES = 0` now reds it.

THE SWEEP WAS A HARD LOCKUP AND IT WRAPPED. Run 45's page reads `a cpu locked up
with interrupts off: cpu7 has taken no interrupt for 60003 ms`, not the boot
deadline the judge demands: the sweep never returned to a scheduler pass, so its
CPU's one-shot was never re-armed. It now takes `deadline::this_cpu`'s three
statements, so the deadline reaches it, and `usb_load_chain` refuses a page
naming the other bound. It also wrapped, so "no block is programmed twice in a
boot" — the `Flash::Ok` rationale for the owner's own stick — was false; the
sweep is bounded to one pass and says so by name if it reaches the end.

Also: `restart_bulk` on a stalled data phase republishes the ring it rebuilt;
two armed wedge phases are refused by name instead of asserting inside the
shutdown syscall; `metaldevices`' text scan reads the kernel's code and not its
comments, with a negative control on the stripper; the three wedge arms leave
`FLASHABLE` and `WEDGE_ARMS`, since no profile builds an image carrying one; the
harness spells each arm's parameters once, with no catch-all; and every prose
finding is answered by deletion, in source and in the tracker.

`usb_reset_finishes_an_open_command` stays `Tier::Fast` with the one-run marker:
`src/tiers.rs` refuses a Nightly row with no CI evidence and refuses the marker
on a Nightly row alike, so the name costs the two CI cycles `tests/CLAUDE.md`
states and this is not landable until the second.

Gates: `cargo test --lib` exit 0, 298 passed; `-p toyos-blackbox -p toyos-xhci`
exit 0, 25 and 46; the mutation exit 101; `--workspace --exclude toyos-build`
exit 0; `cargo run -- --clippy` exit 0; `--build-only` exit 0;
`usb_reset_finishes_an_open_command` PASS 64 s; `usb_reset_hands_devices_back`
PASS 42 s. One T14 boot is owed and staged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK
@Japabu Japabu changed the title A reset settles the command a device is inside, the black box keeps room to say so, and four phases of a cut command did not brick the bench's stick A reset records the Bulk-Only phase a device was left in, and the black box keeps that account readable Sep 14, 2026
Japabu and others added 3 commits September 14, 2026 12:00
… stick came back

The first boot of the corrected stimulus, on 4d97210, and the first time this
arm's judge has executed. Transcript in the tracker's terms:

  the machine answered ssh again after 219 s
  the boot stick enumerated 0 s after the machine answered
  the boot deadline expired: a bound of 120000 ms, reached at 120066 ms

and no `a cpu locked up with interrupts off` line, which is what the re-armed
one-shot bought: run 45 was ended half a bound early by
`toyos_tco::hard_lockup_bound_ms` because that sweep took no interrupt at all.

The page carried the whole account, which no wedge page this bench has taken
before did:

  usb-quiesce: a Bulk-Only command was open in its data phase on slot 5 after
    2000 ms, so this reset cuts it
  usb-quiesce: the controller had that device's data endpoint Running with 236
    TRB(s) it had not reached on the ring

Two hundred and thirty-six transfers ahead of the dequeue pointer on a Running
endpoint is the hardware's own word that the reset landed mid-stream, and the
device enumerated on the next host 0 s after the machine answered. So the
finding grows a row and loses a claim it never had: neither the phase the reset
finds the command in nor how busy the bus is when it lands is what loses this
stick.

The profile prices the arm from that boot — complete_ms 1204, back_secs 219,
stick_secs 0 — and gains `boot.usbload.deadline_lateness_ms`, 66 ms against the
ceiling `boot.deadlinewedge.deadline_lateness_ms` derives. The first judge run
refused the arm for pricing no such number; a measurement with no ceiling cannot
fail. `list.usbload.job_ms` stays although the boot runs no jobs, because the
profile refuses a boot whose list is unpriced and says so by name.

Gates: the offline judge `usb_reset_finishes_an_open_command --metal
--metal-readback target/metal-usbhang` exit 0, PASS, 1 boot; `cargo test --lib`
exit 0, 298 passed; `cargo run -- --clippy` exit 0.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK
… sealed a line at a time

The second review sent the branch back on eight code findings. Each is answered
here; every prose finding is answered by deletion.

THE ONE NUMBER THE CONCLUSION RESTS ON IS NOW A TESTED REFUSAL. "236 TRB(s)" was
produced in `stop.rs` by `dequeue.wrapping_sub(base) / 16` and a `% RING_SIZE`,
which refused nothing: a dequeue pointer outside the ring — a context the
controller never wrote, an `OPEN` read while another CPU was publishing it —
wrapped into some number between 0 and 255 that the account then printed as the
hardware's own word. It is now `toyos_xhci::Ring::pending`, beside
`EndpointState::decode`, which reads the other half of the same endpoint
context: pure, `forbid(unsafe_code)`, and `Result<u16, NotOnTheRing>`. A pointer
before the ring's first TRB or past its last, one on the ring and not at a TRB,
and a driver enqueue point past the ring's end are three refusals with three
`Display` clauses, and the account prints the refusal where it used to print a
count. Seven tests cover the wrap in both directions, both ends of the ring, the
Dequeue Cycle State and the three reserved bits that ride in the same field
(xHCI 1.2 §6.2.3), a base that is not a TRB boundary, and a ring of no TRBs —
which is the division by zero this may not take, since it runs where the kernel
may not panic.

The account needed a base and a tail and never a ring, so `ring()`, `point()`
and its cycle packing, `OPEN`'s host-side block address, the `unsafe
Dma::from_addr` over it and the `MSC_STRIDE`/`PAGE` imports are gone with it —
one of the path's two unsafe reconstructions, and the `Dma` view it rebuilt on
the panic path.

THE ACCOUNT IS SEALED PER LINE, AND THE CONTROL FAILS WITHOUT IT. The kernel's
`Committed` writer committed on every `core::fmt` fragment, and one `writeln!`
is several: a machine ending between two of them sealed a torn line under a
valid checksum, which the next boot reads as a complete account. It is deleted
for `toyos_blackbox::Account`, which seals only when the fragment it was handed
ends the line, and leaves anything past the last newline on the page outside the
sealed length. It lives in the crate because `Committed` was constructible only
in the kernel and nothing could drive it:
`an_account_cut_off_mid_line_is_readable_to_the_last_whole_line` reds under both
mutations — the commit removed, and the commit moved back to per fragment — and
counts the write-backs, because one line built from three fragments must seal
once. The test that was named as the control last round and passed on the base
as well is deleted.

THE SWEEP RESTORES WHAT IT CHANGED. `sweep_under_load` raised the preempt count,
armed a one-shot and set `IF`, then returned on two paths into `quiesce`, so
`drain_all`, `sync_all`, `flush_disks` and `wait_for_durable` ran on that CPU
with the preempt count raised. An `Interruptible` guard establishes the three
and its `Drop` puts them back on every path; whether the caller arrived with
`IF` masked is read before the `sti` rather than assumed.

Also: the registered name is `usb_reset_records_the_phase_it_cut`, which is what
it now asserts — the reset does not finish the command, by this branch's own
argument — renamed at all six sites in this commit. The tracker entry is a
`defect` and not a `finding`, because a fold would lose the three stick kills;
its slug no longer carries a count the next boot moves, and the one citation to
it is written in the same commit, in `stop.rs`'s header where the subject lives.

The `UNMEASURED` marker and `Tier::Fast` stand, and the branch is still not
landable on them: `src/durations.rs:293` refuses a committed marker,
`src/tiers.rs:1520` refuses the marker on a Nightly row and `src/tiers.rs:1570`
refuses a Nightly row with no CI evidence, and 63 s can never be Fast against
`FAST_CEILING_MS`. One CI cycle on this pull request produces the measurement; a
second commit prices the name and re-tiers it Nightly.

Gates, each with the command's own exit status: `cargo run -- --clippy` 0 (5
invocations clean); `cargo run -- --build-only` 0; `cargo test --lib` 0, 298
passed; `cargo test --workspace --exclude toyos-build` 0; `cargo test -p
toyos-blackbox -p toyos-xhci` 0, 26 and 53 passed; the renamed guest arm
`usb_reset_records_the_phase_it_cut --nightly` 0, PASS 64 s, and
`usb_reset_hands_devices_back --nightly` 0, PASS 39 s. The commit-per-line
control both ways: `Account` sealing every fragment 101 (the torn line reads
back, and one line of three fragments seals nine times where three lines owe
three), `Account` sealing none 101 (the page carries the report and no account);
`ACCOUNT_BYTES = 0` 101. Restored, 0. An earlier run of the guest arm exited 2,
invalidated by a 994 s host suspend, and is not quoted as a result. No T14 boot:
the bench's stick is dead awaiting a physical replug.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK
The third review returned LAND with six CODE and five PROSE findings. Every one
of them is dead weight or a decision nothing exercises, most of it added while
answering an earlier round, so each is answered by deletion rather than by an
edit or by a test bought to cover it.

The guard around the load sweep is gone. `Interruptible` established preemption
off, `IF` on and the one-shot armed, and put them back from a `Drop` that no arm
ever ran: the QEMU arm refuses on the disk floor before the guard is taken, and
the metal arm never returns from the sweep. The three statements and the three
that undo them are now in `sweep_under_load` itself, and the disk-error path
breaks instead of returning, so the function has one exit and the restore is
read against the loop that needed it. Neither the order of the lines nor which
of them a boot writes changes, so `usb_load_chain` judges exactly what it did.

The two-phase refusal is gone with its constant, the constant's copy in
`bootlog`, that copy's row in the source scan, and the predicate over it. It
refused an image that armed two `usb-wedge-*` names; nothing arms two, since
`one_wedge_phase` runs one arm per boot, and a kernel that staged the wrong
phase is already caught by name — `usb_wedge_chain` demands the account name
the phase that boot asked for. The refusal answered a question the judge
answers, and the predicate `says_nothing_of(kernel, USB_WEDGE_TWO_PHASES)`
stayed green on a kernel that never wrote the line at all.

`WEDGE_PHASES` spelled `toyos_xhci::bot::Phase::named()`'s three words a second
time, in the file whose neighbouring predicate had just been changed to stop
doing that. The table carries `Phase` values now and the judge formats them
through the `Display` that is `named()`, so the harness spells no phase word:
`git grep "data (unqueued)" -- tests/ src/` exits 1. It reads the declaration
rather than a fourth copy held to it by a scan, which makes a reworded phase a
type error; the dev-dependency that allows it is the arrangement `toyos-tco`
and `toyos-logstream` are already there for.

`Report::commit` had no caller outside its crate and is private. The variant
`NotOnTheRing::Unaligned` had no caller that could produce one: the single
caller's base is `block_device + {0, PAGE}` off a block at a multiple of
`MSC_STRIDE = 16 * PAGE` in a pool of whole 2 MiB pages, so every base is a
multiple of sixteen. The variant, its `Display` clause and its test go; the
check itself is folded into the `Dequeue` refusal, because dropping it would
have let a skewed base round to a plausible index, which is the one thing this
module exists not to do.

The section appended to another worker's issue file is reverted. What it
recorded — an account written into the page and never sealed — is what
`toyos_blackbox::Account` and `ACCOUNT_BYTES` fix on this branch, so it is not
open work and gets no file of its own. Measured off the pages run 43 and run 45
left: the reports ran to 13,773 and 13,763 bytes of the 14,296 a report may
spend, leaving 2,571 and 2,581 free, and the account was in those bytes with no
length or checksum over it.

The five prose findings are deleted and none is rewritten. What went: an earlier
implementation's arithmetic narrated in a test doc; an earlier failure offered
as a predicate's justification; "4 MiB", a count the code beneath it
contradicted, with no corrected number put in its place; `Account`'s
restatement of what `commit` already says at the site that seals; and the
narration of `deadline::this_cpu`'s implementation. The three pre-existing
lines the review passed by are untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK
@Japabu
Japabu enabled auto-merge September 14, 2026 20:33
Japabu and others added 3 commits September 16, 2026 14:50
…ightly

Run 34873852892's `durations` job refused with "committed UNMEASURED profile
marker(s) are provisional and may not land: usb_reset_records_the_phase_it_cut"
— its one bought Fast-tier run priced it at 64,260 ms (shards=12) in that
run's own `test-durations-merged` artifact, six times over FAST_CEILING_MS.
`tests/test-durations` carries that measured value instead of the marker,
`tests/toyos.rs` moves the row from Tier::Fast to Tier::Nightly, and
`src/tiers.rs`'s RELEGATED gains its row (Why::TimerAnchored, beside its
sibling usb_reset_hands_devices_back): each of its four boots waits out a
real per-phase deadline, so a faster machine does not shrink it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK
…GREEN

Seen in the same session's fast-tier run, host carrying two other worktrees'
suites throughout: smp_failed_ap_leaves_no_hole (spawn_init: WouldBlock) and
log_reserve_window (root disk missed its 2000 ms budget, so the GPT scan
found no ROOT partition). Both PASS run alone immediately after. Neither is
touched by this branch; filed rather than chased, and filed separately from
issues/build/parallel-tests-red-under-other-suites.md rather than appended
to a file this agent does not own.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014iqcj4jDKpaiDX8B7CMvmK
@Japabu
Japabu added this pull request to the merge queue Sep 16, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to no response for status checks Sep 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant