Skip to content

feat(ui): live disk/network throughput graphs + visible upload recovery - #290

Merged
pmaxhogan merged 3 commits into
mainfrom
feat/io-throughput-and-recovering
Aug 14, 2026
Merged

pmaxhogan merged 3 commits into
mainfrom
feat/io-throughput-and-recovering

Conversation

@pmaxhogan

Copy link
Copy Markdown
Owner

Two gaps the 2026-08-14 incident's follow-up diagnostics bundle exposed on v2.10.1: a resumed 88 GB upload pushed 140 Mbps while every throughput surface read zero (the Activity graph is an activity-log SQL window and the resume writes no rows until it completes), and the entire reconcile phase sat in an unlabeled indeterminate "Starting backup..." sweep.

What this adds

Live split throughput graphs. One app-global cumulative IoCounters (driven-core) credited by every executor: plaintext disk-read bytes (upload reader stage, resume re-read, reconcile re-hash paths, buffered small-file band) and destination-acked network wire bytes (per ack for resumable sessions - including the resume, which previously credited nothing - and on completion for single-request uploads; bundles by their session acks alone, never double-counted). A 1s sampler task in the app shell (updater-lifecycle pattern, quit-drained, idle-suppressed after one trailing zero) diffs the counters into a 5-minute ring, served by the new io_throughput_series command and pushed live via sync:io_throughput. The Activity header's throughput tile goes live and splits into Network throughput and Disk read variants of one component - both move in real time through every backup phase, including crash recovery.

Visible upload recovery. New OrchestratorState::Recovering { source_id, path, bytes_done, bytes_total }: Executor::reconcile gains a RecoverProgressSink (mirroring execute's on_progress), emitted at resume start - before the prefix re-read produces any acks, so the UI flips out of the sweep immediately - and on every ack, throttled to ~1/s in the orchestrator on the injected clock (first and final ticks always pass; unit-tested). The global progress bar renders it as a determinate byte bar: "Recovering interrupted upload - 8.2 GB of 88.6 GB". Tray maps it to the syncing bucket.

Log-noise fix ridealong: the state-transition INFO log now gates on the variant name, so counter-carrying states (Scanning/Recovering) no longer log every tick (12 lines in 60 ms observed in the follow-up bundle).

Known limitations (deliberate, documented in-code)

  • After the resume finalizes, adopt_reconciled's P1-2 re-hash is a second full disk pass - for an 88 GB file the bar holds at 100% for a few minutes while it runs. The new disk graph shows exactly that activity, so the behavior is now legible; collapsing the second pass for the just-resumed case is a tracked follow-up.
  • Scanner deep-verify hashing and the restore path don't credit disk_read yet; bundle assembly reads are approximated by their wire push.
  • The adaptive ThroughputProbe deliberately still ignores resume bytes (its pool-sizing semantics against an idle upload pool are unverified).
  • The macOS menubar shows idle metrics during Recovering (it is fed only by Progress events) - cosmetic, macOS-only.
  • No visual-test scenario for Recovering (covered by unit tests); the mock backend never emits live IO events, so the new tiles render a fixed deterministic fixture in the visual suite.

Tests

Rust: IoCounters accounting; resume emits monotonic recover ticks ending at completion; resume credits disk (2x: re-read + adopt re-hash) and exactly the unacked tail on the wire; orchestrator throttle (5 stationary-clock ticks -> exactly first + final Recovering states); sampler tick delta/ring/idle-suppression. UI: iostat store seed/live/cap/rate math + garbled-wire coercion; recovering phase precedence + determinate percent; progress-bar recovering labels + aria; disk tile variant + default-unchanged; tile order. Full workspace suites, 788 UI unit tests, and all 102 visual tests (baselines regenerated in the same Docker harness and committed) are green locally.

README updated in this PR (Activity graphs, visible recovery, diagnostics-bundle contents).

Note for release sequencing: this is a feat, so the next release is 2.11.0 (it also carries #288's Zone.Identifier fix).

🤖 Generated with Claude Code

https://claude.ai/code/session_01A7q3CvJzL4zZmDA9CbXyQQ

pmaxhogan and others added 2 commits August 14, 2026 17:54
Two gaps the 2026-08-14 incident's follow-up bundle exposed: a resumed 88 GB
upload pushed 140 Mbps while every throughput surface read zero (the Activity
graph is an activity-log SQL window, and the resume writes no rows until it
completes), and the whole reconcile phase sat in an unlabeled indeterminate
"Starting backup..." sweep.

- driven-core: app-global cumulative IoCounters (plaintext disk-read bytes;
  destination-acked wire bytes, credited exactly once per path - per ack for
  resumable sessions incl. the resume, on completion for single-request
  uploads; bundles by their session acks alone). New
  OrchestratorState::Recovering carries the resume's path + byte totals;
  Executor::reconcile takes a RecoverProgressSink, emitted at resume start
  (before the prefix re-read produces any acks) and per ack, throttled ~1/s
  in the orchestrator on the injected clock. The state-transition INFO log
  now gates on the variant NAME so counter-carrying states no longer spam
  (12 lines/60ms observed in the follow-up bundle).
- src-tauri: a 1s sampler task (updater-lifecycle pattern, quit-drained)
  diffs the counters into a 5-minute ring, emits `sync:io_throughput`
  (idle-suppressed after one trailing zero), and serves the ring via the new
  `io_throughput_series` command. Recovering maps to the syncing bucket in
  the tray.
- ui: the Activity header's throughput tile goes live (probe-fed 1s buckets)
  and splits into Network + Disk read variants of one component; the global
  progress bar renders Recovering as a DETERMINATE byte bar ("Recovering
  interrupted upload - 8.2 GB of 88.6 GB"). Visual baselines regenerated.

v1 scope notes (documented in-code): scanner deep-verify hashing and restore
do not credit disk_read yet; bundle assembly reads are approximated by their
wire push; the adaptive ThroughputProbe deliberately still ignores resume
bytes (its pool-sizing semantics under an idle pool are unverified); the
post-resume adopt re-hash is a second full disk pass (tracked follow-up).

Tests: IoCounters accounting; resume emits monotonic recover ticks ending at
completion; resume credits disk (2x: re-read + adopt re-hash) and exactly the
unacked tail on the wire; sampler tick delta/ring/idle-suppression math;
iostat store seed/live/cap/rates; recovering phase precedence + determinate
percent; progress-bar recovering labels; disk tile variant; tile order. Full
workspace + 788 UI tests + 102 visual green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A7q3CvJzL4zZmDA9CbXyQQ
…ks pass)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01A7q3CvJzL4zZmDA9CbXyQQ
@github-actions

Copy link
Copy Markdown
Contributor

Coverage

Area main this PR delta
Rust (lib crates) 84.58% 84.52% -0.05 (OK)
UI (vue/ts) 93.11% 93.13% +0.02 (OK)

Gate: passed - no coverage regression (epsilon 0.1 pp).

@pmaxhogan
pmaxhogan merged commit 4fe156c into main Aug 14, 2026
18 checks passed
@pmaxhogan
pmaxhogan deleted the feat/io-throughput-and-recovering branch August 14, 2026 23:43
@github-project-automation github-project-automation Bot moved this from Todo to Done in Driven Aug 14, 2026
pmaxhogan added a commit that referenced this pull request Aug 15, 2026
🤖 I have created a release *beep* *boop*
---


##
[2.11.0](v2.10.1...v2.11.0)
(2026-08-14)


### Features

* **ui:** live disk/network throughput graphs + visible upload recovery
([#290](#290))
([4fe156c](4fe156c))


### Bug Fixes

* **core:** exempt Zone.Identifier from the ads_skipped warning
([#288](#288))
([f37ff68](f37ff68))
* let a slow add-source wizard finish instead of failing with a phantom
disk error ([#291](#291))
([1fbba98](1fbba98))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
pmaxhogan added a commit that referenced this pull request Aug 15, 2026
The v2.11.0 tag's release pipeline failed at the visual-regression gate
(setup-wizard baselines predating #291) before publishing any artifacts, so
no 2.11.0 build ever shipped. The baselines were regenerated in #292; this
empty commit gives release-please a releasable change so 2.11.1 carries the
full 2.11.0 payload (#288, #290, #291) through a green pipeline.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A7q3CvJzL4zZmDA9CbXyQQ
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant