Skip to content

Commit d0df17f

Browse files
pmaxhoganclaude
andcommitted
feat(core): adaptive upload parallelism with throughput probe and disk-saturation gate
Close the control loop around the in-flight-file count (DESIGN 11.4.7 / 18.2). A 30s-window ThroughputProbe fed by the executor drives a resizable UploadPool within [1, 32]: shrink when a window's throughput collapses below 50% of the prior one (and neither the pacer nor the disk explains the drop), grow while lifting the pool still improves throughput. A per-OS disk-busy gate (new driven-diskstat crate: PDH on Windows, /proc/diskstats on Linux, IOKit on macOS; fail-open to not-saturated so a broken reader never strangles uploads) blocks growth when the disk is the bound. Default-on with an adaptive_parallelism_enabled kill-switch; off = today's fixed pool at default_concurrent_uploads (now wired as the start size). The control law is a pure, exhaustively unit-tested decide(); an e2e_fake test asserts a real shrink-then-regrow transition through the shared executor/probe/pool wiring. Refs #34 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QZQVP2tUuTLh8oL31D8heC
1 parent fa9d622 commit d0df17f

27 files changed

Lines changed: 2209 additions & 60 deletions

File tree

Cargo.lock

Lines changed: 12 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ members = [
55
"crates/driven-drive",
66
"crates/driven-crypto",
77
"crates/driven-power",
8+
"crates/driven-diskstat",
89
"crates/driven-vss",
910
"crates/driven-vss-helper",
1011
"crates/driven-net",

crates/driven-core/Cargo.toml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,10 @@ sqlx = { version = "0.9", default-features = false, features = [
4747
driven-drive = { path = "../driven-drive" }
4848
driven-crypto = { path = "../driven-crypto" }
4949
driven-power = { path = "../driven-power" }
50+
# Per-OS disk-busy reader for the adaptive upload-parallelism controller
51+
# (DESIGN s11.4.7 / s18.2). Same acyclic shape as driven-power: it carries no
52+
# driven-core dep.
53+
driven-diskstat = { path = "../driven-diskstat" }
5054
# M3.5 Windows VSS reads for exclusively-locked files (ROADMAP M3.5, DESIGN
5155
# s5.3). The executor consults the `VssProvider` seam on its open path; the
5256
# orchestrator owns the per-cycle snapshot lifecycle + orphan cleanup. The

0 commit comments

Comments
 (0)