You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat(telemetry): capture latency percentiles and add rollup query endpoint (#132)
## What
Fills in the telemetry latency percentiles that shipped in V1 as
always-empty wire keys, end to end:
- **Client capture.** A new app-global `LatencyReservoir` in
`driven-core` (bounded ring buffer per metric, cap 4096, nearest-rank
p50/p95). The scanner records per-file scan-processing latency; the
executor records each completed upload op's latency normalized per MiB.
One reservoir is shared (`Arc`) into every account's executor +
orchestrator (mirroring the existing `with_mem_gauge` seam), wired once
in `assembly`.
- **Ping build.** `build_payload` now carries the drained percentiles
instead of the hardcoded `LatencyP50P95::default()`. The window is
snapshotted read-only at build and reset **only after a successful
send**, so a dropped/aborted ping re-uses the same window (matching how
the event-count deltas re-send an un-checkpointed window).
- **Worker ingest.** `writePing` appends the 4 percentiles as AE doubles
(`double7..10`), with a `-1` sentinel for an empty metric so the rollup
can tell "no samples" from a legitimate `0 ms` (a sub-ms per-file scan
rounds to 0).
- **Worker rollup query.** New gated `GET
/telemetry/v1/stats/latency?days=N` returning per-day aggregates via the
Analytics Engine SQL API.
## Why
DESIGN s13 lists "latency histograms (p50, p95) for scan and
upload-per-file" in the payload, but nothing captured per-op durations,
so `latency_p50_p95_ms.{scan,upload_per_mb}` were always empty and the
worker had no read surface. This makes the signal real and queryable.
## Consent / privacy
Capture is gated on the telemetry-enabled pref: the reservoir's enable
flag is initialized from the persisted pref at boot (before any capture)
and flipped in lockstep by `apply_enabled_change`. Turning telemetry off
drops any captured samples. A latency sample is a bare millisecond
duration - no path/name/content.
## Endpoint contract
```
GET /telemetry/v1/stats/latency?days=7
Authorization: Bearer <QUERY_TOKEN>
200 OK
{
"days": 7,
"metrics": {
"scan": [ { "day": "2026-07-14", "avg_p50_ms": 3, "avg_p95_ms": 12, "max_p95_ms": 40, "samples": 9 } ],
"upload_per_mb": [ { "day": "2026-07-14", "avg_p50_ms": 50, "avg_p95_ms": 120, "max_p95_ms": 300, "samples": 4 } ]
}
}
```
`days` defaults to 7, clamped to `[1, 90]`. Per metric, per UTC day:
mean of the pinged p50s, mean + max of the pinged p95s, and the count of
pings that reported the metric. Empty-latency pings (the `-1` sentinel)
are excluded per metric (`WHERE <p50col> >= 0`). Status: `401`
missing/wrong bearer, `405` non-GET, `502` upstream AE failure, `503`
not configured.
## Where things live
- Reservoir: `crates/driven-core/src/telemetry.rs` (new).
- Scan capture: `scanner::scan_with_latency` (thin `scan()` wrapper
keeps existing callers/tests unchanged).
- Upload capture: `ExecOne::run` in `executor.rs`.
- Wiring: `src-tauri/src/assembly.rs`; held on `AppState`'s
`TelemetryRuntime`.
- Ping: `src-tauri/src/telemetry.rs`. Worker:
`telemetry-worker/src/index.ts` + `README.md`.
## Operational caveats (action required post-merge)
- **Set `QUERY_TOKEN` + `CF_API_TOKEN` as wrangler secrets** (see
`telemetry-worker/README.md`). `deploy-telemetry.yml` auto-deploys on
merge, so `/stats/latency` ships **503-until-configured** by design; the
ingest path needs neither. `CF_ACCOUNT_ID` is optional (defaults to the
Driven account).
- **The AE SQL query has only been tested against a mocked `fetch`** -
`toDate()` day-grouping, the `double7..10` column mapping, and
`{meta,data}` parsing have never hit real Analytics Engine. It needs a
one-time live smoke-check after the secrets are set. This is the one
thing the test suite structurally cannot cover.
- **`upload_per_mb` is an op-latency proxy, not pure transfer time:**
the timer wraps the whole upload op (hash -> crypto -> pacer gate ->
network), so on a throttled/metered link it reflects pacer wait, not
Drive throughput. A defensible reading of "upload op"; documented so the
percentiles aren't misread.
## Tests
- Rust (`driven-core`): reservoir + percentile edge cases (empty /
single / even / odd / large-uniform / ring-cap), consent no-op +
disable-drops-samples, `per_mb_ms` normalization + zero guard.
- Rust (`driven-app`): `build_payload` carries the drained percentiles;
ping snapshots then resets on success; a failed send keeps the window; a
disabled ping does not touch the reservoir; `apply_enabled_change`
clears the reservoir on disable.
- Worker (vitest): the 4 latency doubles + `-1` sentinel (legit `0`
preserved); `/stats/latency` 503-unconfigured, 401 no/wrong bearer, 405
wrong method, 200 per-day aggregates (mocked AE `fetch`), `days`
clamping, 502 on upstream failure.
Gates green: `cargo fmt --check`, `clippy --workspace --all-targets -D
warnings`, `cargo test -p driven-core`, `cargo check --workspace`;
worker `typecheck` + `lint` + `test` (56).
## Issue reference
The task said "Closes#5", but `#5` resolves to a **PR**, not an issue
(shared numbering - `gh issue view 5` fails), so "Closes#5" would
attach to a PR and close nothing. Referencing the V2 backlog tracking
issue instead; correct me if a different issue was meant.
Refs #34
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
0 commit comments