feat(telemetry): capture latency percentiles and add rollup query endpoint - #132
Merged
Merged
Conversation
…point Client (driven-core + src-tauri): - New driven-core telemetry module: an app-global LatencyReservoir (bounded ring buffer per metric, cap 4096) with nearest-rank p50/p95, consent-gated capture, snapshot (read-only) + reset (post-send). - Scanner records per-file scan-processing latency; executor records per-completed-upload latency normalized per MiB. - Orchestrator + executor share the reservoir (with_latency_reservoir); wired once in assembly, enable-gate initialized from the persisted pref and flipped in lockstep by apply_enabled_change. - Ping build_payload now carries the drained percentiles; the window resets only after a SUCCESSFUL send so a dropped ping re-uses it. Worker (telemetry-worker): - writePing appends scan/upload_per_mb p50/p95 as AE doubles (7..10), with a -1 sentinel for an empty metric (distinguishes "no samples" from a legit 0 ms). - New gated GET /telemetry/v1/stats/latency?days=N (default 7, cap 90): Bearer QUERY_TOKEN, per-day aggregates via the AE SQL API (CF_API_TOKEN); 503 until configured. README documents the secrets + contract. Tests: reservoir/percentile edge cases + reservoir<->ping integration (rust); latency doubles + stats endpoint auth/clamp/aggregate (vitest, mocked AE fetch). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QZQVP2tUuTLh8oL31D8heC
Contributor
Coverage
Gate: passed - no coverage regression (epsilon 0.1 pp). |
This was referenced Jul 20, 2026
pmaxhogan
added a commit
that referenced
this pull request
Jul 20, 2026
## What Follow-up to #132. Adds a schema-version marker to every latency row and filters on it in the rollup query, so pre-#132 rows are excluded from `/telemetry/v1/stats/latency`. ## Why (the AE default-0 gotcha) The Analytics Engine SQL API has **no NULLs**: any `double` a row never wrote is materialized as **`0`** at query time. Rows written by the pre-latency Worker (before #132) have no `double7..10`, so they read `scan_p50 == 0` — a materialized 0, not a real sample. That passed the rollup's `WHERE <p50col> >= 0` sentinel filter and showed up as legit `0 ms` samples. Caught in the live smoke of the endpoint: a day that predates the deploy reported `samples > 0, avg 0`. The `-1` sentinel can't fix this on its own — it only distinguishes empty from present *within* a row that actually wrote the doubles; a legacy row never wrote them, so there's no sentinel to read. ## Fix - **Write** (`writePing`): append a schema-version marker `double11 = 1` (`LATENCY_SCHEMA_VERSION`) on every row that carries the latency doubles. - **Query** (`queryLatencyMetric`): add `AND double11 >= 1` to each per-metric `WHERE` (in addition to the existing `>= 0` sentinel). A legacy row materializes the marker as `0` and is excluded; a new empty-latency row is marked (`double11 = 1`) but still excluded by its `-1` sentinel; a real `0 ms` still counts. - Both sites comment the AE missing-double=0 behavior. README dataset table + rollup section updated (`double11` row, dual-filter explanation). ## Tests (worker vitest) - `writePing` marks every new latency-schema row with `double11 = 1`. - A new-but-empty-latency row is marked (`double11 = 1`) yet still carries the `-1` sentinels — proving the marker and sentinel filters are independent (the sentinel is what excludes it). - Both per-metric rollup queries include `double11 >= 1` (the mechanism that excludes pre-latency rows, whose marker materializes as 0). Gates green: worker `typecheck` + `lint` + `test` (58, +2 new). ## Notes Cut from latest `origin/main` (which already includes #132). Endpoint still needs the same wrangler secrets as #132; the live smoke that surfaced this bug already has them set. Do not merge — for review. Refs #34 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
23 tasks
pmaxhogan
added a commit
that referenced
this pull request
Jul 24, 2026
🤖 I have created a release *beep* *boop* --- ## [2.1.0](v2.0.1...v2.1.0) (2026-07-24) ### Features * **core:** adaptive upload parallelism with throughput probe and disk-saturation gate ([#143](#143)) ([8ecced6](8ecced6)) * **core:** filesystem timestamp-granularity probe with ctime fallback and per-directory gitignore cascade ([#141](#141)) ([344262c](344262c)) * **drive:** support Google Shared Drive destinations end-to-end ([#142](#142)) ([d9c3161](d9c3161)) * **net:** native OS reachability backends with automatic fallback ([#138](#138)) ([319e85f](319e85f)) * **net:** SOCKS5 and PAC proxy support for all outbound connections ([#145](#145)) ([2f0b7d1](2f0b7d1)) * **net:** support a custom corporate root CA for all outbound connections ([#134](#134)) ([929e93d](929e93d)) * per-source toggle to back up OneDrive cloud-only placeholder files ([#133](#133)) ([6863ea3](6863ea3)) * **telemetry:** capture latency percentiles and add rollup query endpoint ([#132](#132)) ([4e9fde6](4e9fde6)) * **telemetry:** preview exactly what a telemetry ping sends ([#139](#139)) ([95fbd9a](95fbd9a)) ### Bug Fixes * **core:** commit file_state for a create that skipped post-upload so the next scan updates instead of re-creating ([#146](#146)) ([f5230d1](f5230d1)) * **deps:** bump tauri-winrt-notification to drop vulnerable quick-xml (closes [#89](#89)) ([#129](#129)) ([232fd8f](232fd8f)) * **telemetry:** exclude pre-schema rows from latency rollup ([#137](#137)) ([1ae6220](1ae6220)) * **ui:** add cursor pointer to buttons and link-buttons ([#136](#136)) ([dbd4809](dbd4809)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Fills in the telemetry latency percentiles that shipped in V1 as always-empty wire keys, end to end:
LatencyReservoirindriven-core(bounded ring buffer per metric, cap 4096, nearest-rank p50/p95). The scanner records per-file scan-processing latency; the executor records each completed upload op's latency normalized per MiB. One reservoir is shared (Arc) into every account's executor + orchestrator (mirroring the existingwith_mem_gaugeseam), wired once inassembly.build_payloadnow carries the drained percentiles instead of the hardcodedLatencyP50P95::default(). The window is snapshotted read-only at build and reset only after a successful send, so a dropped/aborted ping re-uses the same window (matching how the event-count deltas re-send an un-checkpointed window).writePingappends the 4 percentiles as AE doubles (double7..10), with a-1sentinel for an empty metric so the rollup can tell "no samples" from a legitimate0 ms(a sub-ms per-file scan rounds to 0).GET /telemetry/v1/stats/latency?days=Nreturning per-day aggregates via the Analytics Engine SQL API.Why
DESIGN s13 lists "latency histograms (p50, p95) for scan and upload-per-file" in the payload, but nothing captured per-op durations, so
latency_p50_p95_ms.{scan,upload_per_mb}were always empty and the worker had no read surface. This makes the signal real and queryable.Consent / privacy
Capture is gated on the telemetry-enabled pref: the reservoir's enable flag is initialized from the persisted pref at boot (before any capture) and flipped in lockstep by
apply_enabled_change. Turning telemetry off drops any captured samples. A latency sample is a bare millisecond duration - no path/name/content.Endpoint contract
daysdefaults to 7, clamped to[1, 90]. Per metric, per UTC day: mean of the pinged p50s, mean + max of the pinged p95s, and the count of pings that reported the metric. Empty-latency pings (the-1sentinel) are excluded per metric (WHERE <p50col> >= 0). Status:401missing/wrong bearer,405non-GET,502upstream AE failure,503not configured.Where things live
crates/driven-core/src/telemetry.rs(new).scanner::scan_with_latency(thinscan()wrapper keeps existing callers/tests unchanged).ExecOne::runinexecutor.rs.src-tauri/src/assembly.rs; held onAppState'sTelemetryRuntime.src-tauri/src/telemetry.rs. Worker:telemetry-worker/src/index.ts+README.md.Operational caveats (action required post-merge)
QUERY_TOKEN+CF_API_TOKENas wrangler secrets (seetelemetry-worker/README.md).deploy-telemetry.ymlauto-deploys on merge, so/stats/latencyships 503-until-configured by design; the ingest path needs neither.CF_ACCOUNT_IDis optional (defaults to the Driven account).fetch-toDate()day-grouping, thedouble7..10column mapping, and{meta,data}parsing have never hit real Analytics Engine. It needs a one-time live smoke-check after the secrets are set. This is the one thing the test suite structurally cannot cover.upload_per_mbis an op-latency proxy, not pure transfer time: the timer wraps the whole upload op (hash -> crypto -> pacer gate -> network), so on a throttled/metered link it reflects pacer wait, not Drive throughput. A defensible reading of "upload op"; documented so the percentiles aren't misread.Tests
driven-core): reservoir + percentile edge cases (empty / single / even / odd / large-uniform / ring-cap), consent no-op + disable-drops-samples,per_mb_msnormalization + zero guard.driven-app):build_payloadcarries the drained percentiles; ping snapshots then resets on success; a failed send keeps the window; a disabled ping does not touch the reservoir;apply_enabled_changeclears the reservoir on disable.-1sentinel (legit0preserved);/stats/latency503-unconfigured, 401 no/wrong bearer, 405 wrong method, 200 per-day aggregates (mocked AEfetch),daysclamping, 502 on upstream failure.Gates green:
cargo fmt --check,clippy --workspace --all-targets -D warnings,cargo test -p driven-core,cargo check --workspace; workertypecheck+lint+test(56).Issue reference
The task said "Closes #5", but
#5resolves to a PR, not an issue (shared numbering -gh issue view 5fails), so "Closes #5" would attach to a PR and close nothing. Referencing the V2 backlog tracking issue instead; correct me if a different issue was meant.Refs #34
🤖 Generated with Claude Code