fix(core): commit file_state for a create that skipped post-upload so the next scan updates instead of re-creating - #146
Merged
Conversation
Contributor
Coverage
Gate: passed - no coverage regression (epsilon 0.1 pp). |
… the next scan updates instead of re-creating When a plain first-CREATE upload's local file changes during the hash->upload->post-check window, the executor returned SkipPostUpload and kept the pending op with NO file_state row, relying on the startup-gated reconcile pass to adopt the just-created object by op-uuid. Mid-run nothing adopts it, so the next scan saw a row-less path and planned a SECOND create - duplicating live objects until the next app restart (issue #144; the chaos frequent-edits gate saw ~20 duplicate hot.txt objects under a 3ms store delay). Make create-SkipPostUpload symmetric with reconcile's adopt-requeue arm: for a plain create (no pre-existing row, not a versioned change), settle_post_upload_change durably commits the SAME force-rescan file_state row that arm would eventually write - drive_file_id = the just-created object, mtime_ns = REQUEUE_FORCE_RESCAN_MTIME_NS (i64::MIN) so the next FastPath scan always re-emits the path, the uploaded (now-stale) blake3, the on-Drive md5/size, status Pending - via commit_create_result, which upserts the row and deletes the op in ONE transaction. The next scan then re-uploads as an UPDATE against the same object: exactly one live object per path, no restart required. Design deviates slightly from the issue's "thread the id through the error" sketch: the object id is available at the detection point, so the helper commits in place and returns Ok(Skipped) - a smaller seam into the hardened UploadError enum with an identical durable outcome. Invariants preserved: - Durable-commit ordering / atomicity: the row upsert + op delete are one transaction (commit_create_result), so no split write. - Crash-safety fallback: a crash before the commit leaves the op (still carrying its op-uuid) with no row, so the pre-#144 orphan+reconcile adoption runs on the next boot exactly as before. - Reconcile idempotency: the commit deletes the op atomically with the row, so reconcile never finds this op again - no double-adopt. - In-place UPDATE and versioned-create keep the pre-#144 SkipPostUpload path (their existing row already points at an object; the next scan updates or versions against it, never re-creating the path). - Encryption: the created id + ciphertext encrypted_remote_path flow into the row identically to a plaintext source. - No-delete-on-metadata-error and the DeferToReconcile ambiguous-create path are untouched (that path has no known id to point at, so it stays reconcile-driven). Tests: - create_changed_after_upload_commits_force_rescan_row_no_duplicate: the deterministic race (post-upload hook) then the NEXT scan in the same process UPDATEs the same object id - exactly one live object. - create_skip_post_upload_row_survives_reconcile_no_double_adopt: restart mid-window - the committed row + a reconcile pass stays one object. - update_changed_after_upload_keeps_existing_id_no_duplicate: the update carve-out is unchanged. - create_changed_after_upload_encrypted_commits_force_rescan_row: the encrypted path. - frequent_edits_slow_store_holds_one_object_144: the chaos frequent-edits scenario with a 3ms injected store delay holds exactly one object. Closes #144 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QZQVP2tUuTLh8oL31D8heC
pmaxhogan
force-pushed
the
fix/create-skippostupload-orphan
branch
from
July 24, 2026 21:34
139efe3 to
f58fcf4
Compare
pmaxhogan
enabled auto-merge (squash)
July 24, 2026 21:57
pmaxhogan
added a commit
that referenced
this pull request
Jul 24, 2026
🤖 I have created a release *beep* *boop* --- ## [2.1.0](v2.0.1...v2.1.0) (2026-07-24) ### Features * **core:** adaptive upload parallelism with throughput probe and disk-saturation gate ([#143](#143)) ([8ecced6](8ecced6)) * **core:** filesystem timestamp-granularity probe with ctime fallback and per-directory gitignore cascade ([#141](#141)) ([344262c](344262c)) * **drive:** support Google Shared Drive destinations end-to-end ([#142](#142)) ([d9c3161](d9c3161)) * **net:** native OS reachability backends with automatic fallback ([#138](#138)) ([319e85f](319e85f)) * **net:** SOCKS5 and PAC proxy support for all outbound connections ([#145](#145)) ([2f0b7d1](2f0b7d1)) * **net:** support a custom corporate root CA for all outbound connections ([#134](#134)) ([929e93d](929e93d)) * per-source toggle to back up OneDrive cloud-only placeholder files ([#133](#133)) ([6863ea3](6863ea3)) * **telemetry:** capture latency percentiles and add rollup query endpoint ([#132](#132)) ([4e9fde6](4e9fde6)) * **telemetry:** preview exactly what a telemetry ping sends ([#139](#139)) ([95fbd9a](95fbd9a)) ### Bug Fixes * **core:** commit file_state for a create that skipped post-upload so the next scan updates instead of re-creating ([#146](#146)) ([f5230d1](f5230d1)) * **deps:** bump tauri-winrt-notification to drop vulnerable quick-xml (closes [#89](#89)) ([#129](#129)) ([232fd8f](232fd8f)) * **telemetry:** exclude pre-schema rows from latency rollup ([#137](#137)) ([1ae6220](1ae6220)) * **ui:** add cursor pointer to buttons and link-buttons ([#136](#136)) ([dbd4809](dbd4809)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
pmaxhogan
added a commit
that referenced
this pull request
Jul 24, 2026
#147) ## Why Main CI run 30129432441 (post-#146 merge) flaked on windows-latest: `scan_reports_probe_only_when_unpersisted` asserted the probe returns exactly `Some(0)` but got `Some(2511100)` (~2.5ms). NTFS stamps file times from the interrupt-time clock, which advances in ~0.5-16ms ticks. When the probe's 8-rewrite burst lands inside a single tick, phase 2's first 2ms sleep sees the mtime advance and correctly reports a millisecond-scale window - a coin flip on fast runners. The probe is right; the assertion was too strict. `probe_classifies_local_fs_fine` had the same latent `== 0` assertion. ## Fix Both tests now assert the probed granularity is present and **non-coarse** (`!granularity_is_coarse(g)`, i.e. <= 1s) - the boundary that actually drives behavior (the re-hash fallback) - instead of exactly zero. Test-only change; no production code touched. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_016YnYLjHgYhhnJjUxojsRhn Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
pmaxhogan
added a commit
that referenced
this pull request
Jul 29, 2026
…e append-only-log soak (#192) ## TL;DR - the orphan race is already fixed; what was missing is a gate that proves it The brief asked for a `fix(core)` closing the #144 create-orphan race behind the `append-only-log` chaos flake. I reproduced the race, then established that **PR #146 (merged 2026-07-24) already closed it**. The stale piece was the scenario itself: it only hit the racy code path by luck, so it neither caught the bug reliably before the fix nor proves the fix now. This PR makes it deterministic. **No core behaviour changes** - hence `test(chaos):`, not `fix(core):`. ## What the race is The first upload of `app.log` is a plain CREATE (no `drive_file_id` yet). An append that lands between the hash and the SPEC s8 post-upload `fstat` makes the executor settle it as changed-after-upload. Pre-#146 that left a LIVE object with no `file_state` row, adoptable only by the startup-gated reconcile pass, so the next mid-session scan planned a SECOND create. The scenario's "after reconcile, expected exactly 1 log object, found N" assertion is exactly that duplicate showing up. Since #146 the executor instead commits a force-rescan `file_state` row pointing at the just-created object and drops the op in one transaction (`settle_post_upload_change` -> `commit_create_result`), so the next scan UPDATEs that object. One object, no restart needed. ## Evidence (M4 Mac, debug build, 4-6 way parallel, `driven-chaos scenario run append-only-log`) I gated the #146 settle behind a temporary env switch to get a true negative control, then removed it. | build | injected remote delay | runs | pass | duplicate-object failures | |---|---|---|---|---| | main (with #146) | none | 690 | 690 | **0** | | main (with #146) | 3 ms | 240 | 240 | **0** | | main (with #146) | 10 ms | 20 | 20 | **0** | | #146 settle reverted | none | 200 | 194 | **6** (3.0%) | | #146 settle reverted | 3 ms | 20 | 0 | **20** (100%, 2-14 objects each) | Two things fall out of that table: 1. The historical ~5-10% flake is reproduced at 3% on this host with the fix reverted, and is gone with the fix in - so the flake really was #144, and #146 really closed it. 2. Without an injected delay the scenario reaches the racy code path only occasionally. A green run therefore did **not** mean the create-orphan path was tested; it usually meant the window never opened. That is the actual remaining defect. ## The change - `AppendOnlyLog` now boots over `InMemoryRemoteStore::with_slow_responses(3 ms)` (`CREATE_RACE_DELAY`), deliberately under the 4 ms `MUTATE_EVERY` mutation cadence, so at least one append lands inside the create's upload window on every run. This is the same widening technique the `mid-upload-*` rows already use with `SLOW_REMOTE`. - Refreshed the two stale comment blocks that still described the pre-#146 world ("leaves an orphan ... which is accepted behaviour"). The reconcile + drain step stays - it now stands in for the app restart that backstops the arms #146 deliberately left to reconcile (crash between upload and settle, versioned create, ambiguous `DeferToReconcile`) rather than papering over a routine duplicate. Cost: +246 ms of wall clock per run of one scenario on the Windows CI runner (measured below). ## Verification on windows-latest (the platform the flake was reported on) The measurements above are from macOS, so here is the CI run of this branch's `chaos hermetic (windows-latest)` job: - `append-only-log`: **pass**, `duration_ms: 523`. Baseline for the same scenario on the same runner image, from an unmodified branch (#193's run): `duration_ms: 277`. So the real cost of forcing the window open is **+246 ms** on the 2-core Windows runner, not the ~150 ms I estimated from the Mac. - Whole suite: **75 PASS / 10 SKIP / 0 FAIL / 0 FLAKY**, so widening this row's window does not destabilise the drain cap or any sibling row. - `chaos fake-drive (windows-latest)` also green (this row is hermetic-only, but the sibling `mutator-fs-append-only-log` runs there: pass, unchanged). ## Verification on macOS - `driven-chaos scenario run append-only-log` x **300 consecutive runs: 300 pass, 0 fail** with this change applied. - `driven-chaos run-all --hermetic`: 54 PASS / 31 SKIP / 0 FAIL / 0 FLAKY. - `driven-chaos run-all --fault-injection`: 25 PASS / 2 SKIP / 0 FAIL / 0 FLAKY. - `SQLX_OFFLINE=true cargo test --workspace`: green. - `cargo clippy --workspace --all-targets -- -D warnings`: clean. - `cargo fmt --all -- --check`: clean. ## Follow-up worth knowing The `driven-append-only-log-chaos-flake` triage note ("re-run the Chaos job, it clears ~85-90% of the time") predates #146 and is now wrong: a duplicate-object failure on this row should be treated as a **real regression**, not a re-run. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Mechanism (#144)
When a plain first-CREATE upload's local file changes during the hash->upload->post-check window, the executor returned
SkipPostUploadand kept the pending op with nofile_staterow, relying on the startup-gated reconcile pass to adopt the just-created object by op-uuid. Mid-run nothing adopts it, so the next scan saw a row-less path and planned a second create - duplicating live objects until the next app restart. The chaosfrequent-editsgate saw ~20 duplicatehot.txtobjects under a 3ms injected store delay (found during #141's CI run; pre-existing in v2.0.1, not introduced by #141).Fix - symmetry with reconcile's adopt-requeue arm
upload_and_commit's post-upload identity recheck now routes throughsettle_post_upload_change. For a plain create (no pre-existing row, not a versioned change) it durably commits the same force-rescanfile_staterow that reconcile'sadopt_reconciledrequeue arm would eventually write:drive_file_id= the just-created object,mtime_ns = REQUEUE_FORCE_RESCAN_MTIME_NS(i64::MIN) so the next FastPath scan always re-emits the path,Pending,via
commit_create_result, which upserts the row and deletes the op in one transaction. The next scan re-uploads as an UPDATE against the same object: exactly one live object per path, no restart required.Design note
The issue sketched "thread the created id through the error path". The id is already in scope at the detection point, so the helper commits in place and returns
Ok(Skipped)- a smaller seam into the hardenedUploadErrorenum with an identical durable outcome.Invariants preserved
commit_create_result) - no split write.crash_mid_upload_adopts_orphan_without_duplicate).SkipPostUploadpath (their existing row already points at an object; the next scan updates/versions against it, never re-creating the path).encrypted_remote_pathflow into the row identically to a plaintext source.Tests
create_changed_after_upload_commits_force_rescan_row_no_duplicate- deterministic race, then the NEXT scan in the same process UPDATEs the same id (exactly one live object).create_skip_post_upload_row_survives_reconcile_no_double_adopt- restart mid-window: committed row + reconcile pass stays one object.update_changed_after_upload_keeps_existing_id_no_duplicate- update carve-out unchanged.create_changed_after_upload_encrypted_commits_force_rescan_row- encrypted path.frequent_edits_slow_store_holds_one_object_144- chaos frequent-edits with a 3ms injected store delay holds exactly one object.Full
driven-corelib suite (336) +e2e_fakeacceptance suite (24) + clippy green.Closes #144
🤖 Generated with Claude Code
https://claude.ai/code/session_01QZQVP2tUuTLh8oL31D8heC