Skip to content

Commit 3839778

Browse files
authored
Document scheduled backup recovery (#44)
## Summary - record the 2026-08-03 scheduled-backup root cause and production fix - record guarded recovery and verified replacement R2 backup - record deployment, trigger, test, and integrity evidence ## Validation - `git diff --check`
1 parent dce739b commit 3839778

1 file changed

Lines changed: 31 additions & 0 deletions

File tree

‎docs/V6_PROJECT_LEDGER.md‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1339,3 +1339,34 @@ Bu dosya yalnız sonuçları değil; kararları, reddedilen alternatifleri, migr
13391339
- No D1 migration, production-data mutation, credential operation, cache purge
13401340
or write-bearing live test occurred. Future Plan 007 stage 7, CLI feature
13411341
freeze and the API-only proof window, is next.
1342+
1343+
### 2026-08-03 — Scheduled backup failure isolation and recovery
1344+
1345+
- Investigated daily backup run `019fc59f-f3db-741a-8b82-bf45880fc53e`, which
1346+
remained `running` after the scheduled Worker execution ended. Cloudflare
1347+
Observability showed `D1_ERROR: FOREIGN KEY constraint failed` in identity
1348+
cleanup: an expired idempotency key was still referenced by the immutable
1349+
`media_transform_claims.idempotency_id` foreign key. Because scheduled tasks
1350+
shared `Promise.all`, that cleanup rejection also terminated the backup.
1351+
- Identity cleanup now preserves referenced idempotency keys, scheduled
1352+
maintenance settles and reports each task independently, and a dedicated
1353+
`0 4 * * *` reconciliation cron closes backup runs older than 30 minutes.
1354+
The primary encrypted-backup cron remains `17 3 * * *`; Cloudflare confirms
1355+
both production triggers are active.
1356+
- PR #43 was merged as `dce739bcfd18f2819f26e834f5295144a4375080`.
1357+
Deploy run `30836451493` passed and published deployment
1358+
`2155eb9b-e54c-4393-be61-ccb9f8041751`, Worker version
1359+
`e28b521f-4992-4933-a1d0-a0faada4612f`. Local proof passed 168 D1/workerd
1360+
tests, 19 platform tests, 54 production-config assertions, four Actions-scope
1361+
tests, Astro with zero diagnostics and the production Worker dry-run.
1362+
- Recovery used a guarded update matching only the stale run while it was
1363+
still `running`; exactly one row changed to `failed` with
1364+
`backup_run_stale_timeout`. A fresh encrypted manual backup then succeeded
1365+
as run `019fc8a8-ac16-754c-b8b4-3efea1d1942b`, object
1366+
`orbit-v6/manual/2026-08-03T17-25-24-982Z-019fc8a8-ac16-754c-b8b4-3efea1d1942b.json.enc`,
1367+
manifest checksum `cNG2pVmXBCqb6crvKEM8J-_aH04xJD4hcLx--Szme2c`.
1368+
The private R2 object was downloaded successfully at 246,738 bytes with
1369+
encrypted-object SHA-256
1370+
`53da60dc197cd6adf3fc17b90e52f97aa1d8d575b3448abd899a38c1a6564c42`.
1371+
Production finished with zero `running` backup rows, an empty
1372+
`PRAGMA foreign_key_check`, and a healthy `/healthz` response.

0 commit comments

Comments
 (0)