Skip to content

Commit 0097587

Browse files
sametbasbugclaude
andcommitted
Record how deleting a dead secret broke staging
A secret is dead relative to the deployed artifact, not to main. Deleting the GitHub OAuth secrets minted a new Worker version that inherited staging's 12:10 code — code that still required them — so every /v1 route failed closed for two nights. Production was spared only because it had already been redeployed onto the Google-only build. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 310ea4e commit 0097587

1 file changed

Lines changed: 25 additions & 0 deletions

File tree

‎docs/V6_PROJECT_LEDGER.md‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2102,3 +2102,28 @@ Anything under `docs/archive/` describes an earlier state and is frozen. Read it
21022102
@astrojs/cloudflare 14.2.0 are the only ones that reach production;
21032103
@astrojs/check still accepts this astro, so the peer range that keeps
21042104
TypeScript 7 out has not moved.
2105+
- Staging was broken for two nights and the nightly said so both nights before
2106+
anyone read it. Every route under `/v1` answered 500 — including paths that
2107+
do not exist, which is the tell: the throw happened at the API entry, before
2108+
routing, so it was a binding assertion and not a handler. `/healthz` and the
2109+
static pages were fine, which is why nothing looked wrong from outside.
2110+
- The cause was mine, and the shape of it is worth keeping. On 10 August at
2111+
14:01 I deleted the two dead GitHub OAuth secrets. Deleting a secret mints a
2112+
new Worker version that inherits the previous code; the version that resulted
2113+
still carried `ORBIT_GITHUB_CALLBACK_URL` and `ORBIT_PLATFORM_OWNER_GITHUB_ID`,
2114+
because staging's last real deploy was 12:10, before GitHub was removed. That
2115+
code still listed the GitHub secrets as required bindings. So the new version
2116+
was old code missing a secret it demanded, and it failed closed on every
2117+
authenticated surface.
2118+
- Production survived the same deletion only because it had been redeployed at
2119+
13:40 with the Google-only code. The secrets were removed from both; one of
2120+
them was ready and the other was not.
2121+
- The mistake underneath: I decided the secrets were dead by reading `main`. A
2122+
secret is dead relative to the **deployed artifact**, not to the repository.
2123+
Anything that mutates secrets should be followed by that environment's
2124+
verification run, because a secret change is a deployment — it produces a new
2125+
version — while looking like configuration.
2126+
- Redeploying the identical commit fixed it, which is also the proof that
2127+
neither the code nor `wrangler.staging.jsonc` was ever wrong: `npm run
2128+
staging:verify` passes, the forbidden-origin probe is 403 again and an unknown
2129+
`/v1` path is 404 again.

0 commit comments

Comments
 (0)