Skip to content

Commit 1837e38

Browse files
sametbasbugclaude
andcommitted
Run the staging rehearsal on a schedule instead of on memory
Nothing ran the staging scripts automatically. One of them broke when a publication guardrail landed and stayed broken for three weeks, because the only thing standing between a rotting script and a green repo was somebody remembering to run it. The nightly regression now rehearses staging after the application checks pass. The peppers those scripts need live in a macOS Keychain, which a Linux runner does not have, so they are read from the environment first and fall back to the Keychain. That order matters: a stray local variable must never quietly outrank the Keychain on Samet's machine. A missing credential is reported by name before anything runs. A job that fails for want of a secret looks exactly like a job that fails for a broken contract, and a red that cannot be read is a red that gets ignored. Only what was measured green is wired in. slice2 asks a sponsor to create an agent, an endpoint removed on 22 July when agents began registering themselves. slice3 asserts the exact first two records of the feed, so any other rehearsal that leaves something published turns it red on its own. Both stay runnable by hand until we decide whether to rewrite or retire them; wiring them in red would teach us to ignore the job we just built. The nightly does not deploy to staging. Staging is the gate Samet drives by hand, and overwriting it at four in the morning would delete whatever he left there to look at. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 080fe8a commit 1837e38

6 files changed

Lines changed: 139 additions & 43 deletions

‎.github/workflows/full-regression.yml‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,3 +37,73 @@ jobs:
3737
npm run production:config:check
3838
npm run worker:build:production:live
3939
npm run contract:live
40+
41+
# Staging provaları hiçbir yerde otomatik koşmuyordu ve bir tanesi üç
42+
# hafta boyunca bozuk kaldı: sunucuya yeni bir yayın kısıtı girdi, betik
43+
# ondan habersiz kaldı, kimse çalıştırmadığı için kimse görmedi.
44+
#
45+
# Bu iş staging'e YENİ KOD DAĞITMIYOR. Staging Samet'in elle kullandığı
46+
# kapı; gece yarısı üzerine yazmak, sabah bakmak için bıraktığı şeyi
47+
# silmek olurdu. Burada aranan şey provaların ürüne ayak uydurup
48+
# uydurmadığı — bunun için o an staging'de ne varsa yeter.
49+
staging-rehearsal:
50+
needs: full-regression
51+
runs-on: ubuntu-latest
52+
timeout-minutes: 30
53+
steps:
54+
- name: Checkout
55+
uses: actions/checkout@v7
56+
57+
- name: Set up Node
58+
uses: actions/setup-node@v7
59+
with:
60+
node-version: 24
61+
cache: npm
62+
63+
- name: Install dependencies
64+
run: npm ci
65+
66+
# Eksik bir secret'ın hatası, provanın kendi hatasına benzemesin.
67+
# Kimlik bilgisi olmadan düşen bir iş, gerçekten kırılmış bir
68+
# sözleşmeyle karıştırılırsa görmezden gelinmeye başlar.
69+
- name: Verify the staging credentials are present
70+
env:
71+
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
72+
ORBIT_SESSION_PEPPER_V1: ${{ secrets.ORBIT_STAGING_SESSION_PEPPER_V1 }}
73+
ORBIT_CSRF_PEPPER_V1: ${{ secrets.ORBIT_STAGING_CSRF_PEPPER_V1 }}
74+
ORBIT_AGENT_CREDENTIAL_PEPPER_V1: ${{ secrets.ORBIT_STAGING_AGENT_CREDENTIAL_PEPPER_V1 }}
75+
run: |
76+
missing=""
77+
[ -n "$CLOUDFLARE_API_TOKEN" ] || missing="$missing CLOUDFLARE_API_TOKEN"
78+
[ -n "$ORBIT_SESSION_PEPPER_V1" ] || missing="$missing ORBIT_STAGING_SESSION_PEPPER_V1"
79+
[ -n "$ORBIT_CSRF_PEPPER_V1" ] || missing="$missing ORBIT_STAGING_CSRF_PEPPER_V1"
80+
[ -n "$ORBIT_AGENT_CREDENTIAL_PEPPER_V1" ] \
81+
|| missing="$missing ORBIT_STAGING_AGENT_CREDENTIAL_PEPPER_V1"
82+
if [ -n "$missing" ]; then
83+
echo "::error::Staging rehearsal cannot run. Missing repository secrets:$missing"
84+
exit 1
85+
fi
86+
87+
- name: Rehearse the staging contract
88+
env:
89+
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
90+
ORBIT_SESSION_PEPPER_V1: ${{ secrets.ORBIT_STAGING_SESSION_PEPPER_V1 }}
91+
ORBIT_CSRF_PEPPER_V1: ${{ secrets.ORBIT_STAGING_CSRF_PEPPER_V1 }}
92+
ORBIT_AGENT_CREDENTIAL_PEPPER_V1: ${{ secrets.ORBIT_STAGING_AGENT_CREDENTIAL_PEPPER_V1 }}
93+
# Buraya yalnız yeşil olduğu ölçülmüş provalar giriyor. slice2 ve
94+
# slice3 kasten dışarıda:
95+
#
96+
# slice2 — sponsorun ajan yaratmasını sınıyor; o uç 22 Temmuz'da
97+
# "Open Orbit to guest agents" ile kaldırıldı, ajanlar artık
98+
# kendileri kaydoluyor. Betik silinmiş bir API'ye bakıyor.
99+
#
100+
# slice3 — akışın ilk iki kaydının tam olarak ne olduğunu iddia
101+
# ediyor. Başka bir provanın bıraktığı tek bir yayımlanmış kayıt
102+
# bunu bozuyor, yani her gece kendi kendine kırmızıya döner.
103+
#
104+
# İkisini de kırmızı hâlde bağlamak, gecelik işi görmezden
105+
# gelinen bir işe çevirirdi — düzeltmeye çalıştığımız sorunun ta
106+
# kendisi. Karar verilene kadar elle çalıştırılabilir durumdalar.
107+
run: |
108+
npm run staging:verify
109+
npm run staging:slice4:e2e

‎scripts/orbit-production-config-tests.mjs‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -325,6 +325,27 @@ assert(
325325
deployWorkflow.includes("test \"$(git rev-parse HEAD)\" = \"$GITHUB_SHA\""),
326326
'deploy job does not verify the exact candidate commit',
327327
);
328+
/* Staging provaları gecelik işte kalmalı. Buraya gelmelerinin sebebi
329+
* tam olarak kimsenin onları çalıştırmıyor oluşuydu; işten sessizce
330+
* çıkarılırlarsa aynı üç haftalık körlük geri gelir. */
331+
assert(
332+
fullRegressionWorkflow.includes('staging-rehearsal:')
333+
&& fullRegressionWorkflow.includes('needs: full-regression'),
334+
'nightly regression no longer rehearses staging after the application checks',
335+
);
336+
for (const script of ['staging:verify', 'staging:slice4:e2e']) {
337+
assert(
338+
fullRegressionWorkflow.includes(`npm run ${script}`),
339+
`nightly staging rehearsal dropped ${script}`,
340+
);
341+
}
342+
/* Eksik secret'ın hatası provanın hatasına benzemesin: ayırt edilemeyen
343+
* bir kırmızı, görmezden gelinen bir kırmızıya dönüşür. */
344+
assert(
345+
fullRegressionWorkflow.includes('Missing repository secrets'),
346+
'nightly staging rehearsal no longer names the credential it is missing',
347+
);
348+
328349
assert(
329350
fullRegressionWorkflow.includes("cron: '30 1 * * *'")
330351
&& fullRegressionWorkflow.includes('workflow_dispatch:')

‎scripts/orbit-slice2-staging-e2e.ts‎

Lines changed: 3 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -11,9 +11,9 @@ import {
1111
randomBase64Url,
1212
} from '../src/server/identity/tokens';
1313
import { createEntityId } from '../src/server/foundation/ids';
14+
import { readStagingSecret } from './orbit-staging-secrets';
1415

1516
const ORIGIN = 'https://orbit-v6-staging.samett33710.workers.dev';
16-
const KEYCHAIN_SERVICE = 'staging.orbit.sametbasbug';
1717
const WRANGLER = 'node_modules/wrangler/bin/wrangler.js';
1818
const CONFIG = 'wrangler.staging.jsonc';
1919
const DATABASE = 'DB';
@@ -23,22 +23,6 @@ interface BrowserSession {
2323
csrf: string;
2424
}
2525

26-
function readKeychain(binding: string): string {
27-
const result = spawnSync('security', [
28-
'find-generic-password',
29-
'-s', KEYCHAIN_SERVICE,
30-
'-a', binding,
31-
'-w',
32-
], {
33-
encoding: 'utf8',
34-
stdio: ['ignore', 'pipe', 'ignore'],
35-
});
36-
assert.equal(result.status, 0, `Missing staging Keychain binding: ${binding}`);
37-
const value = result.stdout.trim();
38-
assert.ok(value, `Empty staging Keychain binding: ${binding}`);
39-
return value;
40-
}
41-
4226
function quote(value: string): string {
4327
return `'${value.replaceAll("'", "''")}'`;
4428
}
@@ -175,8 +159,8 @@ async function waitForSlice2Deployment(): Promise<void> {
175159

176160
await waitForSlice2Deployment();
177161

178-
const sessionPepper = readKeychain('ORBIT_SESSION_PEPPER_V1');
179-
const csrfPepper = readKeychain('ORBIT_CSRF_PEPPER_V1');
162+
const sessionPepper = readStagingSecret('ORBIT_SESSION_PEPPER_V1');
163+
const csrfPepper = readStagingSecret('ORBIT_CSRF_PEPPER_V1');
180164
const now = Date.now();
181165
const runSuffix = now.toString(36);
182166

‎scripts/orbit-slice3-staging-e2e.ts‎

Lines changed: 3 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -4,21 +4,12 @@ import { createEntityId } from '../src/server/foundation/ids';
44
import { SESSION_ABSOLUTE_TTL_MS, SESSION_IDLE_TTL_MS } from '../src/server/identity/constants';
55
import { createOpaqueToken, hmacDigest, randomBase64Url } from '../src/server/identity/tokens';
66
import { loadManifest } from './orbit-slice3-manifest';
7+
import { readStagingSecret } from './orbit-staging-secrets';
78

89
const ORIGIN = 'https://orbit-v6-staging.samett33710.workers.dev';
9-
const KEYCHAIN_SERVICE = 'staging.orbit.sametbasbug';
1010
const WRANGLER = 'node_modules/wrangler/bin/wrangler.js';
1111
const CONFIG = 'wrangler.staging.jsonc';
1212

13-
function keychain(binding: string): string {
14-
const result = spawnSync('security', ['find-generic-password', '-s', KEYCHAIN_SERVICE, '-a', binding, '-w'], {
15-
encoding: 'utf8', stdio: ['ignore', 'pipe', 'ignore'],
16-
});
17-
assert.equal(result.status, 0, `Missing staging Keychain binding: ${binding}`);
18-
assert.ok(result.stdout.trim());
19-
return result.stdout.trim();
20-
}
21-
2213
function quote(value: string): string {
2314
return `'${value.replaceAll("'", "''")}'`;
2415
}
@@ -42,8 +33,8 @@ async function ownerSession(): Promise<{ token: string; csrf: string; selector:
4233
`) as Array<{ id?: string }>;
4334
const accountId = rows[0]?.id;
4435
assert.ok(accountId);
45-
const sessionPepper = keychain('ORBIT_SESSION_PEPPER_V1');
46-
const csrfPepper = keychain('ORBIT_CSRF_PEPPER_V1');
36+
const sessionPepper = readStagingSecret('ORBIT_SESSION_PEPPER_V1');
37+
const csrfPepper = readStagingSecret('ORBIT_CSRF_PEPPER_V1');
4738
const session = await createOpaqueToken('session', sessionPepper);
4839
const csrf = randomBase64Url(32);
4940
const csrfDigest = await hmacDigest(`orbit:csrf:v1:${session.selector}:${csrf}`, csrfPepper);

‎scripts/orbit-slice4-staging-e2e.ts‎

Lines changed: 4 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -3,21 +3,13 @@ import { spawnSync } from 'node:child_process';
33
import { createEntityId } from '../src/server/foundation/ids';
44
import { createOpaqueToken, hmacDigest, randomBase64Url } from '../src/server/identity/tokens';
55
import { SESSION_ABSOLUTE_TTL_MS, SESSION_IDLE_TTL_MS } from '../src/server/identity/constants';
6+
import { readStagingSecret } from './orbit-staging-secrets';
67

78
const ORIGIN = 'https://orbit-v6-staging.samett33710.workers.dev';
8-
const SERVICE = 'staging.orbit.sametbasbug';
99
const WRANGLER = 'node_modules/wrangler/bin/wrangler.js';
1010
const CONFIG = 'wrangler.staging.jsonc';
1111
const DATABASE = 'DB';
1212

13-
function keychain(name: string): string {
14-
const result = spawnSync('security', ['find-generic-password','-s',SERVICE,'-a',name,'-w'], {
15-
encoding: 'utf8', stdio: ['ignore','pipe','ignore'],
16-
});
17-
assert.equal(result.status, 0, `Missing staging binding: ${name}`);
18-
return result.stdout.trim();
19-
}
20-
2113
function quote(value: string): string { return `'${value.replaceAll("'", "''")}'`; }
2214

2315
function execute(sql: string): Array<Record<string, unknown>> {
@@ -150,9 +142,9 @@ async function ownerRequest(
150142
await waitReady();
151143
const now = Date.now();
152144
const suffix = now.toString(36);
153-
const agentPepper = keychain('ORBIT_AGENT_CREDENTIAL_PEPPER_V1');
154-
const sessionPepper = keychain('ORBIT_SESSION_PEPPER_V1');
155-
const csrfPepper = keychain('ORBIT_CSRF_PEPPER_V1');
145+
const agentPepper = readStagingSecret('ORBIT_AGENT_CREDENTIAL_PEPPER_V1');
146+
const sessionPepper = readStagingSecret('ORBIT_SESSION_PEPPER_V1');
147+
const csrfPepper = readStagingSecret('ORBIT_CSRF_PEPPER_V1');
156148
const owner = execute(`
157149
SELECT a.id FROM accounts a JOIN auth_identities ai ON ai.account_id = a.id
158150
WHERE ai.provider = 'github' AND ai.provider_user_id = '126420524' LIMIT 1

‎scripts/orbit-staging-secrets.ts‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
import assert from 'node:assert/strict';
2+
import { spawnSync } from 'node:child_process';
3+
4+
/* Staging provalarının ihtiyaç duyduğu üç pepper'ın tek okuma yolu.
5+
*
6+
* Yerelde Samet'in macOS Keychain'inde duruyorlar ve orada kalmalı:
7+
* dosyaya yazılan bir sır er ya da geç commit'lenir. Ama gecelik
8+
* regresyon Linux runner'da çalışıyor ve orada Keychain yok — bu yüzden
9+
* önce ortam değişkenine bakıyoruz.
10+
*
11+
* Sıra bilerek bu yönde: CI ortamı değişkeni verir, yerel makine
12+
* vermez ve Keychain'e düşer. Tersi olsaydı, yerelde yanlışlıkla tanımlı
13+
* kalmış bir değişken sessizce Keychain'in önüne geçerdi. */
14+
export const STAGING_KEYCHAIN_SERVICE = 'staging.orbit.sametbasbug';
15+
16+
export function readStagingSecret(binding: string): string {
17+
const fromEnvironment = process.env[binding];
18+
if (fromEnvironment !== undefined && fromEnvironment !== '') return fromEnvironment;
19+
20+
const result = spawnSync('security', [
21+
'find-generic-password',
22+
'-s', STAGING_KEYCHAIN_SERVICE,
23+
'-a', binding,
24+
'-w',
25+
], {
26+
encoding: 'utf8',
27+
stdio: ['ignore', 'pipe', 'ignore'],
28+
});
29+
/* Hata mesajı iki yolu da söylemeli: CI'da düşen biri Keychain'i
30+
* aramaya kalkmasın, yerelde düşen biri secret arayıp durmasın. */
31+
assert.equal(
32+
result.status,
33+
0,
34+
`Missing staging secret ${binding}: set it as an environment variable, `
35+
+ `or store it in the "${STAGING_KEYCHAIN_SERVICE}" Keychain service.`,
36+
);
37+
return result.stdout.trim();
38+
}

0 commit comments

Comments
 (0)