Skip to content

Commit 2e20fe8

Browse files
authored
Merge pull request #381 from bugdrophq/codex/protocol-v1-compatibility-harness
test: add isolated protocol v1 compatibility harness
2 parents 63827da + 775ee86 commit 2e20fe8

11 files changed

Lines changed: 580 additions & 2 deletions

docs/protocol/v1-compatibility.md

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
# Protocol v1 compatibility harness
2+
3+
This first tranche is an executable, **test-only** service model under
4+
`test/protocol/v1`. It adds no route, runtime import, Worker, binding, signing key,
5+
or deployment configuration. Public widget and Worker behavior is unchanged.
6+
7+
## Source and drift
8+
9+
The approved account-control-plane proposal in `bugdrop-web`,
10+
`docs/account-control-plane-data-layer-proposal.md` (2026-09-16), governs the
11+
boundary. Wire details come from the SDK's `docs/protocol.md` and fixtures at
12+
[SDK commit 08e074d](https://github.com/bugdrophq/bugdrop-sdk-typescript/tree/08e074d2754a9b3f5034de9fde8892b9918c9efa).
13+
`test/protocol/v1/fixtures/upstream.json` records the source path, commit, and
14+
SHA-256 of each byte-for-byte copy. Do not format or hand-edit copied fixtures.
15+
16+
```sh
17+
npx vitest run test/protocol/v1
18+
node scripts/protocol/check-fixture-drift.mjs /absolute/path/to/sdk-checkout
19+
```
20+
21+
Normal unit CI checks the pinned hashes without network access. The second command
22+
also compares every pinned fixture against a supplied SDK checkout; missing files,
23+
changed bytes, or changed hashes fail closed. It does not fetch upstream or assert
24+
that the pin is the latest SDK release. To update, review the SDK protocol changes,
25+
copy the fixtures unchanged, update the commit and hashes, run the explicit drift
26+
check, and review the changed vectors alongside the model. Coordinated SDK/managed
27+
plane changes must run that check before claiming compatibility.
28+
29+
## Executable boundary
30+
31+
The suite consumes the SDK origin, credential, and submission-binding vectors.
32+
It checks exact canonical origins against configured origins (including rejection
33+
of trailing dots, case/default-port aliases, and different hosts/ports), required
34+
stable submission IDs, canonical SHA-256 digest encoding, and hashing of exact
35+
body bytes before delivery. It checks the credential derivation and strict bearer
36+
parsing, captures the explicit SDK package version separately from contract
37+
version, and rejects unknown exchange fields, including reporter identities.
38+
Errors do not reflect rejected canaries.
39+
40+
The signed-only test double rejects authentication, binding, and delivery failures
41+
without issuing a network request or retrying delivery. Authentication is an
42+
injected verifier: the fixture bearer is not a real credential and its syntax is
43+
not proof of authorization. The double is deliberately not connected to the
44+
public widget or either Worker plane. Its no-fallback assertion applies only to
45+
this model; actual SDK and managed-plane implementations require their own tests.
46+
47+
## Integration dependencies and limits
48+
49+
- [SDK reconciliation PR #12](https://github.com/bugdrophq/bugdrop-sdk-typescript/pull/12)
50+
owns the normative fixtures and SDK transport.
51+
Keep the pin synchronized with its reconciliation PR before integrated V1 claims.
52+
- The separate managed-plane PR in `bugdrophq/bugdrop` owns isolated ingress,
53+
credential verification, signed-token verification, private delivery, and
54+
manifests. It must consume these same vectors against its actual implementation.
55+
This PR can land independently because it has no runtime integration.
56+
- End-to-end publication remains blocked on the production-like SDK capability
57+
exchange and managed-plane checks: fixed algorithm/key/issuer/audience,
58+
tenant/application/destination/configuration binding, expiry, revocation,
59+
durable 30-day receipts, and at-most-once GitHub delivery. This harness proves
60+
none of those operational guarantees.
61+
- The identity checks cover exchange fields and model errors. Opaque submission
62+
IDs cannot be proven unrelated to identity by syntax; customers must generate
63+
random logical-submission IDs. Real queues, storage, logs, analytics, GitHub
64+
payloads, and third-party services still need end-to-end privacy canaries.
65+
66+
Public regressions use existing `test/api.test.ts`, widget/flow unit suites, and
67+
`e2e/public-flow.spec.ts`, `e2e/default-flow-compatibility.spec.ts`,
68+
`e2e/legacy-compat.spec.ts`, and `e2e/api.spec.ts`. Run the local venue using
69+
`bugdrop.localhost` and build the widget with `BUGDROP_TEST_HOOKS=1` first.
70+
No production configuration, provisioning, deployment, or signed traffic is
71+
required or authorized by this harness.

e2e/default-flow-compatibility.spec.ts

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -382,7 +382,10 @@ test.describe('paired default-flow screenshot compatibility oracle', () => {
382382
expect(result.requests[0]).toMatchObject({
383383
screenshot: expect.stringMatching(/^data:image\/png;base64,/),
384384
metadata: {
385-
url: 'http://localhost:8787/test/complex-dom',
385+
url: new URL(
386+
'/test/complex-dom',
387+
process.env.PLAYWRIGHT_BASE_URL || 'http://localhost:8787'
388+
).href,
386389
elementSelector: null,
387390
fullElementSelector: null,
388391
},
@@ -450,7 +453,8 @@ test.describe('paired default-flow screenshot compatibility oracle', () => {
450453
expect(result.requests[0]).toMatchObject({
451454
screenshot: expect.stringMatching(/^data:image\/png;base64,/),
452455
metadata: {
453-
url: 'http://localhost:8787/test/redaction',
456+
url: new URL('/test/redaction', process.env.PLAYWRIGHT_BASE_URL || 'http://localhost:8787')
457+
.href,
454458
elementSelector: null,
455459
fullElementSelector: null,
456460
},

package.json

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,8 @@
3838
"release:plan": "node scripts/release/plan.mjs",
3939
"release:live": "node scripts/release/live-release.mjs",
4040
"test": "vitest run",
41+
"test:protocol": "vitest run test/protocol/v1",
42+
"protocol:check-fixtures": "node scripts/protocol/check-fixture-drift.mjs",
4143
"test:release": "vitest run test/release",
4244
"test:release-workflow": "vitest run test/release/git-observer.test.ts test/release/github-adapter.test.ts test/release/github-publication-adapter.test.ts test/release/cloudflare-adapter.test.ts test/release/cloudflare-client.test.ts test/release/cloudflare-capability-drill.test.ts test/release/live-release.test.ts test/release/workflow.test.ts && bash test/release-workflow-contract.test.sh",
4345
"test:static-assets": "vitest run test/release/static-assets.test.ts",
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
import { createHash } from 'node:crypto';
2+
import { readFileSync } from 'node:fs';
3+
import { resolve } from 'node:path';
4+
import { fileURLToPath } from 'node:url';
5+
6+
const fixtures = fileURLToPath(new URL('../../test/protocol/v1/fixtures/', import.meta.url));
7+
const manifest = JSON.parse(readFileSync(resolve(fixtures, 'upstream.json'), 'utf8'));
8+
const sdkRoot = process.argv[2];
9+
for (const [name, expected] of Object.entries(manifest.files)) {
10+
const local = readFileSync(resolve(fixtures, name));
11+
if (createHash('sha256').update(local).digest('hex') !== expected) {
12+
throw new Error(`Pinned fixture changed: ${name}; review and update provenance explicitly`);
13+
}
14+
if (sdkRoot && !local.equals(readFileSync(resolve(sdkRoot, manifest.directory, name)))) {
15+
throw new Error(`SDK fixture drift: ${name}`);
16+
}
17+
}
18+
console.log(
19+
`Protocol fixtures match ${manifest.commit}${sdkRoot ? ' and supplied SDK checkout' : ''}`
20+
);
Lines changed: 215 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,215 @@
1+
import { createHmac } from 'node:crypto';
2+
import { execFileSync } from 'node:child_process';
3+
import { describe, expect, it, vi } from 'vitest';
4+
import origins from './fixtures/origin.v1.json';
5+
import submission from './fixtures/submission-binding.v1.json';
6+
import credential from './fixtures/api-key-credential.v1.json';
7+
import {
8+
canonicalBytes,
9+
exactOrigin,
10+
parseBearer,
11+
parseBinding,
12+
parseExchange,
13+
signedOnlyDouble,
14+
verifyBinding,
15+
} from './service-double';
16+
17+
const origin = origins.valid[0];
18+
const body = () => ({ schemaVersion: 1, ...submission.bound, origin });
19+
function headers() {
20+
return new Headers({
21+
Authorization: credential.authorization,
22+
'Content-Type': 'application/json',
23+
Accept: 'application/vnd.bugdrop.submission-capability.v1+json',
24+
'X-BugDrop-Contract-Version': '1',
25+
'X-BugDrop-SDK-Version': '0.1.0-preview.7',
26+
});
27+
}
28+
29+
describe('SDK protocol v1 compatibility (test-only service boundary)', () => {
30+
it('checks the pinned fixture hashes in ordinary CI', () => {
31+
expect(
32+
execFileSync(process.execPath, ['scripts/protocol/check-fixture-drift.mjs'], {
33+
encoding: 'utf8',
34+
})
35+
).toContain('Protocol fixtures match');
36+
});
37+
38+
it('derives the exact SDK bearer bytes without sending the root credential', () => {
39+
const auth = createHmac('sha256', canonicalBytes(credential.rootSecret, 32))
40+
.update(`bugdrop:auth:v1\0${credential.keyId}`)
41+
.digest('base64url');
42+
expect(auth).toBe(credential.authSecret);
43+
expect(credential.authorization).toBe(`Bearer bd_auth_v1.${credential.keyId}.${auth}`);
44+
expect(parseBearer(credential.authorization)).toEqual({
45+
keyId: credential.keyId,
46+
authSecret: Buffer.from(auth, 'base64url'),
47+
});
48+
});
49+
50+
it.each([
51+
...credential.invalidAuthorizations,
52+
...credential.invalidApiKeys.map(value => `Bearer ${value}`),
53+
null,
54+
'',
55+
`Bearer ${credential.apiKey}`,
56+
credential.authorization.toLowerCase(),
57+
` ${credential.authorization}`,
58+
`${credential.authorization} `,
59+
`${credential.authorization}\n`,
60+
`${credential.authorization}\r\n`,
61+
credential.authorization.replace('Bearer ', 'Bearer '),
62+
`${credential.authorization}=`,
63+
`${credential.authorization.slice(0, -1)}p`,
64+
credential.authorization.replace(credential.keyId, `${credential.keyId}=`),
65+
credential.authorization.replace(credential.keyId, `${credential.keyId.slice(0, -1)}x`),
66+
])('rejects noncanonical or wrong credentials: %s', value => {
67+
expect(() => parseBearer(value)).toThrow();
68+
});
69+
70+
it.each(origins.valid)('accepts canonical origin %s', value => {
71+
expect(exactOrigin(value)).toBe(value);
72+
expect(parseExchange(headers(), { ...body(), origin: value }, value).origin).toBe(value);
73+
});
74+
it.each([
75+
...origins.invalid,
76+
'https://app.example.com/',
77+
'https://app.example.com?',
78+
'https://app.example.com#',
79+
'null',
80+
'https://app.example.com.:8443',
81+
'http://localhost.',
82+
])('rejects origin alias %s', value => {
83+
expect(() => exactOrigin(value)).toThrow();
84+
expect(() => parseExchange(headers(), { ...body(), origin: value }, origin)).toThrow();
85+
expect(() => parseExchange(headers(), body(), value)).toThrow();
86+
});
87+
it.each([
88+
'https://other.example.com',
89+
'https://app.example.com:8443',
90+
'https://sub.app.example.com',
91+
])('rejects canonical but unconfigured origin %s', value => {
92+
expect(() => parseExchange(headers(), { ...body(), origin: value }, origin)).toThrow();
93+
});
94+
95+
it.each(submission.verificationCases)('$name matches the SDK binding result', vector => {
96+
const verify = () =>
97+
verifyBinding(submission.bound, vector.submissionId, Buffer.from(vector.requestBody));
98+
if (vector.accepted) expect(verify).not.toThrow();
99+
else expect(verify).toThrow();
100+
});
101+
it.each([
102+
...submission.invalidPayloadDigests,
103+
`${submission.bound.payloadDigest.slice(0, -1)}R`,
104+
'',
105+
null,
106+
42,
107+
])('rejects invalid digest %s', payloadDigest => {
108+
expect(() => parseExchange(headers(), { ...body(), payloadDigest }, origin)).toThrow();
109+
});
110+
it.each(['', 'a'.repeat(201), 'é'.repeat(101), '\ud800', '\udc00', null, 42])(
111+
'rejects invalid submission ID',
112+
submissionId => {
113+
expect(() => parseExchange(headers(), { ...body(), submissionId }, origin)).toThrow();
114+
}
115+
);
116+
it.each(['a', 'é'.repeat(100), '🪲'.repeat(50), ' opaque id '])(
117+
'preserves valid opaque IDs exactly',
118+
submissionId => {
119+
expect(parseBinding({ ...submission.bound, submissionId }).submissionId).toBe(submissionId);
120+
}
121+
);
122+
it.each(['submissionId', 'payloadDigest', 'schemaVersion'])('requires %s', field => {
123+
const request: Record<string, unknown> = body();
124+
delete request[field];
125+
expect(() => parseExchange(headers(), request, origin)).toThrow();
126+
});
127+
it('captures the explicit SDK version separately from schema version', () => {
128+
expect(parseExchange(headers(), body(), origin)).toEqual({
129+
...submission.bound,
130+
origin,
131+
sdkVersion: '0.1.0-preview.7',
132+
});
133+
const requestHeaders = headers();
134+
requestHeaders.set('X-BugDrop-SDK-Version', '2.3.4');
135+
expect(parseExchange(requestHeaders, body(), origin).sdkVersion).toBe('2.3.4');
136+
});
137+
it.each([
138+
'Authorization',
139+
'Content-Type',
140+
'Accept',
141+
'X-BugDrop-Contract-Version',
142+
'X-BugDrop-SDK-Version',
143+
])('fails closed without %s', field => {
144+
const requestHeaders = headers();
145+
requestHeaders.delete(field);
146+
expect(() => parseExchange(requestHeaders, body(), origin)).toThrow();
147+
});
148+
it.each([
149+
'subject',
150+
'sub',
151+
'reporterId',
152+
'userId',
153+
'pseudonym',
154+
'email',
155+
'reporter',
156+
'metadata',
157+
'applicationId',
158+
'repository',
159+
'installation',
160+
'labels',
161+
])('rejects unexpected %s without reflecting canary data', field => {
162+
const canary = 'private-end-user-canary';
163+
try {
164+
parseExchange(headers(), { ...body(), [field]: { identity: canary } }, origin);
165+
throw new Error('Unexpected acceptance');
166+
} catch (error) {
167+
expect(error).toBeInstanceOf(TypeError);
168+
expect(String(error)).not.toContain(canary);
169+
}
170+
});
171+
172+
it.each(['auth', 'binding', 'delivery'])(
173+
'never invokes anonymous transport after %s failure',
174+
async failure => {
175+
const network = vi
176+
.spyOn(globalThis, 'fetch')
177+
.mockRejectedValue(new Error('No network allowed'));
178+
const managed = vi.fn(async () => {
179+
if (failure === 'delivery') throw new Error('unavailable');
180+
});
181+
const verify = vi.fn(async () => {
182+
if (failure === 'auth') throw new Error('unauthorized');
183+
return submission.bound;
184+
});
185+
try {
186+
expect(
187+
await signedOnlyDouble(
188+
verify,
189+
submission.bound.submissionId,
190+
Buffer.from(
191+
failure === 'binding' ? `${submission.requestBody} ` : submission.requestBody
192+
),
193+
managed
194+
)
195+
).toBe('rejected');
196+
expect(managed).toHaveBeenCalledTimes(failure === 'delivery' ? 1 : 0);
197+
expect(network).not.toHaveBeenCalled();
198+
} finally {
199+
network.mockRestore();
200+
}
201+
}
202+
);
203+
it('only hands a verified exact body to managed delivery once', async () => {
204+
const managed = vi.fn(async () => {});
205+
expect(
206+
await signedOnlyDouble(
207+
async () => submission.bound,
208+
submission.bound.submissionId,
209+
Buffer.from(submission.requestBody),
210+
managed
211+
)
212+
).toBe('accepted');
213+
expect(managed).toHaveBeenCalledExactlyOnceWith(Buffer.from(submission.requestBody));
214+
});
215+
});

test/protocol/v1/drift.test.ts

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
import { execFileSync } from 'node:child_process';
2+
import { cpSync, mkdtempSync, mkdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
3+
import { tmpdir } from 'node:os';
4+
import { join } from 'node:path';
5+
import { expect, it } from 'vitest';
6+
7+
it('detects changed upstream bytes rather than silently accepting fixture drift', () => {
8+
const sdk = mkdtempSync(join(tmpdir(), 'bugdrop-protocol-sdk-'));
9+
const destination = join(sdk, 'packages/contracts/fixtures');
10+
mkdirSync(destination, { recursive: true });
11+
cpSync('test/protocol/v1/fixtures', destination, { recursive: true });
12+
const check = () =>
13+
execFileSync(process.execPath, ['scripts/protocol/check-fixture-drift.mjs', sdk], {
14+
stdio: 'pipe',
15+
});
16+
try {
17+
expect(check).not.toThrow();
18+
const fixture = join(destination, 'origin.v1.json');
19+
writeFileSync(fixture, `${readFileSync(fixture, 'utf8')}\n`);
20+
expect(check).toThrow();
21+
rmSync(fixture);
22+
expect(check).toThrow();
23+
} finally {
24+
rmSync(sdk, { recursive: true, force: true });
25+
}
26+
});
Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
{
2+
"schemaVersion": 1,
3+
"apiKey": "bd_api_v1.AAECAwQFBgcICQoLDA0ODw.EBESExQVFhcYGRobHB0eHyAhIiMkJSYnKCkqKywtLi8",
4+
"keyId": "AAECAwQFBgcICQoLDA0ODw",
5+
"rootSecret": "EBESExQVFhcYGRobHB0eHyAhIiMkJSYnKCkqKywtLi8",
6+
"authSecret": "mrJiVKhkJz8d7tNgPkSTl2tRDQ840gImsfUvb1q0ZPo",
7+
"authorization": "Bearer bd_auth_v1.AAECAwQFBgcICQoLDA0ODw.mrJiVKhkJz8d7tNgPkSTl2tRDQ840gImsfUvb1q0ZPo",
8+
"invalidApiKeys": [
9+
"bd_api_v1.AAECAwQFBgcICQoLDA0ODw.EBESExQVFhcYGRobHB0eHyAhIiMkJSYnKCkqKywtLi8=",
10+
"bd_api_v1.AAECAwQFBgcICQoLDA0ODx.EBESExQVFhcYGRobHB0eHyAhIiMkJSYnKCkqKywtLi8",
11+
"bd_api_v1.AAECAwQFBgcICQoLDA0ODw.EBESExQVFhcYGRobHB0eHyAhIiMkJSYnKCkqKywtLi9",
12+
"bd_api_v2.AAECAwQFBgcICQoLDA0ODw.EBESExQVFhcYGRobHB0eHyAhIiMkJSYnKCkqKywtLi8",
13+
"bd_api_v1.AAECAwQFBgcICQoLDA0ODw.EBESExQVFhcYGRobHB0eHyAhIiMkJSYnKCkqKywtLi8.extra",
14+
"bd_api_v1.AAECAwQFBgcICQoLDA0ODw.EBESExQVFhcYGRobHB0eHyAhIiMkJSYnKCkqKywtLi8\n",
15+
"bd_api_v1.AAECAwQFBgcICQoLDA0ODw.EBESExQVFhcYGRobHB0eHyAhIiMkJSYnKCkqKywtLi"
16+
],
17+
"invalidAuthorizations": [
18+
"Bearer bd_auth_v1.AAECAwQFBgcICQoLDA0ODw.mrJiVKhkJz8d7tNgPkSTl2tRDQ840gImsfUvb1q0ZPo=",
19+
"Bearer bd_auth_v1.AAECAwQFBgcICQoLDA0ODw.mrJiVKhkJz8d7tNgPkSTl2tRDQ840gImsfUvb1q0ZPp",
20+
"Bearer bd_auth_v1.AAECAwQFBgcICQoLDA0ODx.mrJiVKhkJz8d7tNgPkSTl2tRDQ840gImsfUvb1q0ZPo",
21+
"Bearer bd_auth_v2.AAECAwQFBgcICQoLDA0ODw.mrJiVKhkJz8d7tNgPkSTl2tRDQ840gImsfUvb1q0ZPo",
22+
"Bearer bd_auth_v1.AAECAwQFBgcICQoLDA0ODw.mrJiVKhkJz8d7tNgPkSTl2tRDQ840gImsfUvb1q0ZPo.extra",
23+
"Bearer bd_api_v1.AAECAwQFBgcICQoLDA0ODw.EBESExQVFhcYGRobHB0eHyAhIiMkJSYnKCkqKywtLi8"
24+
]
25+
}

0 commit comments

Comments
 (0)