Skip to content

Commit ef2acb6

Browse files
authored
Merge pull request #1575 from cipherstash/docs/migration-harness-design
docs(e2e): specify the two-store v2-to-v3 guide and migration harness
2 parents 3bd2e76 + 9346c87 commit ef2acb6

8 files changed

Lines changed: 954 additions & 0 deletions

File tree

‎CONTEXT-MAP.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,8 @@
1010
- [Migrate](./languages/typescript/packages/migrate/CONTEXT.md) — moves an
1111
existing column between plaintext and encrypted form, resumably, for the
1212
Stack CLI and customer workers.
13+
- [Migration Harness](./e2e/migration-harness/CONTEXT.md) — tests the
14+
Protect.js upgrade guide by having agents follow it and judging the result.
1315

1416
## Relationships
1517

‎e2e/migration-harness/CONTEXT.md‎

Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
# Migration Harness
2+
3+
A harness that tests an upgrade guide by having agents follow it. A Migrator upgrades a Protect.js application using only the guide, an Oracle judges the result, and an Editor revises the guide from what went wrong.
4+
5+
## Language
6+
7+
**Fixture App**:
8+
The small Protect.js application with encrypted data in DynamoDB and Postgres that is the starting point for a Run.
9+
_Avoid_: sample app, demo app, test app
10+
11+
**Run**:
12+
One execution of the harness pipeline for a Fixture App, described by a Run Report even when execution ends before migration or verification.
13+
_Avoid_: trial, attempt, episode
14+
15+
**Legacy Data**:
16+
Records from the unmigrated Fixture App used as the baseline for compatibility and no-rewrite checks, distinct from Mutation Data.
17+
_Avoid_: seed data, fixtures, old data
18+
19+
**Iteration**:
20+
One or more Runs against a single revision of the guide, followed by at most one Editor revision.
21+
_Avoid_: round, cycle
22+
23+
**Migrator**:
24+
The agent responsible for upgrading the Fixture App using the guide, without access to the Oracle or Reference Migration.
25+
_Avoid_: subject, worker, executor
26+
27+
**Oracle**:
28+
The deterministic, agent-free checks of the migrated Fixture App's behaviour, stored data and migration completeness, hidden from the Migrator.
29+
_Avoid_: grader, judge, evaluator
30+
31+
**App Host**:
32+
The execution environment for the migrated Fixture App during Oracle verification.
33+
_Avoid_: runner, worker, sandbox
34+
35+
**Editor**:
36+
The agent that revises the guide from eligible Run Reports in Iteration Results.
37+
_Avoid_: improver, reviewer, author
38+
39+
**Reference Migration**:
40+
A known-correct upgrade of the Fixture App used to establish that the Oracle can be passed, hidden from both Migrator and Editor.
41+
_Avoid_: golden app, solution, answer key
42+
43+
**Friction Log**:
44+
The Migrator's own record of where the guide was unclear, wrong or silent.
45+
_Avoid_: notes, feedback, report
46+
47+
**Mutation Data**:
48+
Records reserved for mutating Oracle checks, separate from Legacy Data.
49+
_Avoid_: reserved records
50+
51+
**Run Slot**:
52+
One configured position in an Iteration, associated with a Run and its infrastructure retries.
53+
_Avoid_: run position
54+
55+
**Loop**:
56+
A bounded sequence of Iterations that evaluates and revises one guide toward consistent passing Runs without actionable friction.
57+
_Avoid_: iteration loop
58+
59+
**Fake Migrator**:
60+
A deterministic substitute for a Migrator in harness self-tests.
61+
_Avoid_: mock migrator
62+
63+
**Review List**:
64+
The review artifact containing claims that need tests, skill drift, and evidence-linked diagnoses supporting or rejecting guide edits.
65+
_Avoid_: Needs-test List
66+
67+
**Run Report**:
68+
The harness-produced record of a Run's outcomes, Oracle results, Run Manifest, diff, Friction Log and transcript.
69+
_Avoid_: run output
70+
71+
**Fake Editor**:
72+
A deterministic substitute for an Editor in harness self-tests.
73+
_Avoid_: mock editor
74+
75+
**Broken Migration**:
76+
A deliberately incorrect migrated Fixture App used to establish a specific Oracle failure or isolation result.
77+
_Avoid_: fake migration
78+
79+
**Protected Resources**:
80+
The material reserved for the harness to preserve independent verification and isolation between Runs.
81+
_Avoid_: hidden Oracle evidence, hidden set
82+
83+
**Iteration Results**:
84+
The collected Run Reports, evidence and available edit disposition for one Iteration.
85+
_Avoid_: iteration output
86+
87+
**Loop Summary**:
88+
The human-review artifact aggregating Iteration Results, pass rates, friction, kept/reverted edits and the Review List across a Loop.
89+
_Avoid_: loop report
90+
91+
**Clean Iteration**:
92+
A complete Iteration with passing Runs and no actionable friction.
93+
_Avoid_: clean round
94+
95+
**Store Set**:
96+
The paired DynamoDB and Postgres stores assigned to one role in a Run.
97+
_Avoid_: database pair
98+
99+
**Migrator Stores**:
100+
The Store Set assigned to a Migrator for exploration and application verification.
101+
_Avoid_: agent databases
102+
103+
**Oracle Stores**:
104+
The separate Store Set accessed by the harness and App Host for deterministic Oracle checks.
105+
_Avoid_: acceptance databases
106+
107+
**Run Manifest**:
108+
The reproducibility metadata attached to a Run Report, identifying Stack, guide, Fixture App and agent configuration.
109+
_Avoid_: run metadata
110+
111+
**Migrator Environment**:
112+
The execution environment for a Migrator's commands.
113+
_Avoid_: Migrator sandbox

0 commit comments

Comments
 (0)