You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 8a226ea
Browse filesBrowse the repository at this point in the historyBrowse files
This prerequisite exercise uses real issue [#10](https://github.com/codeabovelab/codeboost/issues/10) and merged [PR #17](https://github.com/codeabovelab/codeboost/pull/17). It is an attribution exercise, not a timed trial and not evidence for the go/no-go thresholds.
| Plan item | Declared file | Assigned segments | Final attribution | Final scope |
11
+
|---|---|---:|---|---|
12
+
| P1 |`test/review.test.ts`| 4 text | Attributed | In scope |
13
+
| P2 |`test/plant.test.ts`| 4 text | Attributed | In scope |
14
+
| P3 |`docs/experiments/review-summary-edge-cases.md`| 1 file + 1 text | Attributed | In scope |
15
+
| P4 |`AGENTS.md`| 2 text | Attributed | In scope |
16
+
17
+
Codeboost exposed 12 segments before assignment, all in the Unplanned row. Each segment mapped to exactly one item through that item's declared file contract. After the assignments, the per-item segment counts were 4, 4, 2, and 2; all four rows reported `✓ Attributed` and `✓ In scope`. No segment was unclear or multiply owned, so this exercise produced no ambiguity fixture. The earlier rename and literal-path concerns already have focused fixtures in `test/review.test.ts` and `test/plant.test.ts`.
18
+
19
+
The assignment was rerun from a detached worktree at the exact head using a fresh SQLite review store. The stored choices, rather than a hand-written diff classification alone, produced the final attribution and scope results above.
Copy file name to clipboardExpand all lines: docs/experiments/review-protocol.md
+12-8Lines changed: 12 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,19 +1,23 @@
1
1
# Plan-indexed review: go/no-go protocol
2
2
3
-
Status: **not started; issue pairs and human reviewer still required**. This document does not claim a passed gate. Complete the pair table and commit its exact revision before any timed review. No pair or threshold may change once the first review begins.
3
+
Status: **pair selection frozen; implementations and reviews not started**. Max Hwang (`mchwang`) is the blinded reviewer and Codex is the experiment operator. This document does not claim a passed gate. Complete each row's PR and SHA fields and commit that exact revision before the first timed review. No issue, assignment, order, or threshold may change once the first review begins.
4
4
5
5
## Pair selection and order
6
6
7
7
Choose four pairs of comparable, small real issues in a repository owned by the reviewer. Match each pair on scope, language, and estimated review complexity before implementing either issue. Record issue URLs, PR URLs, base/head SHAs, and which review method each gets. Do not reuse the demo fixture as an experimental issue.
8
8
9
-
| Pair | Real issue A / PR | Real issue B / PR | A method | B method |
9
+
All eight issues are in `codeabovelab/guardyx-gstack`. A/B placement within each pair was randomized once before implementation; the method pattern remains the alternating pattern approved in the protocol.
10
+
11
+
| Pair | Real issue A / PR / base..head | Real issue B / PR / base..head | A method | B method |
The pairs are matched before implementation on repository, primary implementation language, subsystem, and expected review surface. Pair 1 changes MCP discovery behavior; pair 2 changes operational/compliance documentation without runtime behavior; pair 3 changes agent-key lifecycle across the API and UI; pair 4 adds per-invocation compliance evidence across export and PDF surfaces. If implementation discovery makes a pair materially unequal or blocked, stop before the first timed review and record a new frozen protocol revision rather than silently substituting an issue.
15
19
16
-
The reviewer must not implement the paired changes or inspect the plants before deciding on the PR. An operator prepares plans and commits by hand, records trusted ledger ownership, and supplies both the plan and code to each review method. The earlier one-issue manual assignment also remains required: record how every change maps to the attribution table and turn unclear cases into fixtures.
20
+
The reviewer must not implement the paired changes or inspect the plants before deciding on the PR. The operator must not expose plant inputs, `sealed.json`, mappings, planted locations, or unblinded implementation notes in the PR, review package, task messages, or protocol updates. An operator prepares plans and commits by hand, records trusted ledger ownership, and supplies both the plan and code to each review method. The earlier one-issue manual assignment is recorded in `manual-assignment.md`; all 12 segments were assigned and in scope, with no unclear case requiring another fixture.
17
21
18
22
## Planting
19
23
@@ -37,4 +41,4 @@ Report declared-file catch rate for both methods without a pass threshold. Stop
37
41
38
42
## Current evidence
39
43
40
-
Engineering tests prove browser interactions, persistence, attribution, and planting mechanics. They do **not** establish human review speed or catch rates. No real issue pairs, timed decisions, or catch-rate results exist yet.
44
+
Engineering tests prove browser interactions, persistence, attribution, and planting mechanics. The separate manual assignment proves that one real merged issue can be completely classified through stored review choices. Neither establishes human review speed or catch rates. Four Guardyx issue pairs are selected, but no paired implementation, timed decision, or catch-rate result exists yet.
0 commit comments