Skip to content

Commit 1750769

Browse files
authored
Merge pull request #423 from PhysShell/feat/p014-prod0-coverage
feat(p014): add opt-in project-aware binding coverage
2 parents 22b29d5 + 4b0321e commit 1750769

96 files changed

Lines changed: 45308 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/ci.yml‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2132,6 +2132,50 @@ jobs:
21322132
<(jq -S . "$RUNNER_TEMP/proj-facts-norefs.json") \
21332133
|| { echo "FAIL: --no-project-refs changed the facts for an unbuilt sample project"; exit 1; }
21342134
echo "OK: --help renders the option list; --no-project-refs is accepted"
2135+
# P-014 PROD0: the opt-in --binding-coverage sidecar is OBSERVATION ONLY. The facts are byte-identical with
2136+
# and without it; a directory or an unrestored project is recorded UNSUPPORTED (never measured by substitution);
2137+
# a restored copy of the sample is measured per project with no foreign binding, deterministically.
2138+
# Evidence: docs/evidence/p014-prod0-2026-10.
2139+
- name: P-014 PROD0 — --binding-coverage sidecar (observation only)
2140+
run: |
2141+
x="dotnet run --project frontend/roslyn/OwnSharp.Extractor --"
2142+
proj=frontend/roslyn/project-input-sample/ProjectInputSample.csproj
2143+
$x "$proj" -o "$RUNNER_TEMP/bc-off.json"
2144+
$x "$proj" -o "$RUNNER_TEMP/bc-on.json" --binding-coverage "$RUNNER_TEMP/bc-unrestored.json"
2145+
cmp "$RUNNER_TEMP/bc-off.json" "$RUNNER_TEMP/bc-on.json" \
2146+
|| { echo "FAIL: --binding-coverage changed the facts document"; exit 1; }
2147+
$x frontend/roslyn/samples -o "$RUNNER_TEMP/bc-dir-off.json"
2148+
$x frontend/roslyn/samples -o "$RUNNER_TEMP/bc-dir-on.json" --binding-coverage "$RUNNER_TEMP/bc-dir.json"
2149+
cmp "$RUNNER_TEMP/bc-dir-off.json" "$RUNNER_TEMP/bc-dir-on.json" \
2150+
|| { echo "FAIL: --binding-coverage changed the facts document (directory input)"; exit 1; }
2151+
# a restored COPY of the sample (the sample itself stays unbuilt for the parity checks above)
2152+
cp -r frontend/roslyn/project-input-sample "$RUNNER_TEMP/bc-sample"
2153+
dotnet restore "$RUNNER_TEMP/bc-sample/ProjectInputSample.csproj"
2154+
for run in 1 2; do
2155+
$x "$RUNNER_TEMP/bc-sample/ProjectInputSample.csproj" -o "$RUNNER_TEMP/bc-r$run-facts.json" \
2156+
--binding-coverage "$RUNNER_TEMP/bc-restored-$run.json"
2157+
done
2158+
cmp "$RUNNER_TEMP/bc-restored-1.json" "$RUNNER_TEMP/bc-restored-2.json" \
2159+
|| { echo "FAIL: the binding-coverage sidecar is not deterministic"; exit 1; }
2160+
python - "$RUNNER_TEMP" <<'PY'
2161+
import json, sys
2162+
t = sys.argv[1]
2163+
def units(name):
2164+
return json.load(open(f"{t}/{name}.json"))["units"]
2165+
[u] = units("bc-unrestored")
2166+
assert u["unit_status"] == "UNSUPPORTED" and u["reasons"]["unsupported"] == ["no-assets"], u
2167+
[u] = units("bc-dir")
2168+
assert u["unit_status"] == "UNSUPPORTED" and u["reasons"]["unsupported"] == ["input-kind:directory"], u
2169+
[u] = units("bc-restored-1")
2170+
assert u["reference_status"] == "COMPLETE", u["reasons"]
2171+
assert u["foreign_binding"] == 0 and u["invariants_failed"] == [], u
2172+
# fail closed: COMPLETE only with no reason on any axis
2173+
reasons = u["reasons"]["reference"] + u["reasons"]["source_set"] + u["reasons"]["unsupported"]
2174+
assert (u["unit_status"] == "COMPLETE") == (not reasons), (u["unit_status"], reasons)
2175+
assert u["sites"]["event_assign"]["event"] >= 1, u["sites"]
2176+
print("OK: PROD0 sidecar —", u["unit_status"], u["tfm"], u["tfm_status"], "reasons:", reasons)
2177+
PY
2178+
echo "OK: --binding-coverage leaves facts byte-identical; UNSUPPORTED inputs recorded; restored sample measured"
21352179
# explain: the diagnostic-catalogue CLI surface lives in the core (one checker). Smoke
21362180
# it end-to-end — explain a code, and harvest+explain every code in a real findings file.
21372181
- name: explain command (code + --json harvest)
Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
# P-014 PROD0 (coverage only) — pre-registration
2+
3+
Written 2026-10-21 12:40 +05, before any gate run and before any production change. Base: `main` `22b29d5b`
4+
(merge of #422). Owner decision: `GO-P014-PROD0-COVERAGE-ONLY`. PROD1 (diagnostic semantics) stays forbidden.
5+
6+
**Budget.** 0.5 agent-day to the next kill-gate, counted from 12:35. STOP at 16:35, not extended. When the budget
7+
runs out, I commit what exists and the verdict is `INCONCLUSIVE-P014-PROD0`.
8+
9+
## 1. What will be built (only if G0 and G1 pass)
10+
11+
**Flag.** An opt-in extractor flag `--binding-coverage <file>`, for `.csproj` inputs. It writes the deterministic
12+
coverage sidecar of the Tiny PoC (#422) into `<file>`:
13+
- one `CSharpCompilation` **per project**, from the existing `obj/project.assets.json`, local targeting packs and
14+
the global packages folder, with project references as `CompilationReference`s;
15+
- binding-site counts, unit status, reasons, reference manifest and conflicts;
16+
- the foreign-binding guard.
17+
18+
**The mechanism.** It is the #422 instrument's code, moved into the extractor as one new file (types public so
19+
that the evidence harness can call the same code). Behaviour changes allowed:
20+
- **Fail closed.** `unit_status = COMPLETE` requires all of the following: reference status `COMPLETE`, **and** no
21+
source-set reason, **and** no unsupported item. Every other case is `UNSUPPORTED`, `REFERENCE_MISSING`
22+
(including `CONFLICT_UNRESOLVED`) or `SOURCE_SET_INCOMPLETE`.
23+
- **No TPA substitution.** The process TPA is never used as a reference in this mode.
24+
- **Unsupported inputs.** A directory, a `.sln` or a `.cs` input gets a sidecar with a single `UNSUPPORTED` record
25+
and no measurement.
26+
- **Separate from facts.** The sidecar is computed **after** the facts document is written, from fresh
27+
compilations, and it never feeds the existing analysis.
28+
29+
**Unchanged:**
30+
- the default behaviour, every existing flag, directory mode;
31+
- OwnIR, OWN001/OWN014/OWN050;
32+
- SARIF and exit codes;
33+
- the facts document: byte-identical with and without the flag;
34+
- the core, the Rust engine, goldens.
35+
36+
A sidecar write failure is reported on stderr and does **not** change the exit code.
37+
38+
## 2. Gates, in order (fixed now)
39+
40+
### G0 — package-vs-package conflict against the SDK oracle (first, before any integration)
41+
42+
**Fixture.** My own synthetic project, `net10.0-windows` with `UseWPF`, referencing `unchihugo.WPF-UI` 4.4.2 and
43+
`WPF-UI.Tray` 4.3.0. These are the packages that produce P5's `Wpf.Ui.dll` conflict: `unchihugo.WPF-UI` 4.4.2,
44+
AssemblyVersion 4.4.1.0, against the transitive `WPF-UI` 4.3.0, AssemblyVersion 4.3.0.0.
45+
- It is restored offline, with the global packages folder as the only source.
46+
- The oracle is `dotnet msbuild -t:FindReferenceAssembliesForReferences -getItem:ReferencePathWithRefAssemblies`.
47+
48+
**PASS.** The #422 instrument's manifest equals the oracle's, compared by path and assembly version, including
49+
which `Wpf.Ui.dll` is chosen.
50+
51+
**Outcomes:**
52+
- **Mismatch.** The resolver is **not** tuned to fit. The supported set is narrowed instead: any package-vs-package
53+
conflict on the same file name becomes `CONFLICT_UNRESOLVED`, so the unit is not `COMPLETE`. That narrowing is
54+
then what gets integrated.
55+
- **The fixture cannot restore offline.** G0 is `INCONCLUSIVE`, and the same narrowing applies.
56+
57+
### G1 — the sidecar does not call P4/P5 fully checked
58+
59+
Run on P4 and P5 with the #422 instrument data (`data/real/P*.exact.sidecar.json`).
60+
61+
**PASS.** Every unit has `unit_status != COMPLETE`. Today's output has every unit at `SOURCE_SET_INCOMPLETE` and
62+
reference status `COMPLETE`.
63+
64+
**KILL if** any unit is `COMPLETE` while it has a named source-set reason.
65+
66+
### G2 — compatibility (after integration)
67+
68+
1. **Facts.** For P4 and P5 (`.csproj`, `--flow-locals --no-project-refs`), and for the extractor's sample tree
69+
`frontend/roslyn/samples` (directory mode), the facts document is **byte-identical** with and without
70+
`--binding-coverage`. It must also equal the extractor built from `main` `22b29d5b` without the flag.
71+
2. **Controls.** K1, K2, K4, K5, K6, K7, K11, K12, F2.3, F2.4, F2.5 and the guard self-test are run against the
72+
**production** code: the evidence harness references the built extractor assembly. K3, K8a, K8b, K8c, K9, K10
73+
and G0 run through the production flag and are compared with the SDK oracle.
74+
- All results must equal #422's.
75+
- G0 must equal the oracle, or the narrowed rule must apply.
76+
3. **Isolation.** `foreign_binding = 0` in every control and in P4/P5.
77+
4. **Determinism.** The sidecar is byte-identical across 3 runs for P4 and P5.
78+
5. **Repository gates.**
79+
- `ruff check .` passes.
80+
- The repository's Python test suite passes, as far as it runs on this host.
81+
- The extractor builds with no new warnings.
82+
83+
### G3 — cost (limits not loosened from the Tiny PoC)
84+
85+
- **Where.** P5 and P4, 3 separate processes each, through the production flag.
86+
- **Wall time.** Measured with and without the flag. The added time for the flag must be ≤ **3 ×** the #422
87+
`legacy-merged` warm median measured in the same session. In practice, the exact-mode warm cost must stay within
88+
the #422 limit: P5 warm exact ≤ 3 × legacy.
89+
- **Peak working set.** ≤ **2.0 GB**.
90+
- **Reported:** the extractor's total wall time with and without the flag.
91+
92+
## 3. Verdict
93+
94+
- **`PASS-P014-PROD0`.** G0 is PASS, or G0 is resolved by the pre-registered narrowing, **and** G1, G2 and G3 hold.
95+
- **`KILL-P014-PROD0`.** Any of:
96+
- a facts byte difference;
97+
- a foreign binding, or a silent false binding;
98+
- G1 fails;
99+
- the cost limit is exceeded;
100+
- a forbidden action is needed.
101+
- **`INCONCLUSIVE-P014-PROD0`.** The budget runs out, or a gate is undecidable because an input is missing.
102+
103+
A PASS allows **only** the opt-in coverage flag. It does not switch the extractor to the new resolver, and it does
104+
not feed any new fact to diagnostics.
105+
106+
## 4. Not done
107+
108+
- Restore or build of third-party projects; any network access.
109+
- MSBuild evaluation of P4/P5.
110+
- Changes to OwnIR, the core, the Rust engine, goldens or SARIF.
111+
- Reopening #407.
112+
- Any change to the lifetime model.
Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# P-014 PROD0 — adversarial review falsifiers (pre-registration addendum)
2+
3+
Written 2026-10-21 15:12 +05, before any run.
4+
- **Trigger.** The owner's code review of #423 at `f84943d`: `CI-PASS / CODE-REVIEW-HOLD / NO-MERGE`.
5+
- **Budget.** 0.1 agent-day, counted from 15:11. STOP at 16:00.
6+
- **Out of scope.** No sweep, no PROD1.
7+
8+
The original pre-registration (`00-preregistration.md`, `6ec551f`) is **not** changed. This file adds three
9+
falsifiers against the binary built from `f84943d`'s sources.
10+
11+
| # | Fixture (own, synthetic, restored offline) | The claim under test | Falsified if |
12+
|---|---|---|---|
13+
| F-PROD0-1 | `net8.0`, no `<DefineConstants>`, a source with `#if NET8_0 … #else … #endif` | a unit whose measured preprocessor branch may differ from the SDK's is never `COMPLETE` | the sidecar says `COMPLETE` |
14+
| F-PROD0-2 | `net8.0`, a source line `Console.WriteLine("/*probe:");` | text in an analysed source cannot stop the sidecar from being written | no sidecar is written, or the run fails |
15+
| F-PROD0-3 | `net8.0`, one `.cs` made unreadable for the current user (ACL deny read) | `files.unreadable > 0` excludes `COMPLETE` | `unreadable > 0` together with `COMPLETE` |
16+
17+
## Consequence, fixed now
18+
19+
- **F-PROD0-1 confirmed.** This is a silent false claim of completeness, which falls under PROD0's original KILL
20+
("silent false binding"). The report then records **`KILL-P014-PROD0`** for `f84943d`. That KILL stands even if a
21+
fix follows.
22+
- **F-PROD0-2 or F-PROD0-3 confirmed.** Recorded as defects, not as a KILL by themselves.
23+
- **Fix gate.** It is allowed only as a new, separately recorded gate: the fix plus F-PROD0-1, F-PROD0-2 and
24+
F-PROD0-3 re-run on the new head, plus all original G0–G3 checks re-run. The original PASS is not re-issued for
25+
`f84943d`. Any new head needs CI again.
26+
- **Fix shape, fixed now (fail closed, no new semantics):**
27+
- any `#if`/`#elif` directive in a unit's sources becomes `SOURCE_SET_INCOMPLETE(preprocessor-conditional)`;
28+
- any unreadable source becomes `SOURCE_SET_INCOMPLETE(unreadable-source)`;
29+
- the research probe-marker parsing is removed from the production path (kept only behind an explicit harness
30+
switch, and bounds-checked).
31+
32+
No defines and no implicit usings are applied (#407 stays separate).
Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
# P-014 PROD0 — adversarial review result
2+
3+
**Verdict for `f84943d`: `KILL-P014-PROD0`.**
4+
- **What it replaces.** The `PASS-P014-PROD0` in `REPORT.md` was issued on G0–G3. It did not survive the owner's
5+
code review.
6+
- **Rule.** Fixed in `01-review-falsifiers.md`, commit `b446be4` (15:11:50 +05), before any run.
7+
- **What is unchanged.** The original pre-registration and `REPORT.md` stay as they were, as the record of what
8+
was claimed.
9+
10+
## Runs [run, 15:13 +05]
11+
12+
- **Binary.** The extractor built from `f84943d`'s sources. `BindingCoverage.cs` and `Program.cs` were compared
13+
byte for byte with `git show f84943d:…`.
14+
- **Fixtures.** `fixtures/F1`, `F2`, `F3`: `net8.0`, restored offline from the global packages folder.
15+
- **Log.** `data/review_f84943d.log`.
16+
17+
| # | Observed | Claim | Result |
18+
|---|---|---|---|
19+
| F-PROD0-1 | `#if NET8_0 … #else … #endif`, no `<DefineConstants>`: exit 0, sidecar written, **`unit_status = COMPLETE`**, no source-set reason | a unit whose measured preprocessor branch may differ from the SDK's is never `COMPLETE` | **falsified.** The SDK defines `NET8_0` for `net8.0` [owner review, SDK behaviour]. The extractor parses with no defines, so it measured the `#else` branch and called the unit complete |
20+
| F-PROD0-2 | a source line `System.Console.WriteLine("/*probe:");`: exit 0, **no sidecar**, stderr `ArgumentOutOfRangeException: length ('-56')` | text in an analysed source cannot stop the sidecar | **falsified.** The research probe-marker parsing in `Coverage.Measure` runs in production and does not check for the closing `*/` |
21+
| F-PROD0-3 | `B.cs` denied read by ACL: exit 0, sidecar written, **`files.unreadable = 1` and `unit_status = COMPLETE`** | `unreadable > 0` excludes `COMPLETE` | **falsified.** `UnitStatus` does not read `Unreadable`, and the source-set scan skips the unreadable file silently |
22+
23+
**Not affected.** The facts document and the exit code. The flag still never changes them; F-PROD0-2's failure was
24+
caught and reported on stderr, as designed. The defects are in what the sidecar **claims**.
25+
26+
## A further limit (from the review, by reading)
27+
28+
`foreign_binding` is computed against an allowlist built from the **same compilation's** references
29+
(`Coverage.Allowed`). An assembly wrongly put into the manifest is therefore "allowed" by construction.
30+
- The counter catches leaks **between source units**; the #422 guard self-test shows that.
31+
- It does **not** independently prove that the manifest is right. The SDK-oracle comparisons (K3, K8a–c, K9, G0)
32+
are the evidence for the manifest; `foreign_binding = 0` is not.
33+
- `REPORT.md` and #423 should not be read as claiming more.
34+
35+
## Proposed fix gate `FIX0` (not started; needs the owner's GO)
36+
37+
**Scope.** Fail-closed changes only. No defines and no implicit usings applied, no new semantics, no PROD1.
38+
39+
| Change | Rule |
40+
|---|---|
41+
| Preprocessor | any `#if`/`#elif` directive in a unit's sources, found when the source set is scanned (so it propagates to project-reference parents), gives `SOURCE_SET_INCOMPLETE(preprocessor-conditional)` |
42+
| Unreadable source | a source the scan cannot read gives `SOURCE_SET_INCOMPLETE(unreadable-source)`; `Compile` also adds it if a later read fails |
43+
| Probe markers | removed from the production path. `Coverage.Measure` records probes only when the harness passes an explicit switch, and then with a bounds check |
44+
| `foreign_binding` | renamed in the report text only, as "cross-unit leak guard"; the sidecar field is unchanged. The manifest-correctness claim stays with the oracle checks |
45+
46+
**Acceptance (all on the new head):**
47+
1. **The three falsifiers.**
48+
- F-PROD0-1: `SOURCE_SET_INCOMPLETE` with `preprocessor-conditional`.
49+
- F-PROD0-2: the sidecar is written.
50+
- F-PROD0-3: `SOURCE_SET_INCOMPLETE` with `unreadable-source`.
51+
2. **G0–G3 re-run unchanged:**
52+
- facts byte-identical with `main`;
53+
- K1–K12 and the oracle fixtures;
54+
- P4/P5 not `COMPLETE`;
55+
- determinism;
56+
- cost within the original limits.
57+
3. **Expected movements.** The only expected change in K-results is K7, whose `#if` now marks the unit
58+
`SOURCE_SET_INCOMPLETE`. Any other difference is a FAIL.
59+
4. **CI** green again on the new head; `Owen: NEUTRAL` is the only accepted exception.
60+
61+
**Budget, proposed.** 0.25 agent-day. **KILL** if any of:
62+
- a falsifier still holds;
63+
- facts change;
64+
- another unexpected K-result changes.

0 commit comments

Comments
 (0)