|
| 1 | +# P-014 Tiny PoC — report |
| 2 | + |
| 3 | +**Verdict: `PASS-P014-TINY-POC`.** All six pre-registered criteria hold (§6). |
| 4 | +- This is a PASS for a **research instrument** that measures binding. It is not a PASS for any change to the |
| 5 | + extractor, OwnIR or public diagnostics, and it produces no findings. |
| 6 | +- Production stays NO-GO. |
| 7 | + |
| 8 | +Rules: [`00-preregistration.md`](00-preregistration.md). |
| 9 | +- Commit `5a9bb32`, 11:04:52 +05. The "11:10" in its header is my clock reading; the commit time is |
| 10 | + authoritative. |
| 11 | +- Amendment 1 is commit `207ef84`, 11:06:54. Both were committed before the instrument existed. |
| 12 | + |
| 13 | +Base `main` `228e38d7`. Work: 11:00 to about 11:25 +05 on 2026-10-21, inside the 0.25 agent-day budget. |
| 14 | + |
| 15 | +Labels used below: |
| 16 | +- **[run]** measured here; |
| 17 | +- **[read]** checked by reading code or diagnostics; |
| 18 | +- **[doc]** dotnet/sdk source; |
| 19 | +- **[inference]** not verified. |
| 20 | + |
| 21 | +## 1. What was built |
| 22 | + |
| 23 | +`instrument/` contains a research program (Roslyn 4.9.2, the extractor's version), built offline. For one root |
| 24 | +`.csproj` it does the following: |
| 25 | +- **Graph.** It reads the existing `obj/project.assets.json` of the root and of every project-reference member, |
| 26 | + and builds **one `CSharpCompilation` per project**. Project references become `CompilationReference`s. |
| 27 | +- **References.** Each unit's references are selected by the pre-registered contract (§1 of the |
| 28 | + pre-registration): |
| 29 | + - targeting-pack `FrameworkList.xml` with profiles; |
| 30 | + - package `compile` assets with the content-hash identity check; |
| 31 | + - conflicts keyed by file name, decided in the dotnet/sdk order: overrides → AssemblyVersion → FileVersion → |
| 32 | + (rank/platform). |
| 33 | +- **Output.** It writes the deterministic coverage sidecar (DESIGN §4), with a built-in isolation guard. |
| 34 | +- **Source set.** It uses the extractor's own text rules, with default parse options. **No implicit usings and no |
| 35 | + defines are applied to any real project.** They are only reported. |
| 36 | + |
| 37 | +No restore or build of third-party code, no network, no MSBuild evaluation of P4/P5. The SDK was run only as an |
| 38 | +**oracle on my own synthetic fixtures**, restored offline with the global packages folder as the only source. |
| 39 | + |
| 40 | +## 2. Environment change (recorded before the first run) |
| 41 | + |
| 42 | +The host's .NET was replaced at about 11:00, and I did not make the change: |
| 43 | +- SDK 10.0.401 became 11.0.100-rc.2; |
| 44 | +- packs 8.0.31/10.0.12 became 8.0.32/10.0.13. |
| 45 | + |
| 46 | +P4/P5 therefore resolve as `PACK_MAJOR_MINOR` against 10.0.13. Whether 10.0.12 and 10.0.13 have the same API |
| 47 | +cannot be checked any more. |
| 48 | + |
| 49 | +## 3. Controls |
| 50 | + |
| 51 | +**In memory** [run, `data/fixtures/controls.json`]: K1, K2, K4, K5, K6, K7, F2.3, F2.4, F2.5, K11, K12 and the guard |
| 52 | +self-test all **PASS**. |
| 53 | + |
| 54 | +| Control | Observed | |
| 55 | +|---|---| |
| 56 | +| K2 | `Lib` 1.0 / 2.0: each unit binds its own version; no CS1704 | |
| 57 | +| K4 | merged: CS0101. Per unit: U2's `One()` unresolved + CS1061. U3 referencing U1 and U2: type site **ambiguous via CS0433** | |
| 58 | +| F2.3 | U2's `File.Exists` unresolved (no leaked implicit usings) | |
| 59 | +| F2.4 | U2 binds `System.Timers.Timer.Elapsed` (event) | |
| 60 | +| F2.5 | U3 without `Dep`: unresolved, foreign = 0 | |
| 61 | +| GUARD | the merged shape's binding of U3's event, judged against U3's own manifest: flagged (3 foreign symbols) | |
| 62 | +| K11 / K12 | `unresolved_no_marker = 2` (lambda, `new EventHandler`); `name_heuristic = 1`, not counted resolved | |
| 63 | + |
| 64 | +**On disk, against the SDK oracle** [run, `data/fixtures/*.compare.json`]. The SDK oracle is |
| 65 | +`FindReferenceAssembliesForReferences` → `ReferencePathWithRefAssemblies`. Both sides are compared by path and |
| 66 | +assembly version. |
| 67 | + |
| 68 | +| Control | Instrument vs oracle | Conflicts decided | |
| 69 | +|---|---|---| |
| 70 | +| K8a (`net8.0` + SCI/SRM 9.0.0) | **163 = 163**, identical | both packages win, rule `assembly-version` (9.0.0.0 > 8.0.0.0) | |
| 71 | +| K3 (same fixture) | compile asset `lib/net8.0`, same as oracle | — | |
| 72 | +| K8b (`net8.0` + System.Text.Json 8.0.5) | **163 = 163** | framework wins, rule `file-version` (equal AssemblyVersion 8.0.0.0) | |
| 73 | +| K8c (`net10.0-windows`, WPF only) | **206 = 206**; no `System.Windows.Forms.dll` | `WindowsBase.dll` pack copy by `assembly-version` | |
| 74 | +| K9 (`net8.0;net10.0`) | without `--tfm`: `UNSUPPORTED(multi-tfm)`; with `--tfm`: 163 = 163 and 167 = 167 | — | |
| 75 | +| K10 (project reference) | with child assets: two `COMPLETE` units, event resolved; child `obj` removed: root `SOURCE_SET_INCOMPLETE(p2p-no-assets:Child)`, event unresolved, foreign = 0 | — | |
| 76 | + |
| 77 | +`TIE_FRAMEWORK_ASSUMED` was never needed. |
| 78 | + |
| 79 | +**Not covered by the oracle:** a **package-vs-package** conflict. P5 has one (`Wpf.Ui.dll`, decided by |
| 80 | +`assembly-version`), and it was not checked against a real build. |
| 81 | + |
| 82 | +## 4. Real projects P4, P5 (#419 anchors) |
| 83 | + |
| 84 | +**Status** [run, `data/real/P*.exact.sidecar.json`]. All 5 units (P4 root + 3 project-reference members, P5) |
| 85 | +have: |
| 86 | +- reference status `COMPLETE`; |
| 87 | +- `PACK_MAJOR_MINOR`; |
| 88 | +- `foreign_binding = 0`; |
| 89 | +- no failed invariants; |
| 90 | +- no CS0433/CS1704/CS0012/CS0518. |
| 91 | + |
| 92 | +All are `SOURCE_SET_INCOMPLETE`, with named reasons: |
| 93 | + |
| 94 | +| Reason | Units | |
| 95 | +|---|---| |
| 96 | +| `package-analyzers:*` (generators from packages, e.g. `bb84.sourcegenerators`, `communitytoolkit.mvvm`) | all 5 | |
| 97 | +| `directory-build:*` | all 5 (P4 sets `ImplicitUsings` in its own `Directory.Build.props`; P5's only hit is a **host** file above all repositories, `source/repos/Directory.Build.targets`, so the detector is conservative) | |
| 98 | +| `wpf-xaml-codegen`, `define-constants`, `implicit-usings-not-applied`, `framework-generator:LibraryImport` | P5 | |
| 99 | +| `framework-generator:GeneratedRegex` | P4 Application | |
| 100 | + |
| 101 | +**Binding coverage, `exact-project` vs `legacy-merged`** (today's shape inside the same instrument) [run]: |
| 102 | + |
| 103 | +| | P4 (4 units) | P5 | |
| 104 | +|---|---|---| |
| 105 | +| invocations resolved | **1798 / 2901 (62.0%)** vs 729 / 2901 (25.1%) | **1284 / 2003 (64.1%)** vs 643 / 2003 (32.1%) | |
| 106 | +| type sites resolved | 1597 / 2341 vs 1183 / 2341 | 775 / 1049 vs 299 / 1049 | |
| 107 | +| `+=` bound to an event | 23 of 34 (1 unresolved) vs 7 (17 unresolved, 13 with no OWN050 marker) | 33 of 65 (8 unresolved, all without marker) vs 7 (34 unresolved, 19 without marker) | |
| 108 | +| disposable-type sites | 24 resolved, 0 by name vs 0 | 13 resolved + 3 by name vs 3 + 3 | |
| 109 | +| methods whose relevant sites all resolve | 20 / 23 | 23 / 37 | |
| 110 | +| error diagnostics | 1667 vs 2595 | 1352 vs 2741 | |
| 111 | + |
| 112 | +**Comparison with #419 B** (counts only, per Amendment 1): |
| 113 | +- **P5.** Same denominator, 2003 calls. #419 B resolved 1272; exact-project resolves 1284, under the stricter rule |
| 114 | + that no error may sit on the site. #419 B had 59 × CS0433 "between two framework references"; exact-project has |
| 115 | + 0. The cause is probably the WPF profile filter [inference]. |
| 116 | +- **P4 root.** #419 counted 353 calls over 23 files. The extractor's rules, and so this instrument, exclude |
| 117 | + `Properties/Resources.Designer.cs`, which holds 129 `GetString`/`GetObject` calls [read]. Hence 223. Type-site |
| 118 | + definitions differ between the instruments, so type counts are not compared. |
| 119 | + |
| 120 | +**Sites resolved in legacy and not in exact (`lost`)** [run, `data/real/P*.lost.json`, `P4.lost.diag.txt`]: |
| 121 | +- **Counts.** P5: 0. P4: 22. All 22 were read, more than the 10 required. |
| 122 | +- **Cause.** |
| 123 | + - `ImplicitUsings=enable` is set in P4's `Directory.Build.props` and is not applied (owner rule). So `Task`, |
| 124 | + `Exception`, `IDisposable` and `IHttpClientFactory` are error types **in the Application unit's own |
| 125 | + declarations**. |
| 126 | + - Across the `CompilationReference`, the root reports them **at the use site**: CS0246 at 15 call and type |
| 127 | + sites. In Infrastructure, the interface signatures no longer match their implementations: CS0535 at 7 base-list |
| 128 | + sites. |
| 129 | + - The legacy merged compilation binds the same methods and types, but reports the error only once, at the |
| 130 | + declaration, so it counts a call whose signature contains error types as "resolved". |
| 131 | +- **Verdict.** In every case exact-project's result is **unresolved, not a different symbol**. None is a wrong |
| 132 | + binding. The difference is an **over-count in legacy**: a source-set limit shows up honestly per project instead |
| 133 | + of being hidden. |
| 134 | + |
| 135 | +**The 7 #419 R3 sites** [run, `data/real/R3.sites.txt`] are all `+=` sites bound to an `IEventSymbol` in |
| 136 | +exact-project: |
| 137 | + |
| 138 | +| Site | Bound event | |
| 139 | +|---|---| |
| 140 | +| P5 actionable OWN014 | `CommunityToolkit.Mvvm.ComponentModel.ObservableObject.PropertyChanged` | |
| 141 | +| P5 FP | `Window.SourceInitialized` | |
| 142 | +| P4 FPs | `IControlledApplicationLifetime.Startup`/`.Exit`, `TopLevel.Closed` | |
| 143 | +| P5 TP-benign `mediaManager` field | type `WindowsMediaController.MediaManager`, resolved | |
| 144 | + |
| 145 | +The six FPs are therefore correctly bound events, which confirms DESIGN §5: their precision problem is the |
| 146 | +lifetime model, not references. No finding was computed and none changed: the extractor was not run with this |
| 147 | +mechanism. |
| 148 | + |
| 149 | +## 5. Cost |
| 150 | + |
| 151 | +[run, `data/real/*.timing.json`]. Each figure is the median of 3 separate processes; each process runs a cold |
| 152 | +iteration (first build in the process) and a warm iteration (second full build, metadata cache kept). Peak is the |
| 153 | +highest process peak working set. |
| 154 | + |
| 155 | +| | legacy cold / warm | exact cold / warm | exact / legacy warm | peak (legacy / exact) | |
| 156 | +|---|---|---|---|---| |
| 157 | +| P4 (4 units) | 2260 / 1611 ms | 2031 / 1642 ms | 1.02× | 232 / 202 MB | |
| 158 | +| **P5** | 1548 / 969 ms | 1776 / 1386 ms | **1.43×** (limit 3×) | 209 / 208 MB (limit 2.0 GB, 3×) | |
| 159 | + |
| 160 | +**Context only** [run, `data/real/extractor_context.log`]. Today's extractor, built offline from `228e38d7`, with |
| 161 | +`--flow-locals --no-project-refs`: |
| 162 | +- P4: about 1.0 s, about 171 MB; |
| 163 | +- P5: about 2.5 s, about 202 MB. |
| 164 | + |
| 165 | +**Its facts are byte-identical across 3 runs** (P4 `AD528B8A1E92F4C8…`, P5 `5F0EA76DA86C623A…`). The instrument never |
| 166 | +touches them. |
| 167 | + |
| 168 | +## 6. Verdict against the pre-registered criteria |
| 169 | + |
| 170 | +| # | Criterion | Result | |
| 171 | +|---|---|---| |
| 172 | +| 1 | Isolation: `foreign_binding = 0` everywhere; K2, K4, F2.3, F2.4, F2.5 | **holds**; guard self-test catches the merged-shape binding | |
| 173 | +| 2 | References: K8a/b/c, K9, K3 = oracle; no tie assumption; K10 | **holds** | |
| 174 | +| 3 | Honest coverage: K5, K6, K7, K11, K12; invariants; byte-identical sidecars on rerun | **holds**: sidecars identical across 3 processes for every mode × project | |
| 175 | +| 4 | Real projects: complete without crash; named statuses; every lost site read has a cause, none a wrong binding | **holds**: 22/22 read, one cause | |
| 176 | +| 5 | Cost: P5 warm ≤ 3× legacy; peak ≤ 2.0 GB and ≤ 3× legacy | **holds**: 1.43×; about 208 MB | |
| 177 | +| 6 | Boundary: nothing outside `docs/evidence/p014-tiny-poc-2026-10/` | **holds** (`git diff --stat 228e38d7`) | |
| 178 | + |
| 179 | +**`PASS-P014-TINY-POC`.** |
| 180 | + |
| 181 | +## 7. Deviations and defects (disclosed) |
| 182 | + |
| 183 | +1. **Amendment 1.** The #419 per-site B lists were never committed. Per-site comparison was replaced by legacy → |
| 184 | + exact lost sites, before any run. |
| 185 | +2. **Instrument defects, fixed before any real-project run:** |
| 186 | + - a type node inside `new T()` has no `TypeInfo`, so `GetSymbolInfo` is used; |
| 187 | + - the K1 check depended on probe order (the observation was already correct); |
| 188 | + - the comparator read the oracle's package version from `AssemblyVersion` metadata, which is empty for |
| 189 | + packages; `FusionName` is used instead. |
| 190 | +3. **A first fixture run used a stale binary.** The fixtures had been copied inside the instrument's build folder, |
| 191 | + so the build picked up their `.cs` files and failed. That run was discarded. Every reported fixture result comes |
| 192 | + from `scripts/run_fixtures.ps1` on a clean folder. |
| 193 | +4. **The `diag` mode** (errors and sites on one line) was added after the real runs, for cause analysis only. The |
| 194 | + measured `project`/`lost` code is unchanged from commit `35e2c8c`. |
| 195 | +5. **Source-set detection is conservative and coarse:** |
| 196 | + - any `Directory.Build.*` above the project counts, including a host file; |
| 197 | + - `ImplicitUsings` set in `Directory.Build.props` is reported only as `directory-build`. |
| 198 | + |
| 199 | +## 8. What this does not show |
| 200 | + |
| 201 | +- Any effect on findings, precision or OwnIR: nothing was emitted or judged. |
| 202 | +- Package-vs-package conflict correctness, the rank/platform steps, and RID-specific or `netstandard`/.NET Framework |
| 203 | + targets. |
| 204 | +- That `PACK_MAJOR_MINOR` (10.0.13) equals the pack P4/P5 were built with. |
| 205 | +- Generality beyond P4, P5 and the fixtures. P1–P3 were not run. |
| 206 | +- The cost of very large closures. P4 has 4 units. |
| 207 | + |
| 208 | +## 9. Files |
| 209 | + |
| 210 | +- `00-preregistration.md`, `REPORT.md` |
| 211 | +- `instrument/`: `Units.cs` (assets, packs, conflicts, source set), `Coverage.cs`, `Controls.cs`, `Program.cs`, |
| 212 | + `P014Poc.csproj` |
| 213 | +- `fixtures/`: K8a, K8b, K8c, K9, K10, my own synthetic projects |
| 214 | +- `scripts/`: `run_fixtures.ps1`, `run_real.ps1`, `run_extractor_context.ps1`, `compare_oracle.py` |
| 215 | +- `data/fixtures/`: controls, sidecars, oracle comparisons, log |
| 216 | +- `data/real/`: sidecars (run 1 of each mode), all timing files, lost sites, diagnostics at lost sites, R3 sites, |
| 217 | + logs |
| 218 | + |
| 219 | +Local paths are replaced by `<nuget>`, `<dotnet>`, `<research>` and `<work>`. The oracle JSON files themselves are |
| 220 | +not committed; their comparison results are. |
0 commit comments