|
| 1 | +# P-014 reference-resolution coverage — report |
| 2 | + |
| 3 | +**Decision: `GO-P014-BOUNDED-DESIGN`**, by the pre-registered thresholds. The GO has narrow content: the measured |
| 4 | +gain in Own.NET diagnostics is small (§5). |
| 5 | + |
| 6 | +Rules: [`00-preregistration.md`](00-preregistration.md), commit `2e73767b` (04:43 +05; the "04:45" in its header is |
| 7 | +my clock reading, and the commit time is the authoritative one). Own.NET `main` `c6e8a3c1`; extractor built offline |
| 8 | +from that commit. Work: 04:37–04:52 +05 on 2026-10-21, inside the 0.25 agent-day budget. |
| 9 | + |
| 10 | +No production, OwnIR, golden or CI change. No restore, build or MSBuild evaluation of third-party code, no package |
| 11 | +download, no `MSBuildWorkspace`. Only artifacts already on disk were read. |
| 12 | + |
| 13 | +## 1. The five answers |
| 14 | + |
| 15 | +**1. How much does Own.NET suffer from wrong references?** At the binding level, a lot. In mode A (today's |
| 16 | +compilation) only **32–48% of call sites bind in 4 of 5 projects**; P2 is the exception at 89%. Correct |
| 17 | +project-specific references (mode B) recover 5–31% of all call sites in 4 projects. Even after B, 35–52% of calls in |
| 18 | +P1, P3, P4 and P5 stay unbound. |
| 19 | + |
| 20 | +**2. Which diagnostic classes lose coverage?** |
| 21 | +- Event-subscription facts are hit hardest: `+=` sites that bind to an `IEventSymbol` go 10→35 in P1, 0→5 in P4 and |
| 22 | + 11→48 in P5. The lost ones surface as OWN050 "cannot verify" advisories. |
| 23 | +- Disposable fields and locals of external types are also lost: OWN001 on a WinForms dialog and on a |
| 24 | + `WindowsMediaController.MediaManager` field appear only with references. |
| 25 | +- DI facts did not change in any project. |
| 26 | + |
| 27 | +**3. How many bindings are recovered correctly?** |
| 28 | +- 1,240 call sites, 1,028 type sites and 67 event sites are recovered by references and not by implicit usings |
| 29 | + (§3). |
| 30 | +- A fixed sample of 50 recovered calls (10 per project) was read: **50/50 correct**. |
| 31 | +- No site that bound in A lost its binding in B. |
| 32 | +- 160 sites changed symbol. On reading, these are A bindings that were imprecise or **wrong for the target TFM**, |
| 33 | + not B errors (§3). |
| 34 | + |
| 35 | +**4. Does recovery reach the existing diagnostics?** Yes, but thinly. Across 5 projects: |
| 36 | +- 24 OWN050 disappear; |
| 37 | +- 9 findings appear: 1 actionable TP, 2 minor/benign TPs, 6 FPs; |
| 38 | +- 2 projects show no change at all. |
| 39 | + |
| 40 | +The chain breaks mostly because Own.NET analyses consume only `+=`, `new` of disposables and DI registrations. Most |
| 41 | +recovered calls (logging, WPF/Avalonia API, System.CommandLine) feed no analysis (§5). |
| 42 | + |
| 43 | +**5. What is cheapest?** |
| 44 | +- **Build-free resolution from existing `obj/project.assets.json`**, plus the TFM reference packs and |
| 45 | + package/framework name-conflict resolution. It recovered the bindings without any build. |
| 46 | +- **Ready-made compilation artifacts** (`bin/`, `obj/ref`, `.rsp`, binlog): none of the five projects had them, so |
| 47 | + this cannot be measured and is unavailable in practice for scanned repositories. |
| 48 | +- **A full project system** (`MSBuildWorkspace`) is not needed for what was measured. |
| 49 | + |
| 50 | +Two limits of the assets route: |
| 51 | +- it works only where a restore has already happened; |
| 52 | +- package identity is not provable from the cache (§2). |
| 53 | + |
| 54 | +A separate, comparably large and cheaper lever is the **source set**: SDK implicit usings recover 172 calls in P3, |
| 55 | +330 in P5 and 20 in P4 on their own (mode C). |
| 56 | + |
| 57 | +## 2. Inputs and what was checked in code |
| 58 | + |
| 59 | +The current-state claims are confirmed in `main` (pre-registration §0): |
| 60 | +- one `CSharpCompilation("own", …)`; |
| 61 | +- the framework set is the TPA of the `net8.0` extractor process, i.e. the 8.0.31 implementation assemblies; |
| 62 | +- `--ref-dir` and `bin/` are loaded recursively, first filename wins; |
| 63 | +- no MSBuild evaluation; |
| 64 | +- no binding metric; |
| 65 | +- default parse options, so no defines and no implicit usings. |
| 66 | + |
| 67 | +| # | Project @ SHA | TFM measured | B completeness | |
| 68 | +|---|---|---|---| |
| 69 | +| P1 | Flow.Launcher `Flow.Launcher.csproj` @ `2efbf865` (WPF) | net9.0-windows… | **TFM-APPROX** (no 9.0 pack, used 10.0.12); 70 packages with compile assets missing from the cache; 47 package assets skipped as framework-name conflicts | |
| 70 | +| P2 | git-credential-manager `Core.csproj` @ `e66693fa` | net10.0 | 20 packages missing | |
| 71 | +| P3 | nats.net `NATS.Client.JetStream.csproj` @ `176e52bc` (multi-TFM) | net8.0 | 2 packages missing | |
| 72 | +| P4 | BB84.IPTV `BB84.IPTV.M3U.Editor.csproj` @ `e9ab4f28` (desktop) | net10.0 | complete | |
| 73 | +| P5 | FluentFlyout `FluentFlyout.csproj` @ `911dfda6` (WPF) | net10.0-windows10.0.22000.0 | complete, including Windows SDK projection 10.0.22000.57 | |
| 74 | + |
| 75 | +**All B references are `APPROXIMATION`.** For every package the `sha512` in the assets differs from the |
| 76 | +`.nupkg.sha512` in the global cache: same id and version, a different package file. P1–P3's assets were written by |
| 77 | +CodeQL buildless extraction on 2026-10-17, before the D13 isolation, and their package folders are gone. |
| 78 | + |
| 79 | +Reference manifests with per-DLL SHA-256 prefixes are in `data/manifest_B_P*.tsv`, and the overview is in |
| 80 | +`data/manifest.json`. |
| 81 | + |
| 82 | +## 3. R0–R2: binding coverage (`data/r0_r1_summary_P*.json`, `data/r2_summary.tsv`) |
| 83 | + |
| 84 | +| Project | Calls bound A / B / C (of all) | Recovered by references | Fixed by usings only | Types A→B | `+=` events A→B | |
| 85 | +|---|---|---|---|---|---| |
| 86 | +| P1 | 925 / 1373 / 925 (of 2844) | **448 (15.8%)** | 0 (ImplicitUsings off) | 2081→2575 | 10→35 | |
| 87 | +| P2 | 3401 / 3462 / 3401 (of 3820) | 61 (1.6%) — below threshold | 0 | 5718→5757 | 6→6 | |
| 88 | +| P3 | 353 / 396 / 525 (of 839) | **43 (5.1%)** | 172 | 1293→1323 | 0→0 | |
| 89 | +| P4 | 169 / 228 / 189 (of 353) | **59 (16.7%)** | 20 | 267→329 | 0→5 | |
| 90 | +| P5 | 643 / 1272 / 973 (of 2003) | **629 (31.4%)** | 330 | 1788→2191 | 11→48 | |
| 91 | + |
| 92 | +Compiler errors in A versus B (CS0246+CS0234+CS0103+CS1061): |
| 93 | + |
| 94 | +| Project | A | B | |
| 95 | +|---|---|---| |
| 96 | +| P1 | 3,429 | 2,106 | |
| 97 | +| P4 | 385 | 219 | |
| 98 | +| P5 | 2,388 | 1,064 | |
| 99 | + |
| 100 | +These numbers are context only; they were not used for the verdict. |
| 101 | + |
| 102 | +**Correctness.** The sample (`data/r2_review_sample.tsv`) was read site by site. Each recovered call binds into the |
| 103 | +library the file imports: WPF `PresentationCore`/`PresentationFramework`, Microsoft.Extensions.DI/Logging/ |
| 104 | +Configuration/Hosting, System.CommandLine, Avalonia, NLog, NAudio, Windows SDK projections. Some recoveries are |
| 105 | +cascades and are also correct, e.g. `window.GetType()` once `window`'s type is known, or an overload that resolves |
| 106 | +once its argument types are known. The result is 10/10 in each of P1, P3, P4 and P5, and 10/10 in P2. "Correct" |
| 107 | +means the symbol and assembly agree with the code's imports and types. It is not checked against a real build. |
| 108 | + |
| 109 | +**Symbol changes** (`data/r2_symbol_changed.tsv`, 160 sites): |
| 110 | +- 116 are A bindings with error-typed parts made precise by B: 112 generic arguments or types that became known, and |
| 111 | + 4 arrays whose element type was an error type that A counted as "resolved". |
| 112 | +- The other 44 keep the method name but change the signature. Most are parameter types that became known. |
| 113 | +- **The TFM effect.** In P1 `TimeSpan.FromHours(5)` binds `FromHours(double)` in A (.NET 8 TPA) and |
| 114 | + `FromHours(int)` in B. That overload exists from .NET 9, and P1 targets net9. **A silently binds an overload the |
| 115 | + project does not use.** |
| 116 | +- No change was judged `RECOVERED_WRONG`. |
| 117 | + |
| 118 | +**Controls** (`data/controls/`): |
| 119 | + |
| 120 | +| Control | Result | |
| 121 | +|---|---| |
| 122 | +| K1 consistent | no change | |
| 123 | +| K2 same filename, different identity | **A's first-filename-wins loads v1, so `V2Only()` is unresolved in A**; B, with the manifest-selected v2, resolves it | |
| 124 | +| K3 TFM | B uses the selected TFM | |
| 125 | +| K5 | `event` vs `int` field `+=` distinguished in both modes | |
| 126 | +| K6 | missing type stays unresolved; nothing invented | |
| 127 | +| K7 | both modes see the `#else` branch (no defines): a source-set limit | |
| 128 | +| **K4** | two `Dup.T` in one compilation produce no "ambiguous": Roslyn binds the first one, in both modes. Definition conflicts are invisible to the single-compilation model, the extractor's included. Since A == B, no false recovery is claimed | |
| 129 | + |
| 130 | +## 4. Instrument corrections made before scoring (disclosed) |
| 131 | + |
| 132 | +1. **Framework/package name conflicts.** B initially referenced package compile assets with the same names as |
| 133 | + framework reference-pack assemblies. In P1 this gave 11,574 × CS0518 and 771 × CS0433. The pre-registration |
| 134 | + forbids that replacement, so such assets are now skipped (P1 47, P2 3, P4 1) and counted. The first output is |
| 135 | + kept as `proj-v1` in the work folder only. P5 still shows 59 × CS0433 between two framework references, which |
| 136 | + is not resolved. |
| 137 | +2. **Site keys.** Nested invocations starting at the same position overwrote each other, which undercounted calls. |
| 138 | + The key now includes the span length, and all projects and controls were re-measured. |
| 139 | +3. **Path patterns.** Two path bugs in my own driver were fixed: the `out/obj/<Project>` assets layout, and the |
| 140 | + Windows SDK projection version match. |
| 141 | + |
| 142 | +No threshold, input or rule changed. |
| 143 | + |
| 144 | +## 5. R3: does it reach Own.NET? (`data/r3/r3_P*.json`) |
| 145 | + |
| 146 | +The unchanged extractor was run with `--flow-locals --stats`, then the Python reference core. No Rust core is built on |
| 147 | +this host, so the Rust core was not run. |
| 148 | + |
| 149 | +- **A-ext:** as shipped. |
| 150 | +- **B-ext:** `OWN_EXTRA_REF_DIRS` = B's framework packs, plus `--ref-dir` per package compile folder. The TPA stays |
| 151 | + first, so B-ext is the most today's extractor can be given. |
| 152 | + |
| 153 | +| Project | OWN050 A→B | OWN001 A→B | Other | Facts | |
| 154 | +|---|---|---|---|---| |
| 155 | +| P1 | 27→16 | 5→8 | — | flow functions 5→6, flow acquires 2→3 | |
| 156 | +| P2 | 0→0 | 15→15 | — | unchanged | |
| 157 | +| P3 | 5→5 | — | — | unchanged | |
| 158 | +| P4 | 1→1 | 6→9 | — | unchanged | |
| 159 | +| P5 | 15→2 | 1→3 | OWN014 0→1 | flow functions 1→5 | |
| 160 | + |
| 161 | +Triage of the 9 new findings (code read): |
| 162 | + |
| 163 | +| Finding | Verdict | |
| 164 | +|---|---| |
| 165 | +| P5 OWN014 `SettingsManager.Current.PropertyChanged` (`SettingsWindow.xaml.cs:213`): a closeable window's handler captures `RootNavigation`/`Dispatcher` on a static singleton's event, no unsubscribe (`Closed` only nulls `instance`) | **TP, actionable** | |
| 166 | +| P5 OWN001 field `mediaManager` (`MainWindow.xaml.cs:48`), never disposed; cleanup only unsubscribes | TP-benign: the window is app-lived | |
| 167 | +| P1 OWN001 WinForms `OpenFileDialog dlg` not disposed (`SettingsPaneGeneralViewModel.cs:360`) | TP-minor (`using`) | |
| 168 | +| P1 `ProgressBar.IsVisibleChanged` (own XAML child), P1 `link.Click` (local hyperlink in own document), P5 `window.SourceInitialized` (own window), P4 `dialog.Closed` (local dialog) | FP: same lifetime | |
| 169 | +| P4 `desktop.Startup` / `desktop.Exit` | FP: app-lived source | |
| 170 | + |
| 171 | +**Where the chain `reference → symbol → fact → diagnostic` breaks:** |
| 172 | +- **symbol → fact.** Of about 1,240 recovered calls, only a handful are `+=` on events or `new` of disposables. The |
| 173 | + rest have no consuming analysis, by design. |
| 174 | +- **fact → diagnostic.** The newly visible subscriptions fall mostly into the known co-lifetime FP class (D11/D13 |
| 175 | + R5). That precision problem is not a reference problem. |
| 176 | +- **#382 (call lowering).** Not separately triggered here. P5's flow functions 1→5 show that newly typed |
| 177 | + disposable locals do enter the flow pass. Whether their calls are then lowered to guesses was not measured. |
| 178 | + |
| 179 | +## 6. Verdict against the thresholds |
| 180 | + |
| 181 | +| Criterion | Result | |
| 182 | +|---|---| |
| 183 | +| 1. ≥ 2 projects with reference-caused loss ≥ 5% of calls and ≥ 20 sites | **yes: P1, P3, P4, P5** (P2 no) | |
| 184 | +| 2. ≥ 9/10 sampled recoveries correct in those projects; K1–K6 pass | **yes**: 10/10 each; controls pass. K4 shows the conflict blind spot, but no false recovery | |
| 185 | +| 3. ≥ 1 useful change in OwnIR facts or findings | **yes**: P5 OWN014 TP; OWN050 replaced by verified facts (P5 `mediaManager` events now seen unsubscribed) | |
| 186 | +| 4. B from on-disk artifacts, no forbidden action, bounded cost without MSBuildWorkspace | **yes, as `APPROXIMATION`**: package identity unprovable; needs an existing restore | |
| 187 | + |
| 188 | +So **`GO-P014-BOUNDED-DESIGN`**. What the GO does and does not support: |
| 189 | +- The hidden-limit hypothesis is confirmed **at the binding level**. Own.NET today understands roughly a third to a |
| 190 | + half of the calls in 4 of these 5 projects, and its finding counts do not show that. |
| 191 | +- The **diagnostic** payoff of fixing references alone is small: +1 actionable TP and 2 minor ones against 6 FPs, |
| 192 | + over 5 projects. Better references mainly convert OWN050 into checked facts. |
| 193 | +- A design should be scoped as **reading existing `project.assets.json`**: select one TFM, add the reference packs, |
| 194 | + apply framework-first conflict resolution, and report binding coverage. It should be measured on implicit usings |
| 195 | + too, which are a source-set fix outside P-014's reference scope and an earlier owner decision. |
| 196 | + |
| 197 | +## 7. Files |
| 198 | + |
| 199 | +- `00-preregistration.md`, `REPORT.md` |
| 200 | +- `instrument/` (Roslyn binding instrument), `controls/make_controls.py` |
| 201 | +- `scripts/build_modes.py` (A/B/C inputs), `scripts/classify.py` (R2), `scripts/run_r3.py` (R3) |
| 202 | +- `data/`: `manifest.json`, `manifest_B_P*.tsv`, `r0_r1_summary_P*.json`, `r2_summary.tsv`, `r2_review_sample.tsv`, |
| 203 | + `r2_symbol_changed.tsv`, `r3/`, `controls/`. Local paths are replaced by `<research>`, `<work>`, `<nuget>` and |
| 204 | + `<dotnet>`. Only short expression excerpts appear; no third-party source files, DLLs or packages are committed. |
0 commit comments