Skip to content

Commit 22b29d5

Browse files
PhysShellPhysShell
authored
Merge pull request #422 from PhysShell/research/p014-tiny-poc
research(p014): validate project-isolated reference resolution and binding coverage
2 parents 228e38d + a685402 commit 22b29d5

60 files changed

Lines changed: 25029 additions & 0 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎docs/evidence/p014-tiny-poc-2026-10/00-preregistration.md‎

Lines changed: 259 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,259 @@
1+
# P-014 Tiny PoC — pre-registration
2+
3+
Written 2026-10-21 11:10 +05, before the instrument exists and before any measuring run. Base: `main` `228e38d7`
4+
(merge of #421). Owner decision: `GO-P014-TINY-POC`, with production still NO-GO.
5+
6+
**Budget.** 0.25 agent-day, counted from 11:00. STOP at 13:00, not extended. When the budget runs out, I commit
7+
the evidence I have and the verdict is `INCONCLUSIVE-P014-TINY-POC`, unless a KILL condition has already been
8+
met.
9+
10+
**Verdict.** Exactly one of `PASS-P014-TINY-POC`, `KILL-P014-TINY-POC`, `INCONCLUSIVE-P014-TINY-POC`. After a
11+
KILL I stop and keep the evidence. Conditions are not changed to reach a PASS.
12+
13+
## 0. Environment change, recorded before the first run
14+
15+
The host's .NET installation was replaced at about 10:59–11:00 today. I did not make this change.
16+
17+
| | Before (#419, #421) | Now |
18+
|---|---|---|
19+
| SDK | 10.0.401 | **11.0.100-rc.2.26504.105** only |
20+
| `Microsoft.NETCore.App.Ref` / `WindowsDesktop.App.Ref` / `AspNetCore.App.Ref` | 8.0.31, 10.0.12 | **8.0.32, 10.0.13**, 11.0.0-rc.2 |
21+
22+
Consequences, fixed now:
23+
- **P4 and P5** resolve against 10.0.13. Their assets record no pack patch, so their status is
24+
`PACK_MAJOR_MINOR` (DESIGN §2 D1).
25+
- **The #419 B manifests** (10.0.12) are no longer reproducible byte for byte, and their pack-DLL hashes are
26+
expected to differ. The comparison with #419 is therefore made by **assembly simple name**, not by hash.
27+
- **API identity between 10.0.12 and 10.0.13** cannot be checked, because 10.0.12 is gone. That remains an open
28+
question.
29+
30+
## 1. The mechanism under test (research instrument only)
31+
32+
A new research program in `instrument/`, built offline. It never touches the extractor, the core, OwnIR, CI or
33+
goldens.
34+
35+
**Unit.** One SDK-style `PackageReference` project at one target. Each unit gets:
36+
- its own `CSharpCompilation`;
37+
- its own source set, which is the extractor's text resolution (`ProjectCsFiles` rules, ported);
38+
- default `CSharpParseOptions`, the same as the extractor: **no defines and no implicit usings in any real
39+
project**;
40+
- its own references.
41+
42+
Project references become `CompilationReference`s to the referenced unit's compilation, built first.
43+
44+
**Target selection.** Use the RID-less key of `targets` in `obj/project.assets.json`.
45+
- One key: that key.
46+
- More than one: `--tfm` is required, otherwise `UNSUPPORTED(multi-tfm)`.
47+
- RID keys are ignored for compile.
48+
- For a referenced project, take the target whose framework equals the root's target entry `framework`. If there
49+
is no unique match, the result is `SOURCE_SET_INCOMPLETE(p2p-target)`.
50+
51+
### Framework references (contract: dotnet/sdk `ResolveTargetingPackAssets`)
52+
53+
- **Targeting packs.** Each name in `project.frameworks[alias].frameworkReferences` maps to a targeting pack:
54+
- `Microsoft.NETCore.App` → `Microsoft.NETCore.App.Ref`;
55+
- `Microsoft.AspNetCore.App` → `Microsoft.AspNetCore.App.Ref`;
56+
- `Microsoft.WindowsDesktop.App[.WPF|.WindowsForms]` → `Microsoft.WindowsDesktop.App.Ref`, profile
57+
`WPF`/`WindowsForms`;
58+
- `Microsoft.Windows.SDK.NET.Ref.Windows` → NuGet `Microsoft.Windows.SDK.NET.Ref`, profile `Windows`.
59+
60+
Any other name gives `UNSUPPORTED(framework-reference)`.
61+
- **Pack version.**
62+
- The exact version in `downloadDependencies` if listed (`PACK_EXACT`).
63+
- Otherwise the highest installed patch of the target's `major.minor` (`PACK_MAJOR_MINOR`).
64+
- None installed: `REFERENCE_MISSING(pack)`, with no substitution.
65+
- **Files** come from the pack's `data/FrameworkList.xml`:
66+
- skip `Type="Analyzer"` and `ReferencedByDefault="false"`;
67+
- when a profile is requested, take only files whose `Profile` list contains it (files with no `Profile` are
68+
skipped);
69+
- with no profile, take all;
70+
- the path is the pack root plus `Path`.
71+
- **Conflict item.** `AssemblyVersion` and `FileVersion` come from FrameworkList; the package id is the pack id.
72+
- **Overrides.** The pack's `data/PackageOverrides.txt` makes the **pack id** override each listed package up to
73+
its version, inclusive.
74+
75+
### Package references
76+
77+
- Every `type: package` entry of the chosen target, `compile` items only, `_._` excluded, from the global
78+
packages folder.
79+
- Absent: `REFERENCE_MISSING(package)`.
80+
- Content hash differs from `.nupkg.metadata` `contentHash`: `REFERENCE_MISSING(package-identity)`.
81+
- `AssemblyVersion` and `FileVersion` are read from the DLL itself.
82+
83+
### Conflict rule (contract: dotnet/sdk `ResolvePackageFileConflicts` + `ConflictResolver`)
84+
85+
References conflict when their **file names** are equal, case-insensitive. A pair is decided by the first rule
86+
that applies:
87+
1. package override, as above;
88+
2. if one file is missing: unresolved;
89+
3. higher `AssemblyVersion`; if only one has one: unresolved;
90+
4. higher `FileVersion`; if only one has one: unresolved;
91+
5. package rank. The SDK's `PackageConflictPreferredPackages` list is not on disk in a form I read, so a framework
92+
item beats a package item here **as an assumption**, labelled `TIE_FRAMEWORK_ASSUMED` and counted;
93+
6. platform item (not reached for compile references from targeting packs);
94+
7. otherwise `CONFLICT_UNRESOLVED`. The unit is then not `COMPLETE`.
95+
96+
Two package items with equal versions are `CONFLICT_UNRESOLVED`. The copy-local tie-break is a runtime rule and is
97+
not used.
98+
99+
### Source-set completeness (separate axis, never merged with references)
100+
101+
`SOURCE_SET_INCOMPLETE(<reason>)` is set when any of these hold:
102+
- `UseWPF`/`UseWindowsForms` with `.xaml`/`.resx` designer generation;
103+
- a package with `analyzers/dotnet/cs`;
104+
- framework generator attributes in the sources: `LibraryImport`, `GeneratedRegex`, `JSImport`/`JSExport`,
105+
`GeneratedComInterface`;
106+
- `Directory.Build.props` or `Directory.Build.targets` above the project;
107+
- a non-empty `DefineConstants`;
108+
- `ImplicitUsings` set to `enable` or `true` (reported, **not applied**);
109+
- a project-reference member without assets.
110+
111+
### Coverage sidecar (DESIGN §4)
112+
113+
- **Per unit:**
114+
- status, reasons and manifest (with chosen conflict winners);
115+
- site counts for invocations, types and event `+=`, with `unresolved_no_marker` (G1);
116+
- disposable-type sites, with `name_heuristic` (G2);
117+
- compiler errors by id;
118+
- file counts.
119+
- **Site status:**
120+
- `SYMBOL_RESOLVED`: `Symbol != null`, no error type in the containing type, and no error diagnostic
121+
intersecting the site;
122+
- `SYMBOL_AMBIGUOUS`: `CandidateReason` ambiguous with ≥ 2 candidates, or CS0104/CS0121/CS0229/CS0433 at the
123+
site;
124+
- `UNRESOLVED`: otherwise.
125+
- **Format.** Deterministic JSON: sorted, relative paths, no timings.
126+
127+
**Built-in isolation check.** Every `SYMBOL_RESOLVED` symbol must have a containing assembly that is either:
128+
- the unit itself;
129+
- an assembly of the unit's own manifest; or
130+
- a unit in its own closure.
131+
132+
Every other case is counted as `foreign_binding`.
133+
134+
**Modes.**
135+
- `exact-project` (above).
136+
- `legacy-merged`: today's extractor shape. One compilation over all units' sources; references are the running
137+
process's TPA, first file name wins. It is used as the cost baseline and the comparison.
138+
139+
## 2. Controls (expected results fixed now)
140+
141+
**In-memory, unit level** (the boundary and site classification; real Roslyn, synthetic references):
142+
143+
| # | Control | Expected in exact-project |
144+
|---|---|---|
145+
| K1 | consistent BCL-only unit | `foreign_binding = 0`; no change vs legacy at its sites |
146+
| K2 | `Lib` 1.0 in U1, 2.0 in U2 | each unit binds its own version; no CS1704 |
147+
| K4 | `Dup.T` in U1 and U2; U3 → U1, U2 | legacy-merged: CS0101. U3's site `SYMBOL_AMBIGUOUS` via CS0433. U2's `One()` `UNRESOLVED` (CS1061) |
148+
| K5 | external `event` and `int` field, both `+=` | event counted `event`, field `non_event` |
149+
| K6 | type from an absent assembly | `UNRESOLVED`; `foreign_binding = 0` |
150+
| K7 | `#if NET8_0 … #else … #endif` | `#else` branch seen (no defines); reported as a source-set limit |
151+
| K-iso F2.3 | implicit usings in U1 only | U2 `File.Exists` `UNRESOLVED` |
152+
| K-iso F2.4 | `Timer` | U2 binds `System.Timers.Timer.Elapsed` (event) |
153+
| K-iso F2.5 | U3 lacks `Dep` | U3 site `UNRESOLVED`; `foreign_binding = 0` |
154+
| K11 | unresolved `x.E += (s, e) => {}` and `+= new EventHandler(H)` | `unresolved_no_marker = 2` |
155+
| K12 | unresolved `FooStream` field | `name_heuristic = 1`, not counted resolved |
156+
157+
**On disk, assets level.**
158+
- Synthetic projects under `fixtures/`, restored **offline** by the local SDK with the global packages folder as
159+
the only source.
160+
- These are my own fixtures, not third-party code.
161+
- The SDK's own resolved compile references (`ReferencePathWithRefAssemblies` after `ResolveAssemblyReferences`)
162+
are the **oracle**. The oracle is used only to judge the instrument; it is not part of the mechanism.
163+
164+
| # | Control | Expected |
165+
|---|---|---|
166+
| K3 | `net8.0` + `System.Collections.Immutable` 9.0.0 (has `lib/net8.0` and `lib/netstandard2.0`) | compile asset `lib/net8.0` |
167+
| **K8a** | `net8.0` + `System.Collections.Immutable` 9.0.0 and `System.Reflection.Metadata` 9.0.0 | **package wins** both (AssemblyVersion 9 > 8). Instrument == oracle |
168+
| **K8b** | `net8.0` + `System.Text.Json` 8.0.5 | instrument == oracle (override list or version decides). Expected framework wins, by override or version |
169+
| **K8c** | `net10.0-windows` with `UseWPF` only | no `System.Windows.Forms.dll` in the manifest (profile). Instrument == oracle on the framework file set |
170+
| K9 | `net8.0;net10.0` | `UNSUPPORTED(multi-tfm)` without `--tfm`; with `--tfm net8.0` the manifest == oracle for that TFM |
171+
| K10 | root → child project reference, child's `obj` removed after restore | root `SOURCE_SET_INCOMPLETE(p2p-no-assets)`; child types `UNRESOLVED`, not invented |
172+
173+
**K8 pass condition.** The instrument's set of reference **file names**, with the chosen **assembly identity**
174+
(name + version), equals the oracle's for K8a, K8b, K8c, K9 and K3. Analyzers are excluded on both sides.
175+
176+
## 3. Real projects (#419 anchors)
177+
178+
**P4** BB84.IPTV `BB84.IPTV.M3U.Editor` (closure of 3) and **P5** FluentFlyout, from the same on-disk trees as
179+
#419. No restore and no build.
180+
181+
Measured per project:
182+
- unit statuses and reasons;
183+
- manifest;
184+
- binding coverage;
185+
- compiler errors;
186+
- `foreign_binding`;
187+
- for each #419 R3 site of these projects (P5 OWN014 `SettingsManager.Current.PropertyChanged`, P5 OWN001
188+
`mediaManager`, P5 `window.SourceInitialized`, P4 `dialog.Closed`, P4 `desktop.Startup`/`desktop.Exit`), its
189+
binding status.
190+
191+
**Comparison with #419 B** (by simple name, §0):
192+
- call sites bound in #419 B and not in exact-project are listed;
193+
- each of the first 10 such sites, in sorted order, is read.
194+
195+
**Cost.**
196+
- 3 separate processes per mode × project.
197+
- Each process does a **cold** iteration (first in process) and a **warm** iteration (a second full build of all
198+
units, same process).
199+
- Recorded: wall time per iteration and the process peak working set.
200+
- Reported: medians.
201+
- Context only, not a criterion: today's extractor on the same `.csproj`, built offline from `228e38d7`.
202+
203+
## 4. Decision (thresholds fixed now)
204+
205+
**`PASS-P014-TINY-POC`** only if all of the following hold:
206+
1. **Isolation.**
207+
- `foreign_binding = 0` in every control and in P4/P5.
208+
- K2, K4, F2.3, F2.4 and F2.5 as expected.
209+
- No silent false binding in any control.
210+
2. **References.** K8a, K8b, K8c, K9 and K3 equal the oracle. `TIE_FRAMEWORK_ASSUMED` is never needed in K8. K10
211+
as expected.
212+
3. **Honest coverage.**
213+
- K5, K6, K7, K11 and K12 as expected.
214+
- Every sidecar invariant (DESIGN §4) holds.
215+
- Two runs of the same input give byte-identical sidecars.
216+
4. **Real projects.**
217+
- P4 and P5 complete without a crash.
218+
- Each unit's reference status is `COMPLETE` with `PACK_MAJOR_MINOR`, or a named non-`COMPLETE` reason.
219+
- Every #419-B-bound, exact-unbound site that was read has a stated cause.
220+
- No such site is a wrong binding of exact-project.
221+
5. **Cost.**
222+
- P5 exact-project **warm** median ≤ 3 × legacy-merged warm median.
223+
- Peak working set ≤ **2.0 GB** and ≤ 3 × legacy-merged peak, for P4 and P5.
224+
6. **Boundary.** No change under `frontend/`, `ownlang/`, `rust/`, `spec/`, `tests/`, `.github/` or `scripts/`.
225+
226+
**`KILL-P014-TINY-POC`** if any of:
227+
- a `foreign_binding` or a silent false binding in any control or real project;
228+
- an instrument/oracle mismatch in K8a/K8b/K8c/K9/K3 that the contract rule above cannot explain;
229+
- the cost limits in item 5 are exceeded;
230+
- a forbidden action is needed (third-party restore or build, network, MSBuild evaluation of a third-party
231+
project, production change).
232+
233+
**`INCONCLUSIVE-P014-TINY-POC`** if:
234+
- the budget runs out first; or
235+
- a criterion cannot be decided because an input is unavailable (for example, the SDK oracle cannot restore a
236+
fixture offline).
237+
238+
## 5. What I will not claim
239+
240+
- Correctness of symbols that were not read.
241+
- That `PACK_MAJOR_MINOR` equals the pack the project was built with.
242+
- Any diagnostic or verdict effect: the extractor is not run with the new mechanism, and no finding is produced by
243+
it.
244+
- Generality beyond P4/P5 and the fixtures.
245+
246+
## Amendment 1 (11:15 +05, before the instrument exists and before any run)
247+
248+
The #419 per-site B lists were never committed (only `r0_r1_summary_P*.json` / `r2_summary.tsv`), and the #419 work
249+
folder no longer exists. The per-site comparison in §3 and the item-4 reading are therefore replaced as follows:
250+
- **Against #419 B.** Counts only: resolved calls and types per project, compared with `r0_r1_summary_P4/P5.json`
251+
B.
252+
- **Per-site regression check.**
253+
- Every call or type site that is `SYMBOL_RESOLVED` in this instrument's `legacy-merged` mode and **not** in
254+
`exact-project` is listed (`lost_sites`).
255+
- The first 10 in sorted order per project are read. Each must have a stated cause: the legacy binding was
256+
wrong (net8 implementation, a foreign unit, etc.), or the exact binding is missing for a named reason.
257+
- **An unexplained lost site counts against item 4.**
258+
259+
No threshold, control or KILL condition changes.

0 commit comments

Comments
 (0)