|
| 1 | +# Leaf / DMIR — assessment against what OwnAudit actually has |
| 2 | + |
| 3 | +Working note. **Trigger:** an outside reading of *"Leaf: An Instrumentation-based |
| 4 | +Dynamic Analysis Framework for Rust"* |
| 5 | +([arXiv:2607.15025](https://arxiv.org/abs/2607.15025), Omidvar Tehrani, Gaboardi, |
| 6 | +Sumner, Ko) proposed it as "a concrete building block for OwnAudit" at 8.5/10, |
| 7 | +with a separate Rust-analysis direction at 9/10, and listed five ideas to adopt. |
| 8 | + |
| 9 | +The paper is real and summarised accurately: Leaf instruments **Rust MIR** via |
| 10 | +`rustc` internals and delivers **DMIR**, an event-driven stream carrying static |
| 11 | +MIR identity plus runtime facts, with a concolic executor / sanitizer / tracer |
| 12 | +built on top. The architecture is genuinely good. |
| 13 | + |
| 14 | +The proposal does not survive contact with the repositories. Recorded here so |
| 15 | +nobody re-derives it — same purpose as |
| 16 | +[P-034](../proposals/P-034-runtime-lifetime-guard.md)'s "do not re-derive" table. |
| 17 | + |
| 18 | +## The blocking fact: wrong runtime |
| 19 | + |
| 20 | +**Leaf instruments Rust MIR. OwnAudit audits .NET.** |
| 21 | + |
| 22 | +Verified against `PhysShell/OwnAudit` at clone time: |
| 23 | + |
| 24 | +- README: an orchestrator over "a legacy **.NET 4.7.2 / WPF / DevExpress** app". |
| 25 | +- File census: 98 `.sarif`, 70 `.py`, 46 `.json`, 38 `.md`, 31 `.cs` — and |
| 26 | + **zero `.rs`, no `Cargo.toml`**. |
| 27 | +- The runtime half is a **ClrMD heap walk** over a CLR process, emitting |
| 28 | + `runtime.json` (`schema: "ownAudit/runtime/v1"`). |
| 29 | + |
| 30 | +There is no MIR in .NET and Leaf cannot observe a CLR process. The proposed |
| 31 | +`Leaf → adapter → RuntimeEvidenceEvent → OwnAudit rules` pipeline is not |
| 32 | +buildable at any effort level. The 8.5/10 is for a wire that has no endpoints. |
| 33 | + |
| 34 | +**The 9/10 is further off still.** [P-017](../proposals/P-017-multi-stack-frontends.md) |
| 35 | +takes the core beyond .NET to **OwnTS** and **OwnJVM (Java/Kotlin)** — Rust is |
| 36 | +not a target and structurally should not be: this project's entire thesis is |
| 37 | +bringing ownership discipline to a **GC language that lacks it**, and the |
| 38 | +ROADMAP carries a standing section on *why not a Rust-style borrow checker for |
| 39 | +C#*. Rust already has the borrow checker; there is no gap to sell into. |
| 40 | + |
| 41 | +The likely source of the error is real and easy to trip on: **Own.NET's core is |
| 42 | +being migrated *to* Rust** ([P-022](../proposals/P-022-rust-core-migration.md)). |
| 43 | +That is the analyzer *written in* Rust, not the analyzer *analyzing* Rust. |
| 44 | + |
| 45 | +## The five "adopt these", scored against the tree |
| 46 | + |
| 47 | +| # | Proposed idea | Reality | |
| 48 | +|---|---|---| |
| 49 | +| 1 | Provider-agnostic Runtime Evidence IR, so we don't become a shell over Leaf | **Seam already exists.** `runtime.json` is a versioned schema consumed by `runtime/cli.py`; the collector is already swappable. What differs is snapshot vs event stream — a real question, but not this one | |
| 50 | +| 2 | Correlate static site with runtime fact | **Already shipped.** Member-aware matching (2026-07-19): a `subscription-leak` finding's canonical `event 'A.B.Holder.Member'` identity must equal a retention root's `(short(holder), member)`; the matched root is reported as `root_holder`/`root_member`. Evidence in `docs/evidence/gtd-runtime-transition.md` | |
| 51 | +| 3 | Static pass drives targeted instrumentation | **Not built — and the motivation does not transfer.** Leaf needs targeting because per-event MIR instrumentation is reportedly 19×–396×. A post-scenario heap snapshot has no per-event cost to target down. Only becomes relevant *if* OwnAudit moves to event-stream instrumentation, which is a decision on its own merits | |
| 52 | +| 4 | Explicit evidence gaps: `COMPLETE / PARTIAL / OPAQUE / UNSUPPORTED` | **Already a machine contract, and stronger.** [`runtime-witness-operations.md`](../runtime-witness-operations.md): exit 0 = heap read, nothing retained; exit 1 = read, retained; **exit 2 = the heap was not read**. A refused attach writes **no** `runtime.json` — there is no verdict to record. *"Proven by CI: a denied attach exits 2, names the policy that refused, and leaves no artifact behind."* Plus `NO-TOOL: skipped` coverage maps, the OWN050/051/052 advisories, and P-036's rule that a check "may not pretend the call was proven harmless" | |
| 53 | +| 5 | Use it as an oracle for static rules | **Pattern already shipped**, minus Leaf. The 3-way oracle (Own.NET vs Infer# vs CodeQL) plus the runtime buckets: **confirmed** (gate on these), **static-only** (suspect FP or unexercised path), **runtime-only** = *"the analyzer's blind spot — candidate for a new rule"* | |
| 54 | + |
| 55 | +So: two already shipped, one shipped as a pattern, one already has its seam, and |
| 56 | +one solves a cost problem this stack does not have. |
| 57 | + |
| 58 | +The sharpest inversion is #4. The proposal offered explicit evidence gaps as |
| 59 | +something to import — *"сделанная частью машинного контракта, а не красивой |
| 60 | +фразой в README"*. It is already the machine contract, it is CI-proven, and |
| 61 | +distinguishing *not looking* from *looking and finding nothing* is one of the |
| 62 | +oldest rules in this codebase. |
| 63 | + |
| 64 | +## What is actually worth taking |
| 65 | + |
| 66 | +Two things, both concept-level, neither depending on Leaf. |
| 67 | + |
| 68 | +**(a) One runtime substrate, many analyzers.** Leaf's real contribution is |
| 69 | +decoupling: instrumentation exists separately from any particular analysis, so |
| 70 | +one semantic layer (50 DMIR callbacks) serves a concolic executor, a sanitizer |
| 71 | +and a tracer. The boundary transfers; the MIR does not. |
| 72 | + |
| 73 | +**(b) Separate the *collection* substrate from the *analysis-facing* |
| 74 | +representation.** Subtler than "one shared schema", and the part worth stealing |
| 75 | +outright. Leaf does not hand probes to analyzers: probes are shaped for cheap |
| 76 | +instrumentation, and a Probe-to-DMIR adapter reconstructs structured events for |
| 77 | +analysis. The two formats are deliberately different. The failure this avoids is |
| 78 | +one universal envelope that everything gets poured into *because the schema |
| 79 | +already exists* — a snapshot and an event stream should not be forced to |
| 80 | +pretend they are one datatype. |
| 81 | + |
| 82 | +### Measured against the tree, the question is already overdue |
| 83 | + |
| 84 | +The natural framing — "decide before the second dynamic consumer" — is **too |
| 85 | +late by two consumers**. There are **five runtime-evidence producer |
| 86 | +implementations across the two repositories**: four under |
| 87 | +`Own.NET/audit/runtime/`, plus the lift-out producer in |
| 88 | +`OwnAudit/src/OwnAudit.Runtime`. |
| 89 | + |
| 90 | +"Collector" is the wrong word for them, and the distinction is load-bearing |
| 91 | +rather than pedantic — it decides which layer a new contract belongs to. |
| 92 | +Verified by the APIs each one actually calls: |
| 93 | + |
| 94 | +| runtime evidence producer | acquisition it performs | material it interprets | output | schema field | |
| 95 | +|---|---|---|---|---| |
| 96 | +| `RetentionPath` | `AttachToProcess`, `procdump` | live CLR **or** `.dmp` | `runtime.json` | `"own-runtime/1"` | |
| 97 | +| `OwnAudit.Runtime` (`RuntimeReport.cs`) | stand-side collector | live heap | `runtime.json` | `"ownAudit/runtime/v1"` | |
| 98 | +| `LeakHarness` | `procdump` (orchestrates the scenario) | `.dmp` | `leak-harness.json` | none | |
| 99 | +| `DuplicateDetector` | `procdump` (optional) | `.dmp` | `duplicate-detector.json` | none | |
| 100 | +| `PropertyChangedStorm` | **none** | `.etl` captured externally | `propertychanged-storm.json` | none | |
| 101 | + |
| 102 | +`PropertyChangedStorm` is the clean case: it opens an `ETWTraceEventSource` over |
| 103 | +an `.etl` that PerfView/xperf/logman captured against the target's |
| 104 | +`OwnNet-Sematix-INPC` EventSource. It acquires nothing. So the real layering is |
| 105 | +**acquisition** (live attach / `.dmp` / `.etl`) → **interpretation** → |
| 106 | +**analysis-facing artifact** → `correlate.py` / `ingest.py` → SARIF. |
| 107 | + |
| 108 | +That reframes takeaway (b) more sharply than "one substrate, many analyzers": |
| 109 | +*the acquisition representation need not — and should not — be the evidence |
| 110 | +representation.* A `.dmp` and an `.etl` are already good acquisition substrates |
| 111 | +for their jobs; turning them into a universal runtime-event JSON would unify the |
| 112 | +wrong layer. The thing worth sharing is a **runtime artifact contract**, not a |
| 113 | +collection substrate. |
| 114 | + |
| 115 | +So [`Plan.md`](../../Plan.md) categories 6 (PropertyChanged storms) and 11 |
| 116 | +(duplicate immutable heap data, flagged there as *"the project's gold"*) are not |
| 117 | +future work to design around — they exist as code, each with its own bespoke |
| 118 | +JSON. Three findings follow, all checkable: |
| 119 | + |
| 120 | +1. **Two producers of `runtime.json` emit two different schema strings.** |
| 121 | + `RetentionPath` (both of its writers) says `own-runtime/1`; the newer |
| 122 | + stand-side collector, `docs/runtime-contract.md`, and every fixture say |
| 123 | + `ownAudit/runtime/v1`. This is a lift-out artifact — the collector plan |
| 124 | + (2026-07-18) ports `stackpeek` into `OwnAudit.Runtime` while the older |
| 125 | + `RetentionPath` stays canonical in Own.NET — not sloppiness. But it is live. |
| 126 | +2. **Nothing enforces either.** `runtime/correlate.py` consumes exactly |
| 127 | + `retained / type / count / expected / bytes / roots / kind / holder / |
| 128 | + member`. It never reads `schema`, `scenario`, `iterations`, `collector`, or |
| 129 | + `verdict`. The only assertion on the string is |
| 130 | + `HeapCollectorContractTests.cs:83`, and it checks the producer against |
| 131 | + itself. A `runtime.json` with the wrong version — or none — correlates |
| 132 | + silently. |
| 133 | +3. **The envelope fields are decorative.** `scenario` and `iterations` are |
| 134 | + documented and present in fixtures but unread; `collector` (a genuine |
| 135 | + provenance fingerprint) and `verdict` are emitted by `RetentionPath` but |
| 136 | + appear in neither the documented schema nor the consumer. |
| 137 | + |
| 138 | +**No impact today** — the correlator is deliberately tolerant of unknown fields, |
| 139 | +which is a reasonable forward-compat choice. The risk is latent and the fix is |
| 140 | +cheap *now*, which is exactly the argument for settling it before the runtime |
| 141 | +artifact surface expands further. |
| 142 | + |
| 143 | +### Counted by artifact family, which is what would get versioned |
| 144 | + |
| 145 | +Producers are the wrong unit — five implementations emit **four** analysis-facing |
| 146 | +artifact families, and it is the families that need identity: |
| 147 | + |
| 148 | +| artifact family | producer(s) | identity status | |
| 149 | +|---|---|---| |
| 150 | +| heap-retention | `RetentionPath` + `OwnAudit.Runtime` | **two conflicting identities** (`own-runtime/1` vs `ownAudit/runtime/v1`) | |
| 151 | +| leak-growth | `LeakHarness` | **none** | |
| 152 | +| duplicate-immutable | `DuplicateDetector` | **none** | |
| 153 | +| propertychanged-storm | `PropertyChangedStorm` | **none** | |
| 154 | + |
| 155 | +> **All four shipped runtime artifact families have an identity problem: |
| 156 | +> heap-retention has two conflicting schema identities; the other three have |
| 157 | +> none.** |
| 158 | +
|
| 159 | +### The cheap next step is versioning, not unification |
| 160 | + |
| 161 | +A shared envelope carrying capture/process/scenario identity, producer |
| 162 | +fingerprint, config digest and provenance, with typed-distinct payloads, is the |
| 163 | +right *destination*. It is the wrong *first move*: it designs commonality up |
| 164 | +front, which is precisely the mistake Leaf avoids. Leaf's lesson is **define the |
| 165 | +boundary first, derive the representation second**. |
| 166 | + |
| 167 | +So the minimal work that closes the real hole found above, in order: |
| 168 | + |
| 169 | +1. **Every analysis-facing artifact gets its own mandatory schema identity** — |
| 170 | + one per family, e.g. `heap-retention`, `leak-growth`, |
| 171 | + `duplicate-immutable`, `propertychanged-storm` (names illustrative, not |
| 172 | + normative). Today three of the five producers emit no version at all, and |
| 173 | + the two that do disagree with each other — see the family table above. |
| 174 | +2. **Readers must validate family and major version.** Tolerant of unknown |
| 175 | + *fields*, intolerant of unknown *identity*: |
| 176 | + |
| 177 | + | input | verdict | |
| 178 | + |---|---| |
| 179 | + | known family + known major + unknown extra fields | accept | |
| 180 | + | missing schema | reject | |
| 181 | + | wrong artifact family | reject | |
| 182 | + | unknown major | reject | |
| 183 | + |
| 184 | + This is the actual gap. `correlate()` currently takes a `dict` and |
| 185 | + structurally duck-types it on `retained`/`type`/`count`/… — so *any* JSON |
| 186 | + containing a `retained` array is treated as the contract. |
| 187 | +3. **Only then extract the envelope**, from commonality *proven* by four |
| 188 | + shipped schemas rather than drawn in advance. Fields have to earn the right |
| 189 | + to be called common. |
| 190 | + |
| 191 | +This closes the found defect now, changes no fail-closed semantics, and commits |
| 192 | +to no universal-envelope architecture. |
| 193 | + |
| 194 | +### Split `completeness` into three orthogonal things |
| 195 | + |
| 196 | +The caution raised earlier resolves cleanly once the word is broken up. These |
| 197 | +are three different states and are currently drifting toward one label: |
| 198 | + |
| 199 | +- **Collection outcome.** Success → an artifact may exist. Refusal or failure → |
| 200 | + exit 2 and the artifact **must not** exist. *Unchanged where it already |
| 201 | + holds* — and it does **not** hold everywhere: `RetentionPath` has the |
| 202 | + documented contract, `DuplicateDetector` and `PropertyChangedStorm` catch |
| 203 | + broadly and return 2, but `LeakHarness` catches only `ScenarioException`, so |
| 204 | + a missing scenario file or a failed dump escapes `Main` with a different exit |
| 205 | + code. Normalizing that is a behaviour change, not a clean-up; see the task |
| 206 | + spec's caveat. |
| 207 | +- **Artifact validity.** Given an artifact: known schema, required fields, |
| 208 | + well-formed, producer contract satisfied → valid, else reject. *This is the |
| 209 | + layer that is missing today* (step 2 above). |
| 210 | +- **Evidence coverage.** Belongs to the **artifact family**, not to a global |
| 211 | + enum. For heap retention the family contract can simply be |
| 212 | + total-or-no-artifact — no `PARTIAL` ever. For an ETW trace the physics differ |
| 213 | + and coverage is real data: capture interval, events lost, buffers lost, trace |
| 214 | + truncated, provider-enabled interval. A valid storm artifact can legitimately |
| 215 | + rest on a bounded trace if the claim it makes is bounded accordingly. |
| 216 | + |
| 217 | +That keeps "refusal → no artifact" intact, because *collector refused* and |
| 218 | +*collector succeeded over bounded observation* are different states, not two |
| 219 | +shades of one. The invariant to write down: |
| 220 | + |
| 221 | +> **Artifact existence proves successful execution of its producer contract, |
| 222 | +> not completeness of every possible observation. Coverage semantics belong to |
| 223 | +> the artifact family and must never be inferred from artifact existence |
| 224 | +> alone.** |
| 225 | +
|
| 226 | +This is strictly better than a global `"completeness": "PARTIAL"`, which within |
| 227 | +a year means five different things per producer and then grows a |
| 228 | +`PARTIAL_BUT_USABLE`. |
| 229 | + |
| 230 | +## The invariant worth stating separately |
| 231 | + |
| 232 | +The error that produced the original proposal is natural, will recur, and is |
| 233 | +cheap to inoculate against: |
| 234 | + |
| 235 | +> **Do not infer analyzed-language support from the implementation language of |
| 236 | +> the Own.NET core.** P-022 moves the *analyzer's implementation* to Rust. It |
| 237 | +> does not change the audited runtime, does not introduce Rust MIR, and does not |
| 238 | +> make Rust an analysis target — P-017's targets are OwnTS and OwnJVM. |
| 239 | +
|
| 240 | +Implementation language and analyzed language are orthogonal axes. Conflating |
| 241 | +them is what turned a Rust-only instrumentation framework into an 8.5/10 |
| 242 | +recommendation for a .NET auditor. |
| 243 | + |
| 244 | +## Provenance and limits |
| 245 | + |
| 246 | +- The paper's existence, title, authors and subject were verified against arXiv. |
| 247 | +- The **overhead figures** (19×–396×, ~3.5× for concrete types, ~2.8× for raw |
| 248 | + addresses, ~50 DMIR callback types, the `nightly-2026-07-01` pin, and the |
| 249 | + three demo analyses) were **not** checked here against the paper. The author |
| 250 | + of the original summary re-verified them against it afterwards and reports |
| 251 | + them correct; that is second-hand for this note's purposes, and they are |
| 252 | + quoted only to show that the motivation for idea #3 is cost, not to rely on |
| 253 | + the magnitudes. |
| 254 | +- OwnAudit was read at a shallow clone of `main`; the collector inventory and |
| 255 | + schema-string findings are about that tree plus `Own.NET/audit/runtime/` at |
| 256 | + this branch's base commit. |
| 257 | +- **Recorded, not scheduled** — with one exception. The identity/validation |
| 258 | + defect measured above is no longer an idea from a paper but a demonstrated |
| 259 | + contract hole, so it has a written spec at |
| 260 | + [`docs/tasks/runtime-artifact-identity.md`](../tasks/runtime-artifact-identity.md) |
| 261 | + (status: *spec, ready to implement*, explicitly **not** inserted into the |
| 262 | + current sequence). Everything else here remains recorded only. |
0 commit comments