Skip to content

Commit 76324fe

Browse files
authored
Merge pull request #335 from PhysShell/claude/retentionpath-doc-remainder
docs(runtime): give the retention witness its section, and take back the comment that outlived its method
2 parents e06e6b9 + 754444d commit 76324fe

2 files changed

Lines changed: 113 additions & 8 deletions

File tree

‎audit/runtime/README.md‎

Lines changed: 109 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,10 @@ audit/runtime/
4444
PropertyChangedStorm/ # C# PropertyChanged-storm profiler — Windows/build-required, NOT CI-gated
4545
PropertyChangedStorm.csproj # net472; Microsoft.Diagnostics.Tracing.TraceEvent
4646
Program.cs # TraceEvent over an .etl: per-property raise frequency, storm findings
47+
RetentionPath/ # C# retention witness — net8.0, CROSS-PLATFORM, build smoked in CI
48+
RetentionPath.csproj # net8.0; Microsoft.Diagnostics.Runtime (ClrMD 3.x) + Newtonsoft.Json
49+
Program.cs # census / roots verbs, exit-code tiers, runtime.json writer
50+
Heap.cs # mark-from-roots, BFS root->object paths with field names
4751
```
4852

4953
## How the leak-harness works (Plan.md §4.1)
@@ -121,6 +125,105 @@ python audit/runtime/ingest.py --propertychanged-storm \
121125
# a located storm clusters with a static INPC0xx in the same file.
122126
```
123127

128+
## Retention paths — is it retained, and by whom (Plan.md §4)
129+
130+
The stack table above has promised *"Heap analysis (retained, duplicates, **retention
131+
paths**) — ClrMD"* since Plan.md §4. `RetentionPath/` is that half. `HeapCounter`
132+
counts instances of named types; this answers the two questions that actually decide
133+
a leak hunt — *is any of it retained at all*, and *who is holding it*.
134+
135+
Unlike its three neighbours it is **net8.0 and cross-platform**, needs no procdump,
136+
and can attach to a live PID. CI builds it and smokes its usage surface on every
137+
push; the end-to-end demo runs on Linux and the WPF pair on Windows.
138+
139+
### `census` — is there anything to hunt?
140+
141+
```console
142+
$ retention-path census --pid 1234 --out runtime.json
143+
144+
roots : 308 objects
145+
on the heap : 4 270 155 objects 573 MB
146+
REACHABLE from roots : 4 144 653 objects 403 MB
147+
uncollected garbage : 125 502 objects 170 MB
148+
>>> 70,4% of the heap is genuinely RETAINED — something holds it; run `roots`
149+
```
150+
151+
**This distinction is not pedantry.** `ClrHeap.EnumerateObjects()` walks the heap
152+
segments linearly and returns *everything allocated, including garbage the GC has
153+
not collected yet*. A big heap is not evidence of a leak. `HeapCounter` mitigates
154+
that by forcing a GC in the target first, which works when you can drive the target;
155+
marking from the roots answers it directly and needs no cooperation. If the retained
156+
share is low, **stop** — there is no reference to hunt, and the next question is
157+
about GC timing, not about who holds what.
158+
159+
### `roots` — what holds the TYPICAL instance
160+
161+
```console
162+
$ retention-path roots --pid 1234 --type GTDGoody
163+
164+
verdict: RETAINED — BrokerDataClasses.GTDGoody: 130 000 on the heap, 129 903 reachable …
165+
166+
#1 25/50 resolved (50,0%) — via [static-event], 7 hops
167+
[PinnedHandle] System.Object[]
168+
BrokerDataClasses.GTD
169+
BrokerDataClasses.CalcProcentGTD (.k__BackingField)
170+
BrokerDataClasses.GTDGoody (.fMainObject)
171+
```
172+
173+
The field names come from ClrMD's `EnumerateReferencesWithFields`; they are what turn
174+
*"this object is alive"* into *"**this field** is holding it"* — the sentence a
175+
developer can act on. A **delegate hop** in the path is what makes it a static
176+
*event* rather than a plain static field, which is the distinction OwnAudit's
177+
`correlate.py` keys its `high` tier on.
178+
179+
**Why it samples, and what the percentages mean.** *"Who holds this object"* is
180+
ill-posed for an object reachable from many roots: there are as many answers as
181+
there are paths, and the shortest is an arbitrary pick rather than an explanation.
182+
So the walk resolves a SAMPLE of paths and ranks them as a histogram — the retainer
183+
accounting for 129 900 of 130 000 is the leak; the three hanging off the stack are
184+
noise. The shares are shares **of the resolved sample**, never of the population.
185+
The verdict is not sampled: reachability and the durable/transient census run over
186+
every instance, so no display budget (`--sample`, `--max-hops`) can change the
187+
diagnosis or the exit code.
188+
189+
### What it does not do — read this before trusting it
190+
191+
- **No dominator tree.** The principled form of "who holds it" is dominance: which
192+
single reference, if cut, makes the object collectable, and how much that frees.
193+
It also answers honestly when two references hold an object jointly, naming the
194+
point where the paths meet instead of picking one. That is what Eclipse MAT and
195+
dotMemory are built on. The A3 witness was extracted from PR #280 **without** it,
196+
deliberately; Own.NET#334 records what the implementation argued. The sampled
197+
histogram is a weaker instrument, and the ranking is how it stays honest about it.
198+
- **A `[stack]` root is not retention** — the object is live in a frame right now,
199+
and is labelled so it is not mistaken for a leak. Same for `[finalizer]`. A
200+
verdict of `RETAINED` requires at least one *durable* retainer.
201+
- **It matches the type, not the type's spelling.** `--type GTDGoody` must not match
202+
`System.Func<…GTDGoody…>` — a cached lambda whose generic *argument* mentions it.
203+
It did, during development, and confidently reported a 2-hop path to the wrong
204+
object.
205+
- **Attaching suspends the target.** On a multi-GB heap the mark pass is minutes,
206+
not seconds — take a dump.
207+
208+
### Exit codes, and why exit 2 exists
209+
210+
| Exit | Meaning |
211+
| --- | --- |
212+
| 0 | The heap was read. `ABSENT` / `OBSERVED_ONLY` — nothing durably retains the type. |
213+
| 1 | The heap was read. `RETAINED` — a durable path exists, and it is printed. |
214+
| 2 | **The heap was not read.** Usage error, unreadable target, refused attach. |
215+
216+
*Not looking* and *looking and finding nothing* are different outcomes, and
217+
collapsing them is how a monitoring pipeline learns to report health it never
218+
measured. Permissions, the Yama `ptrace_scope` cases and the CI rules are in
219+
[`docs/runtime-witness-operations.md`](../../docs/runtime-witness-operations.md).
220+
221+
Output is the `runtime.json` contract (`OwnAudit/docs/runtime-contract.md`), so
222+
`OwnAudit/runtime/correlate.py` consumes it with no adapter — giving the three-way
223+
split its missing input: **confirmed** (a static leak finding whose type also shows
224+
up retained), **static-only** (probable FP), and **runtime-only** — retention with
225+
nothing static to explain it, which is a rule request rather than a report.
226+
124227
## Selftest
125228

126229
`ingest.py` carries embedded-fixture selftests (no harness, no Windows needed) and
@@ -138,8 +241,12 @@ python audit/runtime/ingest.py --selftest
138241
the duplicate detector and the PropertyChanged-storm profiler), the leak-harness
139242
scenario schema + one scenario, runtime rule mappings in the taxonomy (categories
140243
2/3/4/6/11), the C# leak-harness skeleton, the C# duplicate-immutable detector
141-
(strings), and the C# PropertyChanged-storm profiler (ETW).
142-
- **Deferred:** duplicate detection for arbitrary immutable types (field-by-field
244+
(strings), the C# PropertyChanged-storm profiler (ETW), and the **retention
245+
witness** (`RetentionPath/`: census + ranked root paths, `runtime.json`, exit-code
246+
tiers, CI-gated build and demo).
247+
- **Deferred:** the **dominator tree / retained sizes** for the retention witness
248+
(Own.NET#334 — the A3 extraction left it out on purpose; the sampled histogram is
249+
what ships), duplicate detection for arbitrary immutable types (field-by-field
143250
content equality), the diagnostic-build INPC `EventSource` instrumentation in the
144251
target + PerfView/SematixTrace capture wiring, and a scenario corpus for the top-N
145252
screens.

‎audit/runtime/RetentionPath/Heap.cs‎

Lines changed: 4 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,10 @@ namespace OwnNet.Audit.Runtime
3131
/// is how a leak hunt goes wrong.
3232
///
3333
/// The principled version of this is a dominator tree (which single reference, if cut,
34-
/// frees the object — and how much memory that frees). See the README.
34+
/// frees the object — and how much memory that frees). It is NOT implemented here: the
35+
/// A3 witness was extracted from PR #280 without it, deliberately. See
36+
/// `audit/runtime/README.md` § "Retention paths" for what the sampled histogram does and
37+
/// does not answer, and Own.NET#334 for what the dominator implementation argued.
3538
/// </summary>
3639
internal sealed class RetentionWalker : IDisposable
3740
{
@@ -358,11 +361,6 @@ internal static bool IsType(string heapType, string wanted)
358361
return string.Equals(simple, wanted, StringComparison.Ordinal);
359362
}
360363

361-
/// <summary>
362-
/// The dominator tree of the whole live graph, with retained sizes. This is the well-posed
363-
/// version of "who holds it": not a path, but the one reference whose removal frees the object.
364-
/// </summary>
365-
366364
public void Dispose()
367365
{
368366
_runtime.Dispose();

0 commit comments

Comments
 (0)