Skip to content

Commit 7bae944

Browse files
authored
Merge pull request #172 from PhysShell/claude/ownir-contract-hardening
feat(ownir): OwnIR contract hardening — spec, fail-loud guards, version single-sourcing
2 parents 2ecab43 + fc9b7a2 commit 7bae944

10 files changed

Lines changed: 446 additions & 131 deletions

File tree

‎docs/notes/tech-debt-register.md‎

Lines changed: 51 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -122,21 +122,24 @@ The formalization stack, in order of actual protection delivered:
122122
normalized facts for the pinned samples, and feed the same goldens to
123123
`test_ownir.py` so the Python suite also consumes *extractor-produced*
124124
facts, not only hand-written ones.
125-
3. **`spec/OwnIR.md` + `spec/ownir.schema.json`** (JSON Schema draft 2020-12,
126-
`ownir_version` as a `const`) *(now/short)*. Validate all
127-
`tests/fixtures/ownir/*.json` and the extractor's CI output against it.
128-
Note: the schema must encode the *deliberate* open points — unknown
129-
resource kinds coerce to `subscription` by documented design
130-
(`ownir.py:66-70`), so the resource-kind enum is open, and the schema's job
131-
is shape/type/enum guarantees, not vocabulary closure.
132-
4. **A written evolution policy** *(now, one paragraph in the spec)*: additive
133-
optional fields do not bump `OWNIR_VERSION`; a **new op or changed op
134-
semantics does**. Today's policy only covers the first half, which is
135-
exactly the gap item 1 exploits.
136-
5. Version single-sourcing across the three producers (`ownir.py:159`,
137-
`Program.cs:4360`, `ownts.py:580`) *(with next touch)* — worth doing, but
138-
understand what it does not protect: the dangerous case is *nobody* bumping,
139-
which only items 1–4 catch.
125+
3. **`spec/OwnIR.md`** ✅ (shipped) **+ `spec/ownir.schema.json`** (JSON Schema
126+
draft 2020-12, `ownir_version` as a `const`) — **the schema's trigger has now
127+
fired** (see the box below). Validate all `tests/fixtures/ownir/*.json` and
128+
each frontend's output against it. The resource-kind enum is **closed** for
129+
*present* values — a present-but-unknown kind is rejected at load (it changes
130+
routing; a new kind bumps `OWNIR_VERSION`), while an *absent* `resource` field
131+
still defaults to `subscription`; the schema should mirror that (enum of known
132+
kinds, field optional). Its job is shape/type/enum guarantees.
133+
4. **A written evolution policy** ✅ *(now shipped in `spec/OwnIR.md` §2, rules
134+
IR1–IR6)*: additive optional fields / new resource kinds do not bump
135+
`OWNIR_VERSION`; a **new op or changed op semantics does**.
136+
5. Version single-sourcing ✅ *(enforced)* — a `test_ownir.py` check now greps
137+
each frontend producer (`Program.cs`, `ownts.py`) and asserts its
138+
`ownir_version` literal equals the core's `OWNIR_VERSION`, so "bumped the
139+
core, forgot a frontend" (or the reverse) fails loudly. Understand what it
140+
does **not** protect: the dangerous case is *nobody* bumping, which only
141+
items 1/4 catch. When the P-022 `own-ir` crate lands (a **fourth** producer),
142+
add it to this check — or, better, generate all four from the schema (§below).
140143
6. Reconcile or explicitly document the BCL fresh-factory asymmetry *(now,
141144
small)*: the extractor's `IsOwningFactory` knows the ADO.NET family
142145
(`ExecuteReader`/`CreateCommand`/`BeginTransaction`,
@@ -146,9 +149,28 @@ The formalization stack, in order of actual protection delivered:
146149
only consulted for first-party `call` ops), but the lockstep is
147150
comment-enforced and has already quietly diverged.
148151

149-
Later, if a Rust/C# core materializes: generate types from the same schema
150-
(serde via typify / System.Text.Json source-gen). Protobuf/FlatBuffers only if
151-
fact volume ever makes JSON parse time measurable — no evidence today.
152+
> **Why the schema is now worth building (the argument that was strong all
153+
> along, recovered).** OwnIR is not a nice-to-have file — it is *the one language
154+
> every frontend and the core speak* ("frontends only produce/consume OwnIR",
155+
> ROADMAP design philosophy). **A language without a formal grammar is a language
156+
> that lies about itself.** Until now the schema looked redundant: `ownlang`'s
157+
> hand-written `load()` was the sole validator, so a JSON Schema only *duplicated*
158+
> its shape checks — a second source of truth to drift against, needing a
159+
> `jsonschema` dependency the zero-dep core avoids. **That calculus flips the
160+
> moment there is a second language-native type set** — which has now arrived:
161+
> P-022 #170 ships `crates/own-ir`, hand-writing Rust `serde` structs under the
162+
> rule "accept exactly what Python `load()` accepts," kept in sync only by a
163+
> fixture round-trip. That is two hand-maintained validators of one contract, in
164+
> two languages — the exact drift the OwnIR formalization exists to kill. The
165+
> schema stops being a *shadow* of `load()` and becomes the *source both are
166+
> generated from* (Rust `serde` via `typify`/`schemars`; C#/TS types via
167+
> source-gen), collapsing the hand-synced fact shapes **and** the now-four
168+
> `ownir_version` literals into one authority. So: `spec/OwnIR.md` (done) is the
169+
> prose grammar; `spec/ownir.schema.json` is the machine grammar, and it should be
170+
> **co-designed with the `own-ir` crate** (generate the Rust types from it, don't
171+
> hand-write them twice) rather than bolted on after. JSON stays the wire format;
172+
> Protobuf/FlatBuffers only if measured parse cost ever demands it — no evidence
173+
> today.
152174
153175
## 3. The extractor↔bridge "mirror" (LowerFlowStmt / _lower_flow)
154176

@@ -186,18 +208,22 @@ Python walkers (§2.1).
186208

187209
### Now (standalone value; days-scale; ordered by protection-per-effort)
188210

211+
> **Progress (branch `claude/ownir-contract-hardening`):** N1 ✅, N4 ✅ (spec
212+
> shipped; JSON Schema still pending), N6 ✅ (validation, not a full enum — see
213+
> below), N9 ✅. N8 re-scoped (not surgical — see note). The rest stand.
214+
189215
| # | Item | Where | Why now |
190216
|---|------|-------|---------|
191-
| N1 | `else: raise` on unknown flow ops (+ the five walkers) | `ownlang/ownir.py:1692-1836` | Silent fact-swallowing → fabricated/missed verdicts (§2.1) |
192-
| N2 | Auto-discovery in the test runner | `tests/run_tests.py:1144-1149` | 25-term hand-rolled `or`; a forgotten `rc` silently stops gating |
217+
| N1 ✅ | `else: raise` on unknown flow ops | `ownlang/ownir.py` `_lower_flow` | Silent fact-swallowing → fabricated/missed verdicts (§2.1). *Done: raises `OwnIRError`; pinned by test_ownir. The five read-only walkers legitimately ignore ops they don't consume, so the guard is scoped to `_lower_flow` (the only pass that must handle every op or lose facts).* |
218+
| N2 ✅ | Auto-discovery in the test runner | `tests/run_tests.py` | 25-term hand-rolled `or`; a forgotten `rc` silently stopped gating. *Done: the runner now globs `test_*.py`, calls each module's `run()`, and gates on `any(rc)` — a new test file is auto-included, a module without `run()` fails the suite, and a discovery of zero modules fails loud. `test_codegen_props` keeps its lighter in-suite iteration count via a small special-case map.* |
193219
| N3 | Golden facts snapshots in CI + feed to `test_ownir.py` | `ci.yml` wpf-extractor job | The seam is never diffed at the facts level (§2.2) |
194-
| N4 | `spec/OwnIR.md` + JSON Schema + evolution policy | new; `ownir.py:1-97` is the source | §2.3–2.4 |
220+
| N4 ◑ | `spec/OwnIR.md` ✅ + evolution policy ✅ + version single-sourcing ✅ + JSON Schema (trigger now fired) | `spec/OwnIR.md`, `tests/test_ownir.py` | §2. *Done: normative spec with the version evolution policy (IR1–IR6), registered in `spec/README.md`; and a producer-version-consistency check (each frontend's `ownir_version` must equal the core's). Remaining: `spec/ownir.schema.json` — its trigger has now fired (P-022 #170's `own-ir` crate = a second hand-written type set), so **co-design it with that crate and generate the Rust types from it**, per the §2 box; do NOT bolt a `jsonschema` CI validator onto the zero-dep core first.* |
195221
| N5 | DI001–005 + EFF001 into `spec/` + `Diagnostics.md` + `test_spec.py` | `ownlang/di.py`, `effects.py` | Second-largest analyzer has zero normative governance |
196-
| N6 | Diagnostic `Code` enum (replace bare string literals) | `ownlang/diagnostics.py:243` + emit sites | New codes land weekly; a typo'd code silently renders `""` |
222+
| N6 ✅ | Fail-loud on unknown diagnostic code | `ownlang/diagnostics.py` `Diagnostic.__post_init__` | New codes land weekly; a typo'd code silently rendered `""`. *Done: `Diagnostic` now rejects any code absent from `TITLES` at construction (full suite confirms every emitted code is registered), and `title` indexes instead of `.get(...,"")`. A full `StrEnum` is the heavier follow-up; this closes the silent-`""` hole cheaply.* |
197223
| N7 | Split `ownir.py` → `ownir/{render,load,lower,inference,check}` | `ownlang/ownir.py` (2430 lines) | Fastest-accreting file in the repo; `check_facts` grows a branch per resource kind; deferring = paying the god-file tax on every feature |
198-
| N8 | Dedicated syntax-error code (stop filing `ParseError` as OWN020) | `ownlang/__main__.py:80` | Miscategorized as "unsupported construct" |
199-
| N9 | Bare `assert` on the loan invariant → raise | `ownlang/analysis.py:200-203` | Stripped under `python -O`; silent wrong answer if block-scoping is ever relaxed |
200-
| N10 | Reconcile/document BCL table asymmetry | `Program.cs:2545-2568` vs `ownir.py:1179-1199` | Lockstep already quietly diverged (§2.6) |
224+
| N8 ⚠️ | Dedicated syntax-error code (stop filing `ParseError` as OWN020) | `ownlang/__main__.py:80` | **Re-scoped: not surgical.** OWN020 is the suite-wide code for *any* parse/lex rejection — `for`/`loop`/`async` are lexer-*rejected keywords* (LexError → OWN020 by design), and the corpus/gallery/wpf/rid/loops test helpers all treat OWN020 as "parse rejected". Splitting a genuine syntax error from an unsupported-construct rejection means distinguishing the two at the lexer/parser and updating ~6 test files + `Grammar.md`/`CLI.md`/`OwnCore.md`. A real UX win, but a small refactor, not a one-liner. |
225+
| N9 ✅ | Bare `assert` on the loan invariant → raise | `ownlang/analysis.py` `join()` | Stripped under `python -O`. *Done: mirrors the explicit-raise pattern already used at `_join_handle_rid`, so the invariant holds in every build.* |
226+
| N10 ✅ | Reconcile/document BCL table asymmetry | `Program.cs` `IsOwningFactory` + `ownir.py` `_BCL_FRESH_BY_NS` | Lockstep already quietly diverged (§2.6). *Done: documented, not reconciled — because the two are NOT a mirror. `IsOwningFactory` is the emission authority (a match emits an `acquire`), so it is the SUPERSET incl. ADO.NET + network `Accept*`; the bridge table is consulted only for `call`-op results (`_callee_returns_fresh`), and owning factories are emitted as `acquire` not `call`, so it is a deliberate SUBSET — the C#-only entries never arrive as `call`s. Forcing the lists identical would add dead entries; instead the misleading "kept in lockstep" comments on both sides now state the real roles + the safety invariant (owning factory ⇒ `acquire`) + the generate-from-schema exit.* |
201227

202228
### With next touch
203229

‎frontend/roslyn/OwnSharp.Extractor/Program.cs‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2509,9 +2509,15 @@ void Visit(ExpressionSyntax e)
25092509
// `JsonDocument.ParseAsync` is excluded.
25102510
// Curated + symbol-resolved, so a borrowed/cached disposable handed back by some other API is
25112511
// never mistaken for an owned acquire (precision over recall — the set grows only as
2512-
// ownership is certain). Kept in lockstep with the bridge table `_BCL_FRESH_BY_NS`
2513-
// (ownlang/ownir.py): the extractor decides whether to EMIT the factory call fact, the bridge
2514-
// recognises the callee name as `fresh`.
2512+
// ownership is certain).
2513+
//
2514+
// Relationship to the bridge table `_BCL_FRESH_BY_NS` (ownlang/ownir.py) — NOT a mirror. This
2515+
// function is the emission AUTHORITY: a match makes the extractor emit the local as an `acquire`
2516+
// fact. So this list is the SUPERSET — it also covers ADO.NET and network `Accept*`, which the
2517+
// bridge table omits. The bridge table is consulted only on the OTHER path — a `call` op whose
2518+
// result the bridge decides is `fresh` — and since owning factories are emitted as `acquire` (not
2519+
// `call`), it need only cover the shared File/crypto/Xml/Json seam, never the ADO.NET/network
2520+
// entries. If a factory here is ever emitted as a `call` instead, add it to the bridge table too.
25152521
static bool IsOwningFactory(ExpressionSyntax? e, SemanticModel model)
25162522
{
25172523
if (e is not InvocationExpressionSyntax i

‎ownlang/analysis.py‎

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -197,10 +197,14 @@ def join(a: State, b: State) -> State:
197197
# holds across loop back-edges too: a borrow opened inside a loop body closes
198198
# within the same iteration, so the loan set at the body exit equals the one on
199199
# the entry edge. Assert the invariant rather than paper over a builder bug.
200-
assert set(a.loans) == set(b.loans), (
201-
"active loans differ at a control-flow merge; this should be impossible "
202-
"for block-scoped borrows (they close within the scope that opened them)"
203-
)
200+
# An explicit raise, not `assert`: `python -O` strips asserts, and a silently
201+
# mismatched loan set at a merge would defeat the invariant this locks (same as
202+
# `_join_handle_rid` above). Keep it loud in every build.
203+
if set(a.loans) != set(b.loans):
204+
raise AssertionError(
205+
"active loans differ at a control-flow merge; this should be impossible "
206+
"for block-scoped borrows (they close within the scope that opened them)"
207+
)
204208
out.loans = dict(a.loans)
205209
out.handle_rid = _join_handle_rid(a.handle_rid, b.handle_rid)
206210
out.moved_at = _join_sites(a.moved_at, b.moved_at)

‎ownlang/diagnostics.py‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -239,9 +239,20 @@ class Diagnostic:
239239
# contract (code, message, line, severity, subject, resource_kind) is preserved.
240240
evidence: tuple[Evidence, ...] = ()
241241

242+
def __post_init__(self) -> None:
243+
# A code with no TITLES entry is a bug, not a blank finding: `title` used to
244+
# silently return "" for a typo'd or forgotten code. Fail loudly at
245+
# construction so a mis-typed code can never ship a titleless diagnostic —
246+
# the one stringly-typed contract in a --strict codebase.
247+
if self.code not in TITLES:
248+
raise ValueError(
249+
f"unknown diagnostic code {self.code!r} (not in diagnostics.TITLES); "
250+
f"a code and its title must be added together"
251+
)
252+
242253
@property
243254
def title(self) -> str:
244-
return TITLES.get(self.code, "")
255+
return TITLES[self.code]
245256

246257
def _kind_suffix(self) -> str:
247258
return f" [resource: {self.resource_kind}]" if self.resource_kind else ""

‎ownlang/ownir.py‎

Lines changed: 49 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -66,8 +66,11 @@
6666
advisory OWN050 "leakage analysis skipped" note, never a leak (P-014 Tier A).
6767
6868
An unreleased entry is the core's OWN001 (owned-but-not-released) at the C#
69-
`line`. The `resource`/`type` fields are additive and optional, so they do NOT
70-
bump `ownir_version`: an older core just reads every entry as a subscription.
69+
`line`. The `resource`/`type` FIELDS are additive and optional — an older core
70+
that predates them reads an entry with no `resource` as a subscription. But a new
71+
resource-kind VALUE is a vocabulary change, not additive metadata: the kind
72+
selects the analysis path, so a PRESENT-but-unknown kind is rejected at load
73+
(fail-loud), and introducing one MUST bump `ownir_version` (spec/OwnIR.md §2).
7174
A `capture` entry instead routes through the lifetime/region engine and surfaces
7275
as OWN014 (region escape) when its source provably outlives — see docs/proposals/
7376
P-004 and docs/lifetimes.md (the C# facts now reach the region core, not only the
@@ -214,6 +217,16 @@ def _esc_prop(s: str) -> str:
214217
"pool": ("PooledBuffer", "pooled buffer"),
215218
}
216219

220+
# Every resource-kind discriminator the core routes on (the `_RESOURCES` owned
221+
# kinds, plus `capture` — routed to the lifetime/region engine — and
222+
# `unresolved-subscription` — skipped to an advisory OWN050). The kind CHANGES the
223+
# analysis path (§4 of spec/OwnIR.md), so it is NOT metadata: a PRESENT-but-unknown
224+
# kind is rejected at load (fail-loud, like an unknown flow op), never silently
225+
# routed as a `subscription`. An ABSENT `resource` field still defaults to
226+
# `subscription` (an old extractor that predates the field). Consequence: adding a
227+
# new kind is a vocabulary change that MUST bump OWNIR_VERSION (spec/OwnIR.md §2).
228+
_KNOWN_RESOURCE_KINDS = frozenset(_RESOURCES) | {"capture", "unresolved-subscription"}
229+
217230
# --- P-004 region escape (the `capture` resource kind) ----------------------
218231
# A `capture` is a tokenless strong subscription routed NOT through the
219232
# acquire/release ownership model but through the lifetime/region engine
@@ -472,6 +485,15 @@ def load(path: str) -> dict[str, Any]:
472485
if not isinstance(r, str):
473486
raise OwnIRError(
474487
f"subscription 'resource' must be a string, got {r!r}")
488+
# A present-but-unknown kind changes routing (§4), so reject it rather
489+
# than let it fall through to the owned/subscription path and mis-route
490+
# — the same fail-loud rule as an unknown flow op. A new kind must bump
491+
# OWNIR_VERSION (spec/OwnIR.md §2). An absent field defaulted to
492+
# "subscription" above, which is known.
493+
if r not in _KNOWN_RESOURCE_KINDS:
494+
raise OwnIRError(
495+
f"unknown resource kind {r!r} — a new kind is a vocabulary "
496+
f"change that must bump OWNIR_VERSION (see spec/OwnIR.md §2)")
475497
t = s.get("type")
476498
if t is not None and not isinstance(t, str):
477499
raise OwnIRError(
@@ -1188,8 +1210,20 @@ def _infer_return_skeleton(nodes: Any, param_names: set[str],
11881210
# P1a (stdlib contract pack): more well-known static factories whose result the caller
11891211
# OWNS and must dispose. `XmlReader`/`XmlWriter.Create` return a fresh reader/writer; a
11901212
# `JsonDocument` from `Parse` pools memory and must be disposed (a `var doc =
1191-
# JsonDocument.Parse(json)` that drops `doc` is a common real leak). Kept in lockstep with
1192-
# the extractor's `IsOwningFactory` (frontend/roslyn/.../Program.cs).
1213+
# JsonDocument.Parse(json)` that drops `doc` is a common real leak).
1214+
#
1215+
# Relationship to the extractor's `IsOwningFactory` (frontend/roslyn/.../Program.cs) — NOT a
1216+
# mirror, despite the old "kept in lockstep" wording. The two play different roles. The
1217+
# extractor is the emission AUTHORITY: seeing `var x = Factory()` it emits `x` as an `acquire`
1218+
# fact directly, so its list is the SUPERSET — File, crypto by `Create*`-prefix, Xml/Json, PLUS
1219+
# ADO.NET (`ExecuteReader`/`CreateCommand`/`BeginTransaction`) and network `Accept*`. THIS table
1220+
# is consulted only on the other path: a `call` op whose result the bridge must decide is
1221+
# `fresh` (`_callee_returns_fresh`). Owning BCL factories are emitted as `acquire`, not `call`,
1222+
# so this table is a deliberate SUBSET — the C#-only entries never arrive here as `call`s, so
1223+
# adding them would be dead weight. Safety invariant: an owning BCL factory is emitted as
1224+
# `acquire`. If the extractor ever emits a `call` op for one, add it here too (better: generate
1225+
# both sides from `spec/ownir.schema.json` — see the tech-debt register). Keep the shared
1226+
# File/crypto/Xml/Json entries aligned on that seam.
11931227
"System.Xml": (
11941228
"XmlReader.Create", "XmlWriter.Create",
11951229
),
@@ -1834,6 +1868,17 @@ def _lower_flow(nodes: list[Any], ffile: str, fname: str,
18341868
"ever_released": result in released_vars,
18351869
"pool": False}
18361870
body.append(Let(handle, Acquire("Disposable", [], line), line))
1871+
else:
1872+
# Fail loud on an op this core cannot lower. Silently skipping it would
1873+
# drop the acquire/release facts nested inside it — fabricating a leak (a
1874+
# lost release) or hiding one (a lost acquire) while every existing fixture
1875+
# still passes. A NEW op is a vocabulary change, not an additive optional
1876+
# field, so it must bump OWNIR_VERSION on both the extractor and the core
1877+
# (see spec/OwnIR.md); until then, refuse rather than mis-analyze.
1878+
raise OwnIRError(
1879+
f"unknown OwnIR flow op {op!r} ({ffile}:{line}) — extractor/core "
1880+
f"vocabulary skew; a new op must bump OWNIR_VERSION (see spec/OwnIR.md)"
1881+
)
18371882
return body
18381883

18391884

0 commit comments

Comments
 (0)