Skip to content

Commit c0df0a7

Browse files
committed
docs: tech-debt register — buckets, triggers, OwnIR formalization plan
Follow-up to the July 2026 architecture review. Records: formalize OwnIR instead of replacing it (fail-loud unknown ops, golden facts snapshots, spec + JSON Schema, evolution policy); the extractor/bridge mirror is producer/consumer, pin it rather than remove it; rewrite is parked behind explicit triggers with the C#-vs-Rust target decision left open; now / with-next-touch / on-trigger / rejected buckets. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FJSfUpRbRXegrwJNV1rx1X
1 parent 20077b3 commit c0df0a7

1 file changed

Lines changed: 233 additions & 0 deletions

File tree

‎docs/notes/tech-debt-register.md‎

Lines changed: 233 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,233 @@
1+
# Tech-debt register — what to fix, when, and what triggers it
2+
3+
> Status: **living register** (not normative, not a proposal). Derived from the
4+
> July 2026 architecture review
5+
> ([architecture-review-2026-07.md](architecture-review-2026-07.md)) and a
6+
> follow-up investigation of the OwnIR seam, the extractor↔bridge "mirror", and
7+
> the rewrite question. It **links, not duplicates**,
8+
> [consolidation-and-positioning.md](consolidation-and-positioning.md) (which
9+
> already tracks the OwnIR `subscriptions→resources` rename, the
10+
> `Program.cs` split trigger, and the `WPFxxx` catalog rename) and the trigger
11+
> discipline in `AGENTS.execution-surfaces.md` §7–8
12+
> (`trigger = цифры из профилировщика или реальная боль в коде`).
13+
>
14+
> Standing priority (unchanged): **this register is subordinate to shipping the
15+
> alpha** ([alpha-readiness.md](alpha-readiness.md) item A — the
16+
> `ownsharp check MyApp.sln` front door). Every "now" item below is days-scale
17+
> hygiene that must not displace packaging work.
18+
19+
## 0. How to read the buckets
20+
21+
- **Now** — standalone value today; cheap; the cost of *not* doing it is paid on
22+
every feature (the repo's fastest-churning surfaces).
23+
- **With next touch** — fix when the surrounding code is next edited anyway;
24+
not worth a dedicated PR.
25+
- **On trigger** — parked behind an explicit condition, recorded here so the
26+
condition (not aesthetics) pulls it off the shelf.
27+
- **Rejected** — considered and refused, with the reason on record.
28+
29+
A deliberate framing decision: **no item below is justified by "prepares the
30+
rewrite."** A core rewrite is not on the record anywhere (the only mentions are
31+
the hypothetical trigger rows in `AGENTS.execution-surfaces.md:317,327`), and
32+
the repo's own rule says work needs profiler numbers or real pain, not a
33+
speculative future. Every item here pays rent standalone. What a rewrite *would*
34+
need is §1.
35+
36+
## 1. The rewrite question (recorded, not scheduled)
37+
38+
**Current position: no rewrite until a trigger fires.** Candidate triggers, in
39+
the spirit of the existing trigger table:
40+
41+
- measured performance pain at mining scale (profiler numbers on 50+-repo runs,
42+
not vibes);
43+
- Gate A/B of [incremental-computation.md](incremental-computation.md)
44+
(interprocedural whole-program inference, or live in-IDE feedback);
45+
- the imperative rule code becomes unmanageable (>30–50 interdependent rules —
46+
the existing Datalog threshold).
47+
48+
**If a trigger fires, the target language is an open decision, not a default.**
49+
Both candidates must be argued in a P-NNN proposal before any code:
50+
51+
- **C# core.** The strongest *packaging* case: extractor + core become one
52+
self-contained `dotnet tool` / Roslyn analyzer; the two-runtime install
53+
(.NET SDK **and** Python 3.11+) — which is exactly alpha-readiness gap A —
54+
disappears; the in-IDE story unblocks (`docs/howto-visual-studio.md:7-13`
55+
documents that the Python core is *why* there is no Roslyn analyzer today:
56+
an in-process analyzer would be a second checker). Multi-stack neutrality is
57+
**not** an argument against C#: neutrality lives in the OwnIR JSON seam, not
58+
in the core's implementation language (the OwnTS frontend is itself Python
59+
today; CodeQL's engine language is invisible to its users).
60+
- **Rust core.** The case: Ascent/Soufflé (already the named Datalog
61+
candidates) and Salsa (the named IDE-incrementality candidate) are
62+
Rust-native; a single static binary distributes without any runtime; the
63+
domain (borrow checking) has its reference implementations there. The cost:
64+
a third language in the repo, and none of the packaging gaps close (the
65+
extractor stays C#, users still install two things).
66+
67+
**Non-negotiable guardrail either way** (from incremental-computation.md): the
68+
new core is an *optimization, never a new decider* — it must produce
69+
bit-identical verdicts to the Python core over the full corpus, proven by
70+
differential testing, or "one checker" is broken.
71+
72+
**What is actually rewrite-durable** (the oracle a port would be built
73+
against): `spec/` + `tests/test_spec.py` conformance, the two-layer corpus
74+
(`corpus/` + `scripts/benchmark.py` recall/specificity over real C#), the
75+
differential codegen fuzzer, and end-to-end golden runs (sample `.cs` →
76+
expected diagnostics). **Not** the OwnIR fact schema — it is at version 0,
77+
churning weekly, and would be redrawn at any seam re-cut (§3). Sharpening the
78+
durable set is ordinary test hygiene and is already mostly built.
79+
80+
## 2. OwnIR: formalize, do not replace
81+
82+
Investigated and settled: OwnIR is a *data schema for facts at rest*, not a
83+
language. Config/scripting languages (Starlark, Lua, CUE, Dhall) solve a
84+
different problem (executing logic to produce data); MLIR/LLVM are
85+
instruction-level compiler frameworks (massive cost, wrong abstraction, no
86+
fact-interchange precedent); SCIP/LSIF are code-navigation formats; SARIF is
87+
for *results* (already used correctly on the output side); Joern's CPG is the
88+
closest adoptable format but demands a whole-program graph — a frontend-scale
89+
rewrite for no analysis gain. The facts-seam we have is the industry-standard
90+
shape: CodeQL TRAP files, Doop/Soufflé `.facts`, Polonius input relations,
91+
Glean's versioned JSON facts, Infer's Textual `.sil`. Datalog *relations*
92+
become the right vocabulary only if the engine ever moves to a fixpoint/Datalog
93+
core — derivable then as an export from validated OwnIR.
94+
95+
The formalization stack, in order of actual protection delivered:
96+
97+
1. **Fail-loud unknown ops** *(now)*. `_lower_flow`'s `if/elif` chain has no
98+
`else` — an unknown `op` is silently dropped (`ownlang/ownir.py:1692-1836`),
99+
and the five structural walkers that hardcode the `if`/`while` recursion
100+
(`_collect_vars`, `_has_bare_return`, `_call_result_callees`,
101+
`_param_signals`, `_forward_targets`) share the hole. A newer extractor
102+
emitting a new compound op under an unbumped version would silently swallow
103+
nested acquires/releases — fabricated OWN001s and missed leaks while every
104+
hand-written fixture stays green. Add `else: raise OwnIRError(...)`.
105+
This single guard is worth more than any schema file.
106+
2. **Golden facts snapshots in CI** *(now)*. Today the extractor→bridge seam is
107+
pinned only at the rendered-diagnostics level (~174 `grep -q` assertions);
108+
the `facts.json` itself is `cat`'d, never diffed. CI already has the
109+
`jq -S` diff machinery and its own rationale for why diagnostics-level
110+
diffing is insufficient (`ci.yml:1002-1019`) — apply it to the seam: snapshot
111+
normalized facts for the pinned samples, and feed the same goldens to
112+
`test_ownir.py` so the Python suite also consumes *extractor-produced*
113+
facts, not only hand-written ones.
114+
3. **`spec/OwnIR.md` + `spec/ownir.schema.json`** (JSON Schema draft 2020-12,
115+
`ownir_version` as a `const`) *(now/short)*. Validate all
116+
`tests/fixtures/ownir/*.json` and the extractor's CI output against it.
117+
Note: the schema must encode the *deliberate* open points — unknown
118+
resource kinds coerce to `subscription` by documented design
119+
(`ownir.py:66-70`), so the resource-kind enum is open, and the schema's job
120+
is shape/type/enum guarantees, not vocabulary closure.
121+
4. **A written evolution policy** *(now, one paragraph in the spec)*: additive
122+
optional fields do not bump `OWNIR_VERSION`; a **new op or changed op
123+
semantics does**. Today's policy only covers the first half, which is
124+
exactly the gap item 1 exploits.
125+
5. Version single-sourcing across the three producers (`ownir.py:159`,
126+
`Program.cs:4360`, `ownts.py:580`) *(with next touch)* — worth doing, but
127+
understand what it does not protect: the dangerous case is *nobody* bumping,
128+
which only items 1–4 catch.
129+
6. Reconcile or explicitly document the BCL fresh-factory asymmetry *(now,
130+
small)*: the extractor's `IsOwningFactory` knows the ADO.NET family
131+
(`ExecuteReader`/`CreateCommand`/`BeginTransaction`,
132+
`Program.cs:2545-2568`) and crypto `Create*` statics; the bridge's
133+
`_BCL_FRESH_BY_NS` (`ownir.py:1179-1199`) lists neither, despite "kept in
134+
lockstep" comments on both sides. Precision-safe today (the Python table is
135+
only consulted for first-party `call` ops), but the lockstep is
136+
comment-enforced and has already quietly diverged.
137+
138+
Later, if a Rust/C# core materializes: generate types from the same schema
139+
(serde via typify / System.Text.Json source-gen). Protobuf/FlatBuffers only if
140+
fact volume ever makes JSON parse time measurable — no evidence today.
141+
142+
## 3. The extractor↔bridge "mirror" (LowerFlowStmt / _lower_flow)
143+
144+
Settled by code reading: **this is not duplicated logic and does not have to be
145+
removed.** The two sides are producer and consumer of one flow-op vocabulary —
146+
C# lowers syntax→facts (throw-edge injection, `using`/`switch`/`do`
147+
desugaring exist *only* there), Python lowers facts→core AST (handle minting,
148+
MOS inference, hoisting exist *only* there). The genuine double-encodings are
149+
three enumerable spots: the BCL fresh-factory tables (§2.6), the
150+
`kind:"pool"` tag, and the op-vocabulary structure re-walked by the five
151+
Python walkers (§2.1).
152+
153+
- **Now:** pin the contract mechanically instead of by comment — §2.1 + §2.2 +
154+
§2.3 cover it (~1–2 days total).
155+
- **On trigger (P-017 multi-stack becomes real, or a core rewrite fires):**
156+
re-cut the seam so the *core* owns control-flow desugaring (structured
157+
`try`/`using`/`switch` ops with `may_throw` annotations on leaves) and the
158+
~900 lines of contract inference that today live in the bridge
159+
(`solve`, `_infer_return_skeleton`, `_infer_param_effect`, hoisting).
160+
Otherwise every new frontend reimplements the exception-edge machinery per
161+
language. **What must stay frontend-side regardless:** everything that needs
162+
the `SemanticModel` — pool/factory/adopt classification, may-throw
163+
reasoning, escape shapes. "Dumb frontend, all semantics in core" is not
164+
achievable and is not the goal; the goal is *one* implementation of path
165+
enumeration and ownership inference.
166+
- **Before any core rewrite (hard prerequisite):** the bridge's inference
167+
layer is verdict-determining and has **no normative description** — it is
168+
outside `spec/`, outside the schema, and pinned only by examples in
169+
`test_ownir.py`. Either spec it (consume/borrow/fresh/alias/overwrite rules)
170+
or relocate it into `ownlang/` proper first, so "the core" is coextensive
171+
with "what must be ported." Discovering this mid-port is the expensive
172+
surprise this register exists to prevent.
173+
174+
## 4. Register
175+
176+
### Now (standalone value; days-scale; ordered by protection-per-effort)
177+
178+
| # | Item | Where | Why now |
179+
|---|------|-------|---------|
180+
| N1 | `else: raise` on unknown flow ops (+ the five walkers) | `ownlang/ownir.py:1692-1836` | Silent fact-swallowing → fabricated/missed verdicts (§2.1) |
181+
| N2 | Auto-discovery in the test runner | `tests/run_tests.py:1144-1149` | 25-term hand-rolled `or`; a forgotten `rc` silently stops gating |
182+
| N3 | Golden facts snapshots in CI + feed to `test_ownir.py` | `ci.yml` wpf-extractor job | The seam is never diffed at the facts level (§2.2) |
183+
| N4 | `spec/OwnIR.md` + JSON Schema + evolution policy | new; `ownir.py:1-97` is the source | §2.3–2.4 |
184+
| N5 | DI001–005 + EFF001 into `spec/` + `Diagnostics.md` + `test_spec.py` | `ownlang/di.py`, `effects.py` | Second-largest analyzer has zero normative governance |
185+
| N6 | Diagnostic `Code` enum (replace bare string literals) | `ownlang/diagnostics.py:243` + emit sites | New codes land weekly; a typo'd code silently renders `""` |
186+
| N7 | Split `ownir.py` → `ownir/{render,load,lower,inference,check}` | `ownlang/ownir.py` (2430 lines) | Fastest-accreting file in the repo; `check_facts` grows a branch per resource kind; deferring = paying the god-file tax on every feature |
187+
| N8 | Dedicated syntax-error code (stop filing `ParseError` as OWN020) | `ownlang/__main__.py:80` | Miscategorized as "unsupported construct" |
188+
| N9 | Bare `assert` on the loan invariant → raise | `ownlang/analysis.py:200-203` | Stripped under `python -O`; silent wrong answer if block-scoping is ever relaxed |
189+
| N10 | Reconcile/document BCL table asymmetry | `Program.cs:2545-2568` vs `ownir.py:1179-1199` | Lockstep already quietly diverged (§2.6) |
190+
191+
### With next touch
192+
193+
| Item | Do it when |
194+
|------|-----------|
195+
| `di.py`: one parameterized graph walk instead of five copies; `kw_only=True` on `Service` | next DI feature (DI006+ or P-006 open questions) |
196+
| `buffers.branches()` / `codegen._buffer_lowering` single mode→backend table | next buffer/codegen work |
197+
| `emit_*` template placeholder validation; drop the `or 'cond'` literal | next codegen work |
198+
| Version literal single-sourcing across three producers | next OwnIR-schema-touching PR (per consolidation note: fold with the `subscriptions→resources` rename) |
199+
| Move the 900-line wpf-extractor grep job into a Python golden test | next time it breaks on a message rewording (it will) |
200+
| CI: composite action for the 3× duplicated framework-refs block; remove dev-branch push sentinels | next workflow edit |
201+
202+
### On trigger (parked; the trigger pulls it, not aesthetics)
203+
204+
| Item | Trigger | Candidate |
205+
|------|---------|-----------|
206+
| Core rewrite (language decision per §1) | profiler numbers at mining scale, Gate A/B, or the >30–50-rule threshold | C# (packaging case) vs Rust (Ascent/Salsa case) — P-NNN required |
207+
| Seam re-cut: control-flow desugaring + bridge inference → core | P-017 goes real, or the rewrite fires | §3 |
208+
| `cfg.py` three-pass split (resolver / typecheck / lowering); `Symbol` freeze; typed `check_module` | `cfg.py` starts churning again, or the rewrite fires | low churn today; contained blast radius |
209+
| `Program.cs` decomposition by concern | merge pain (already recorded in consolidation-and-positioning.md) | EventSubscriptions / DisposableFields / ArrayPool / DiGraph / FlowLowering / ProjectResolution / Cli |
210+
| Datalog core | existing trigger-table row | Ascent / Soufflé |
211+
| Incrementality | Gate B (IDE) | Salsa |
212+
| Protobuf/FlatBuffers for OwnIR | measured JSON parse cost | typify/schemars codegen path first |
213+
214+
### Rejected (on the record)
215+
216+
- **Replace OwnIR with an existing language/format** — Starlark/CUE/Dhall
217+
(config/scripting, wrong domain), MLIR/LLVM (instruction-level, massive
218+
cost), SCIP/LSIF (code navigation), CPG/Joern (whole-program graph for no
219+
analysis gain). The facts-seam shape is already the industry standard (§2).
220+
- **Lowering fully in the extractor** — the "second checker" the charter
221+
forbids (`ownir.py:5-7`, ROADMAP "One checker").
222+
- **Runtime `released?` flags in codegen** — already rejected in README; if we
223+
don't trust the static result we don't ship it.
224+
225+
## 5. What changed vs the July review's sequencing
226+
227+
The review's recommendation list keyed several items to "during the rewrite."
228+
This register corrects that: with no rewrite on the record, "during the
229+
rewrite" resolves to "never" for the highest-churn surfaces. The `ownir.py`
230+
split (N7) and the `Code` enum (N6) move to **now**; `cfg.py`/`Symbol`/
231+
`check_module` stay parked (genuinely low churn). The OwnIR formalization is
232+
re-scoped from "schema + version single-sourcing" to the protection stack in
233+
§2 (fail-loud ops first, schema third, version constant last).

0 commit comments

Comments
 (0)