Skip to content

Commit d1310fe

Browse files
committed
fix(transient): D-369 — deftype editable/transient dispatch + O-045 fusion double-realization
Smell-audited: 2: the row's own finished form (DA Alt 3, recorded 2026-06-10) landed verbatim on its recall trigger firing; O-045's "invisible to laziness" claim falsified by a side-effecting mapped fn and amended in optimizations.md. The pre-v1.2.0-tag conformance sweep caught flatland.ordered 17→8 and data.generators 20→19 — both present in released 1.1.0 (brew binary repro), NOT ADR-0170 regressions: the 1.1.0-cycle instance?/set? recognition work opened -editable?'s transient path for interface-recognised deftype sets, landing on D-369's never-wired consult side. - -editable? (core.clj) = rt/-instance-of? IEditableCollection — the JVM-exact gate (oracle-verified matrix: sorted/record/list/nil false, declared deftype true) - transient/persistent!/conj!/disj!/assoc!/dissoc!/pop! dispatch a .typed_instance to IEditableCollection/-as-transient + the ITransient* protocol methods (transient.zig dispatchBang) - the inverse direction — a user transient deftype driving NATIVE transients via interop calls — via TRANSIENT_METHOD_MAP (clojure_lang_method.zig): valAt→get, assoc→assoc!, without→dissoc!, conj→conj!, persistent→persistent!, disjoin→disj!, pop→pop!; asTransient→transient joins the persistent-side METHOD_MAP - O-045 fusion gate: chunked-ness decided from the fuse chain's BASE TAG (fuseBaseKind), never by realizing the head — the seqFn probe ran the mapped fn once and the fused transduce re-ran it from the base (observable double side-effect + a dropped draw for stateful fns; the data.generators seeded-*rnd* break) Verified: flatland.ordered 17/17 + data.generators 20/20; full conformance sweep 0 DRIFT across all 17 libs (COVERAGE.md regenerated); corpus seqfns pins single-realization semantics against clj; smoke green.
1 parent c43a6e3 commit d1310fe

7 files changed

Lines changed: 154 additions & 41 deletions

File tree

‎.dev/debt.yaml‎

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -595,18 +595,6 @@ active:
595595
(incremental flush / max-bytes). Tie to D-347 (both are wasm/run resource
596596
bounds). Low priority while wasm/run is demo/trusted-module scoped.
597597
last_reviewed: "2026-06-09"
598-
- id: "D-369"
599-
status: "open (2026-06-10, off-critical-path follow-up to D-286 ADR-0102 am1) — transient DISPATCH wiring for user deftypes. The editable/transient family is RECOGNISED + LOAD-LEVEL (methods register; D-286b makes the persistent clj-named methods dispatch), but cljw's native `conj!`/`assoc!`/`persistent!`/`disj!`/`pop!` (src/lang/primitive/transient.zig) switch over the `.transient_*` Value tags and raise `transient_kind_mismatch` on a `.typed_instance` — they do NOT consult a user deftype's ITransientCollection/-conj! etc. ALSO: cljw's `into` (core.clj) + `-editable?` branch on native-tag predicates (set?/vector?/map?), all false for a typed_instance, so a user IEditableCollection type takes the plain `conj` path and NEVER reaches the transient surface. Finished form (DA Alt 3, F-011 JVM-parity): make `-editable?`/`into` consult IEditableCollection on a typed_instance + wire the transient builtins to the ITransient* protocols, so `(into (ordered-set) xs)` rides the transient fast-path like JVM. Deliberately OFF the ordered critical path (DA finding) — recorded so it is not lost, NOT a cycle-budget defer (it is a genuinely separable concern + its own AD/ADR for the -editable? native-tag divergence). recall trigger: a lib whose public api actually exercises a user transient, OR a perf cycle on user editable types."
600-
category: "polymorphism"
601-
barrier: |-
602-
Native `conj!`/`assoc!`/`persistent!`/`disj!`/`pop!` (transient.zig) +
603-
`into`/`-editable?` (core.clj) do not consult a `.typed_instance`'s
604-
ITransient*/IEditableCollection methods, so a user editable/transient deftype
605-
registers its methods (D-286 am1) but they never dispatch. Off the ordered
606-
critical path (`-editable?` native-tag-only ⇒ transient path unreached). Wire
607-
the typed_instance consult + classify the `-editable?` native-tag divergence
608-
(bug→fix vs AD). Cross-ref D-286 (recognition), ADR-0102 am1 (the DA Alt 3).
609-
last_reviewed: "2026-06-10"
610598
- id: "D-383"
611599
status: "open (2026-06-10, reader asymmetry found during the clojure.instant backfill). A regex literal `#\"…\"` LOADS + works at RUNTIME (`(re-find #\"a+\" \"baaab\")`→\"aaa\") but FAILS in a bootstrap-loaded `.clj` (anything `@embedFile`d from src/lang/clj/**): `regex literal (unsupported syntax in cycle 1) is not supported`. So a bundled namespace cannot use `#\"…\"` — clojure.instant had to guard with `(some #{\\\" \\\\} s)` instead of `(re-find #\"[\\\"\\\\]\" s)`. The bootstrap/cycle-1 reader is a narrower reader than the runtime one; regex-literal support is the gap. Also: the error renderer mis-attributes the location to `<bootstrap>` core.clj (D-058 renderer context gap), so the caret points at the wrong file. LOW priority (clean char-set/ string workaround; few bundled ns need regex), but a real definition-coverage gap (F-013): a bundled ns should reach the full reader."
612600
category: "reader"
@@ -6429,6 +6417,18 @@ discharged:
64296417
category: "clj-parity"
64306418
barrier: "DISCHARGED (was: rt-free constraint on keyEqValue — resolved by structural walk, no rt needed)."
64316419
last_reviewed: "2026-07-09"
6420+
- id: "D-369"
6421+
status: "DISCHARGED 2026-07-14 (the recall trigger FIRED: the 1.1.0-cycle instance?/set? recognition work opened -editable?'s transient path for interface-recognised deftype sets, and the pre-v1.2.0-tag conformance sweep caught flatland.ordered 17→8 broken in the FIELD — present in released 1.1.0 too). Landed the row's own finished form (DA Alt 3): (a) -editable? = rt/-instance-of? IEditableCollection (JVM-exact gate, oracle-verified matrix incl. sorted/record/list false + declared deftype true); (b) transient/persistent!/conj!/disj!/assoc!/dissoc!/pop! dispatch a .typed_instance to IEditableCollection/-as-transient + ITransient*/-conj! etc. (transient.zig dispatchBang); (c) the INVERSE direction — a user transient deftype driving native transients via interop methods — via a TRANSIENT_METHOD_MAP in clojure_lang_method.zig (valAt→get, assoc→assoc!, without→dissoc!, conj→conj!, persistent→persistent!, disjoin→disj!, pop→pop!, assocN→assoc!, nth/count/get pass-through) + asTransient→transient on the persistent side. ALSO fixed the co-surfaced O-045 fusion gate double-realization (reduce head-force + fused re-run = observable double side-effect; data.generators seeded *rnd* broke) — chunked-ness now decided from the fuse chain BASE TAG, never by realizing (fuseBaseKind). flatland.ordered 17/17 + data.generators 20/20 restored; corpus seqfns pins single-realization vs clj. ORIGINAL: open (2026-06-10, off-critical-path follow-up to D-286 ADR-0102 am1) — transient DISPATCH wiring for user deftypes. The editable/transient family is RECOGNISED + LOAD-LEVEL (methods register; D-286b makes the persistent clj-named methods dispatch), but cljw's native `conj!`/`assoc!`/`persistent!`/`disj!`/`pop!` (src/lang/primitive/transient.zig) switch over the `.transient_*` Value tags and raise `transient_kind_mismatch` on a `.typed_instance` — they do NOT consult a user deftype's ITransientCollection/-conj! etc. ALSO: cljw's `into` (core.clj) + `-editable?` branch on native-tag predicates (set?/vector?/map?), all false for a typed_instance, so a user IEditableCollection type takes the plain `conj` path and NEVER reaches the transient surface. Finished form (DA Alt 3, F-011 JVM-parity): make `-editable?`/`into` consult IEditableCollection on a typed_instance + wire the transient builtins to the ITransient* protocols, so `(into (ordered-set) xs)` rides the transient fast-path like JVM. Deliberately OFF the ordered critical path (DA finding) — recorded so it is not lost, NOT a cycle-budget defer (it is a genuinely separable concern + its own AD/ADR for the -editable? native-tag divergence). recall trigger: a lib whose public api actually exercises a user transient, OR a perf cycle on user editable types."
6422+
category: "polymorphism"
6423+
barrier: |-
6424+
Native `conj!`/`assoc!`/`persistent!`/`disj!`/`pop!` (transient.zig) +
6425+
`into`/`-editable?` (core.clj) do not consult a `.typed_instance`'s
6426+
ITransient*/IEditableCollection methods, so a user editable/transient deftype
6427+
registers its methods (D-286 am1) but they never dispatch. Off the ordered
6428+
critical path (`-editable?` native-tag-only ⇒ transient path unreached). Wire
6429+
the typed_instance consult + classify the `-editable?` native-tag divergence
6430+
(bug→fix vs AD). Cross-ref D-286 (recognition), ADR-0102 am1 (the DA Alt 3).
6431+
last_reviewed: "2026-06-10"
64326432
conventions: |-
64336433
- Structure: two top-level lists — `active:` and `discharged:`. Active entries carry id / status / category / barrier (+ optional quality_floor / last_reviewed); discharged entries carry id / discharged_at / resolution. Edit as YAML (no Markdown table to align).
64346434
- New debt: grep `.dev/debt.yaml` for class-overlap before adding (per `.claude/rules/debt_dedup.md`).

‎.dev/optimizations.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -94,7 +94,7 @@
9494
| O-048 | `eval/backend/intrinsic.zig` (`fastGet`) (ADR-0130/ADR-0148; destructure, gc_large_heap) | The `op_get` intrinsic's `fastGet` did `if (map_mod.contains(coll,k)) map_mod.get(coll,k) else nil_val` — **two full scans** of the map per `(get coll k)`. But `map_mod.get` already returns `nil_val` for an absent key (identical to the 2-arg nil default), so the `contains` pre-check was pure redundancy: every lookup scanned twice. destructure (`{:keys [a b c]} m` ×100k = 3 gets/iter) + gc_large_heap (`(get m :val)` ×100k) are get-bound. | Drop the `contains` pre-scan: `.array_map, .hash_map => try map_mod.get(coll, k)`. One scan; behaviour-identical (nil for absent OR present-nil, same as before; `keyEq`/`eqConsult` error behaviour unchanged). | **`37_destructure` ReleaseSafe 55.0→48.3 ms (−12%); `27_gc_large_heap` 33.5→32.5 ms.** Broad win for every 2-arg `(get map k)` on the VM intrinsic path. | `zig build test -Dwasm` ×2 (diff oracle TreeWalk≡VM, identical Values) + corpus + smoke; map hit/miss/present-nil unchanged | O-043, ADR-0130, ADR-0148, D-450 |
9595
| O-047 | `runtime/numeric/big_int.zig` (`wrapManaged` + add/sub/mul/divFloor direct-into-cell) + `runtime/numeric/promote.zig` (`wrapArithCell` + add/sub/mul Managed arms) (ADR-0148; bigint_factorial) | Every BigInt arith result was computed into a TEMP `Managed` then **deep-copied** to the GC cell via `allocFromManaged` (`cloneWithDifferentAllocator` = alloc + limb-memcpy + free the temp). `bigint_factorial` (`(reduce *' …)`, accumulator grows to ~9 limbs) cloned the growing result on EVERY `*` (~100k multiplies). O-039 removed the OPERAND clones; this is the RESULT clone, the symmetric other half. The hot path is `promote.zig`'s Managed arm (BigInt×Long), not `big_int.allocMulManaged` (Long×Long overflow only). | Compute the result DIRECTLY into a fresh `gc.infra` heap `*Managed` (the final cell). `wrapManaged` attaches the GC `BigInt` wrapper with NO clone; `promote.zig`'s `wrapArithCell` consumes the cell — inline-Long collapse frees it, a heap wrap MOVES it (struct-copy the limbs slice, no memcpy). Ownership: two errdefers (`destroy` the Managed alloc + `deinit` its limbs) cover `gc.alloc` failure until the wrap stores the pointer; the collapse path frees explicitly on the success return (errdefers don't fire). | **`32_bigint_factorial` ReleaseSafe 21.3→19.0 ms (A/B 20 runs); cross-lang cljw 20.2 ms now FASTEST-script (python 20.4, babashka 20.7) — the 1.31×-behind target CLOSED.** Broad win for ALL BigInt `+ - * /` (every result previously cloned). | `zig build test -Dwasm` ×2 (diff oracle TreeWalk≡VM, identical Values) + lint + smoke (corpus 3181 + leak-detecting units exercise the move/collapse paths); output unchanged (158 = digit count of 100!) | O-039, O-037, ADR-0148, D-450 |
9696
| O-046 | `runtime/numeric/ratio.zig` (canonical two-tier Ratio) + `promote.zig` (i64 fast paths in `ratioArith`/`divPromoting` + `partsOf`/coerce/sign/toI64/bigdec arms) + `equal.zig` (`ratioKeyEq` + `.ratio` hash) + `print.zig` + `math.zig`/`json.zig`/`ratio_methods.zig`/`analyzer.zig` (parts() branches) (ADR-0149; ratio_sum) | `ratio_sum` `(reduce + (map #(/ 1 %) (range 1 51)))` ×1000 was ALLOC-bound: each tiny ratio (`1/x`, x≤50) paid ~6 heap allocs (2 coerce + arena + 2 result BigInt + Ratio) through `std.math.big`. The lone far target (2.34×). | Small-ratio inline-i64 representation (ADR-0149 Alt 2): a Ratio stores numer/denom as inline i64 when both fit (no BigInt), auto-promoting to the BigInt tier on i64 overflow. **CANONICAL** (small iff reduced fits i64) so the rt-free `equal`/`hash` arms stay correct by construction (small-vs-big never equal; `hashLong` gives cross-rep hash parity). `divPromoting` int/int + `ratioArith` small⊗small use overflow-guarded i64 cross-multiply + i64 gcd (MIN_I64 → Managed fallback), allocating only the small Ratio struct. | **`33_ratio_sum` ReleaseSafe 81→31.6 ms (2.34×→0.90× — now FASTER than Babashka 35 ms; the LONE far target CLOSED)**; div-only 33→16 ms. Broad win for all small-rational arithmetic. | `zig build test -Dwasm` ×2 (diff oracle + ratio unit tests incl. canonical small/big/collapse + leak detector) + corpus 3157; verified vs clj: basic/collapse(1N/2N/2)/big/numerator/denominator/double/sort/bigdec + **canonical: `(= 1/2 (/ 1e11 2e11))`→true, hash-eq→true, map-key→:half, set→true** + harmonic sum exact | O-037, O-038, O-039, ADR-0149, ADR-0148, D-450 |
97-
| O-045 | `lang/primitive/higher_order.zig` (`reduceFn` fusion gate) (ADR-0148; gc_large_heap) | The O-023 fused-reduce path fired for ANY 3-arg `(reduce f init (map/filter … src))` with a fuse descriptor. Measurement showed it is a **2.5× REGRESSION for a CHUNKED source** (range/vector): the `.clj`-transducer-per-element fused path is slower than the generic walk's in-Zig chunk drain (`-chunk-map-step`, O-032). `(into [] (map f (range 100k)))` = 39 ms fused vs 16 ms walked. gc_large_heap's `(into [] (map (fn[i]{…}) (range)))` paid this. | Gate the fusion: force `(seq coll)` once (memoised; the generic walk re-seqs the same node) and fire `-fused-reduce` ONLY when the realized head is **non-chunked** (a cons-list chain — O-023's actual win, map_filter_reduce). A chunked head falls through to the generic walk's O-004/O-032 chunk path. | **`27_gc_large_heap` ReleaseSafe 59→34.9 ms (1.99×→1.18× vs Babashka 29.7 ms)** — the 2nd-worst target CLOSED to near-parity. `map_filter_reduce` (cons-list) UNREGRESSED (still fuses, 11.7 ms). Broad win for `into`/`reduce` over `map`/`filter` of range/vector. | `zig build test -Dwasm` ×2 (diff oracle) + corpus 3157 (covers map/filter/reduce/into; fixture lacks core so no diff_test arm) + manual: into+map / cons-list map+filter / map+filter+set all == clj | O-023, O-032, ADR-0148, D-450 |
97+
| O-045 | `lang/primitive/higher_order.zig` (`reduceFn` fusion gate) (ADR-0148; gc_large_heap) | The O-023 fused-reduce path fired for ANY 3-arg `(reduce f init (map/filter … src))` with a fuse descriptor. Measurement showed it is a **2.5× REGRESSION for a CHUNKED source** (range/vector): the `.clj`-transducer-per-element fused path is slower than the generic walk's in-Zig chunk drain (`-chunk-map-step`, O-032). `(into [] (map f (range 100k)))` = 39 ms fused vs 16 ms walked. gc_large_heap's `(into [] (map (fn[i]{…}) (range)))` paid this. | Gate the fusion on the fuse chain's **BASE TAG** (`fuseBaseKind`, amendment 2026-07-14): `.list`/`.cons`/`.nil` base → fuse (O-023's win); `.range`/`.vector`/chunked → generic walk; opaque lazy base → generic walk. The original gate FORCED `(seq coll)` to test chunked-ness — the memoised head realization ran the mapped fn once and the fused transduce re-ran it from the base: an observable double side-effect + one dropped draw for stateful fns (broke data.generators' seeded `*rnd*`, found by the 2026-07-13 pre-tag conformance sweep). Never realize to decide. | **`27_gc_large_heap` ReleaseSafe 59→34.9 ms (1.99×→1.18× vs Babashka 29.7 ms)** — the 2nd-worst target CLOSED to near-parity. `map_filter_reduce` (cons-list) UNREGRESSED (still fuses, 11.7 ms). Broad win for `into`/`reduce` over `map`/`filter` of range/vector. | `zig build test -Dwasm` ×2 (diff oracle) + corpus 3157 (covers map/filter/reduce/into; fixture lacks core so no diff_test arm) + manual: into+map / cons-list map+filter / map+filter+set all == clj | O-023, O-032, ADR-0148, D-450 |
9898
| O-044 | `eval/backend/vm/opcode.zig` (op_nth2) + `intrinsic.zig` (`fastNth2`) + `vm.zig` dispatch + `vm/compiler.zig` 2-arg emit (ADR-0130 extended; gc_alloc_rate) | O-043 intrinsified only 3-arg `nth`; the 2-arg `(nth v i)` (no default — RAISES on OOB) still compiled to a generic `op_call`. gc_alloc_rate's `(+ sum (nth v 2))` ×200K uses 2-arg nth. | `op_nth2` (reuses the cached `nth` Var + `core_coll_pristine`): `fastNth2` inlines an in-range vector index; every error case (OOB / negative / non-int / non-vector / nil) defers to the cached `nth` builtin so the raise is identical. Compiler emits op_nth2 for a recognised 2-arg nth. | **`26_gc_alloc_rate` ReleaseSafe 45.8→40.5 ms (1.30×→1.15× vs Babashka 35.3 ms); cumulative 108.4→40.5 (2.81×→1.15×).** | `zig build test -Dwasm` ×2 + corpus + smoke; verified in-range (10/30), list-defer (6), OOB raises (:oob), negative raises (:neg) == clj | O-043, ADR-0130, ADR-0148, D-450 |
9999
| O-043 | `eval/backend/vm/opcode.zig` (op_get/op_nth) + `intrinsic.zig` (CollOp/recognizeColl/fastGet/fastNth3) + `vm.zig` dispatch + `vm/compiler.zig` emit + `runtime.zig` (coll_vars/core_coll_pristine) + `bootstrap.zig` cache + `core.zig` deopt (ADR-0130 extended; destructure, gc_large_heap) | `(get coll k)` (2-arg) + `(nth coll i default)` (3-arg) compiled to a generic `op_call`: `op_get_var` callee push + var-resolution + BuiltinFn dispatch per call. destructure does 3 get + 3 nth/iter ×100K (0% malloc — pure dispatch); gc_large_heap does `(get m :val)` ×100K. | Collection-accessor intrinsic opcodes mirroring the ADR-0130 arith family: the compiler recognises the canonical `get`/`nth` Var (pointer identity) and emits op_get/op_nth (no callee push). The VM arm runs `fastGet` (map/nil inline = getFn 2-arg) / `fastNth3` (vector inline = nthFn 3-arg vector), deferring every other collection kind to the cached Var; `core_coll_pristine` (cleared by `alter-var-root` on get/nth) deopts to the builtin so a redefinition is honoured. Allocation-free → no GC op_top sync. | **`37_destructure` ReleaseSafe 73.7→53.6 ms (1.68×→1.22× vs Babashka 43.9 ms); `27_gc_large_heap` 63.0→59.0 ms (2.12×→1.99×, get-bound portion; closures still dominate).** Broad win for all `(get map k)` / `(nth vec i d)`. | `zig build test -Dwasm` ×2 (diff oracle + new `op_get / op_nth` diff test: map hit/miss, nil, vector, set, OOB/negative default, list defer, destructure=66, string/nested checkEqual) + corpus; deopt verified (`alter-var-root #'get` → redefinition honoured); all values == clj | O-014, O-018, ADR-0130, ADR-0148, D-450 |
100100
| O-042 | `lang/primitive/core.zig` (`strFn` single-int fast path) (ADR-0148; string_ops) | `(str i)` always built a heap `std.Io.Writer.Allocating` (gpa buffer alloc + free) + `writeStrValue` + the final `string.alloc` — even for a single small integer. string_ops does `(str i)` ×100K. | Single immediate-integer arg (`args.len == 1 and args[0].isInt()`): `std.fmt.bufPrint("{d}")` into a `[24]u8` stack buffer → `string.alloc` directly, skipping the Allocating-writer alloc+free. Heap-Long / BigInt (tag `.big_int`, not `isInt()`) keep the full path → value-exact. Multi-arg / non-int unchanged. | **`string_ops` ReleaseSafe 28.6→23.0 ms (1.35×→1.08× vs Babashka 21.3 ms).** | `zig build test -Dwasm` ×2 + corpus + smoke; spot-checked `(str 0/-5/123)` + bigint 10^24 (full value) + `(str 1 2 3)`→"123" + `(str "x" 5)`→"x5" | O-029, ADR-0148, D-450 |

0 commit comments

Comments
 (0)