Skip to content

Commit 1bcca2a

Browse files
authored
Merge pull request #60 from BuddhiLW/land/round7
Round 7: type redefinition and identity (D-587), VM error locations (ADR-0036), seq class dispatch test
2 parents e87aba6 + 65bb395 commit 1bcca2a

22 files changed

Lines changed: 1135 additions & 245 deletions

‎.dev/debt.yaml‎

Lines changed: 11 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -24,50 +24,6 @@
2424
# md-table-align no longer pads it to ~800 KB of whitespace.
2525

2626
active:
27-
- id: "D-587"
28-
status: |-
29-
open (2026-09-09). USE-AFTER-FREE, user-reachable panic. Re-registering a
30-
deftype/defrecord name frees a TypeDescriptor that live instances still
31-
point at. `type_descriptor.zig registerType` keys `rt.types` by the SIMPLE
32-
name, and on re-registration does `fetchRemove(name)` then frees the old
33-
descriptor (each field_layout entry name, the layout slice, fqcn,
34-
defining_ns, method_table entries, protocol_impls, then
35-
`gpa.destroy(old.value)`). Every constructed instance holds
36-
`inst.descriptor` as a raw pointer to it, so `print.zig
37-
printTypedInstance` then reads `field_layout` out of freed memory: the
38-
symptom is `thread panic: integer overflow` (a garbage field index or
39-
layout length), the defect is the free.
40-
Repro (ReleaseSafe v1.14.3): `(ns aaa) (defrecord Point [x y]) (def a
41-
(->Point 1 2)) (ns bbb) (defrecord Point [x y]) (def b (->Point 1 2))
42-
(prn (pr-str aaa/a))` panics; printing `bbb/b` is fine. Variants: two ns
43-
with DIFFERENT names is fine; two ns SAME name panics; SAME ns redefining
44-
the same name panics (this is the ordinary REPL / `require :reload` path
45-
`dev/user.clj` depends on); same name with a different field count panics;
46-
the deftype form raises `WriteFailed` instead, same root.
47-
TWO separable defects: (1) the registry is keyed by simple name, so
48-
`aaa/Point` and `bbb/Point` collide with no redefinition at all (clj gives
49-
each ns its own class); (2) re-registration frees a descriptor live
50-
instances reference.
51-
ADR-0019 forbids a panic on a user-reachable path, and this one is reached
52-
by re-evaluating a defrecord form. It also caps the native-suite migration:
53-
`test/clj/suites/` shares ONE image, so no two suites may define the same
54-
type name (worked around 2026-09-09 by renaming defrecord_test's record to
55-
DPoint; 45 type definitions across 103 suites, no other duplicate today).
56-
Found while migrating phase7_defrecord.sh to a native suite.
57-
Card `[CLJW-TD-UAF]` 20260909211032-6471bc49.
58-
category: "correctness"
59-
quality_floor: "correctness: a user-reachable panic from an ordinary redefine"
60-
barrier: |-
61-
Needs an ADR: the free is deliberate (comments cite row 7.7 cycle 5 and
62-
D-190 / ADR-0068) and exists to avoid leaking on redefine, so removing it
63-
trades the leak back. Candidate shapes: key `rt.types` by the qualified
64-
name when `defining_ns` is present and RETIRE rather than free a replaced
65-
descriptor (a leak bounded by redefinition count, which is what clj does
66-
with the old class), or make descriptors GC-managed so a descriptor dies
67-
only when no instance references it. Devil's-advocate fork is mandatory at
68-
this depth. Not blocked on anything external: the repro is one file and the
69-
owning function is ~70 lines.
70-
last_reviewed: "2026-09-09"
7127
- id: "D-586"
7228
status: |-
7329
open (2026-09-05, ADR-0195 trigger). UPSTREAM: zwasm/D-584. zwasm's JIT
@@ -7583,6 +7539,17 @@ discharged:
75837539
refs: |-
75847540
src/runtime/collection/literal.zig (the one literal construction); src/eval/analyzer/analyzer.zig (the constant fold); src/eval/backend/vm/compiler.zig and vm.zig (the extend opcodes); docs/spec/formats/chunk-v12.edn (bytecode format v12).
75857541
discharged_at: "2026-10-02"
7542+
- id: "D-587"
7543+
status: |-
7544+
DISCHARGED 2026-10-14 by ADR-0198 plus ADR-0201. Both defects the row named are fixed. (1) rt.types is keyed by <defining_ns>.<Name>, so aaa.Point and bbb.Point are two classes (ADR-0198), and a bare constructor binds the class of the namespace its form was analyzed in ([CLJW-CTOR-ANALYZE-TIME]: a ctor site carries definer_ns, bytecode v13), so `aaa/->Point` builds aaa.Point even after bbb registered a namesake. (2) A displaced descriptor is retired, never freed, until Runtime.deinit (ADR-0198; ADR-0201 Part B makes that the one lifetime rule for every user descriptor). ADR-0201 also made membership descriptor identity: instance?, .isInstance, isa? and multimethod dispatch agree with (= (class x) C) for a redefined record, a namesake in another namespace and a reify class. The original repro prints both records and the narrowed redefine on both backends.
7545+
Was: open (2026-09-09), USE-AFTER-FREE, user-reachable panic. Re-registering a deftype/defrecord name freed a TypeDescriptor that live instances still pointed at, and the registry was keyed by the simple name, so two namespaces defining Point collided with no redefinition at all. Symptom `thread panic: integer overflow` from printTypedInstance, or `WriteFailed` for a deftype.
7546+
category: "correctness"
7547+
barrier: |-
7548+
none, discharged. Pinned by test/clj/suites/type_redefine_test.clj (the six scenarios of the former type_descriptor_redefinition e2e, incl. 3000 redefinitions interleaved with reify under a forced collection and the defining-namespace ctor binding), test/clj/suites/user_class_identity_test.clj, the type_redefinition diff corpus, and the class_name.zig / analyzer.zig unit tests. Still open and separate: rt.types_by_simple still exists for the bare-name paths that are not ctor sites (card 20260909231443-1ba4303c), and D-574 (reify mints a descriptor per evaluation).
7549+
last_reviewed: "2026-10-14"
7550+
refs: |-
7551+
ADR-0198, ADR-0201; src/runtime/type_descriptor.zig (registerType retires), src/runtime/class_name.zig (isInstanceOfClass, classIsaOf, descriptorIsa), src/eval/analyzer/special_forms.zig (constructInstance definer_ns lookup), docs/spec/formats/chunk-v13.edn.
7552+
discharged_at: "2026-10-14"
75867553
conventions: |-
75877554
- Category vocabulary is a closed set (guarded by scripts/check_debt_category.sh; synonyms were normalized 2026-10-08: structural->structure, tests/test->testing, numerics->numeric, performance->perf). Allowed: app backend bench bootstrap build clj-parity concurrency correctness debt-hygiene demo discipline distribution docs docs-onboarding ecosystem error-system eval eval-semantics feature gc governance host-surface infra interop isolation license macro meta-reorg missing-ns namespace numeric perf polymorphism process project-flow reader refactor regex robustness runtime scope security structure testing tooling wasm. A new category is added here first, in the same commit as the row that needs it.
75887555
- Structure: two top-level lists — `active:` and `discharged:`. Active entries carry id / status / category / barrier (+ optional quality_floor / last_reviewed); discharged entries carry id / discharged_at / resolution. Edit as YAML (no Markdown table to align).

‎CHANGELOG.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,25 @@ first stable `1.0.0` tag; pre-1.0 `alpha` / `rc` tags may still change surfaces.
3131
- **A shorter README** led by calling C from Clojure, with the architecture,
3232
size and startup charts kept.
3333

34+
### Fixed
35+
36+
- **Redefining a deftype or defrecord is safe, and a user class is matched by
37+
identity, not by name.** Re-registering a type no longer frees a descriptor
38+
live instances still use (the `thread panic: integer overflow` on printing
39+
an old instance is gone). `aaa.Point` and `bbb.Point` are two classes, and
40+
`instance?`, `.isInstance`, class `isa?` and multimethod dispatch agree with
41+
`(= (class x) C)` for a redefined record, a namesake in another namespace
42+
and a reify class. A bare `(Point. x y)` or `->Point` builds the class of the
43+
namespace the form was written in, even after another namespace registered a
44+
namesake of a different shape. Compiled chunks move to format v13.
45+
46+
- **The VM reports errors at their form.** `set!` on an unbound var, an
47+
unresolved or unsupported `.member` call, a static method call, a
48+
constructor arity failure, `import` and `require`/`refer` failures raised on
49+
the VM with no source position (`0:0`) while TreeWalk reported line and
50+
column. Both backends now report the same position, and the dual-backend
51+
test oracle checks it for every failing case.
52+
3453
## [1.15.0] - 2026-10-13
3554

3655
### Added

‎docs/spec/formats/chunk-v13.edn‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
;; Add-only bytecode archive amendment, 2026-10-14 (ADR-0034 D11, ADR-0198,
2+
;; ADR-0201, [CLJW-CTOR-ANALYZE-TIME]). Earlier records remain unchanged.
3+
;; v13 keeps the chunk-body framing of chunk-v12.edn except for one field
4+
;; added to each ctor_sites entry, so a decoder of an older version must
5+
;; reject the chunk.
6+
{:chunk-format
7+
{:version 13
8+
:magic "CLJW"
9+
:base-layout "chunk-v12.edn"
10+
:semantic-history
11+
[{:version 13
12+
:change "A ctor site carries definer_ns, the namespace its `(Name. args)` form was analyzed in, so a bare user-type name resolves to <definer_ns>.<Name> before any reading the calling namespace could give it (ADR-0198, ADR-0201)."}]
13+
:ctor-site
14+
{:fields [{:name :type_name :body "u32 length, then bytes"}
15+
{:name :definer_ns :body "u32 length, then bytes; empty when the name was qualified at analysis or no namespace was current"}
16+
{:name :arg_count :body "u16"}]}
17+
:compatibility "The runtime rejects a chunk version unequal to 13; rebuild older payloads."}}

‎src/eval/analyzer/analyzer.zig‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2444,6 +2444,27 @@ test "ctor call (Foo. ...) analyses into interop_call_node .constructor" {
24442444
try testing.expect(n.interop_call_node.kind == .constructor);
24452445
try testing.expectEqualStrings("Foo", n.interop_call_node.type_name);
24462446
try testing.expectEqual(@as(usize, 2), n.interop_call_node.args.len);
2447+
// [CLJW-CTOR-ANALYZE-TIME]: a bare type name carries the namespace it was
2448+
// analyzed in, so eval-time lookup qualifies it lexically.
2449+
try testing.expectEqualStrings(fix.env.current_ns.?.name, n.interop_call_node.definer_ns);
2450+
}
2451+
2452+
test "constructInstance resolves <definer_ns>.<Name> before the last-wins simple name" {
2453+
const td_mod = @import("../../runtime/type_descriptor.zig");
2454+
var fix: TestFixture = undefined;
2455+
try fix.init(testing.allocator);
2456+
defer fix.deinit();
2457+
// aaa.Point has two fields; bbb.Point, registered later, has one, so a
2458+
// simple-name lookup would pick bbb's and reject two arguments.
2459+
const aaa = try td_mod.registerType(&fix.rt, "Point", "aaa", &.{ "x", "y" }, &.{}, .defrecord);
2460+
const bbb = try td_mod.registerType(&fix.rt, "Point", "bbb", &.{"q"}, &.{}, .defrecord);
2461+
const two = [_]Value{ Value.initInteger(1), Value.initInteger(2) };
2462+
const a = try special_forms.constructInstance(&fix.rt, &fix.env, "Point", "aaa", &two, .{});
2463+
try testing.expect(a.tag() == .typed_instance);
2464+
try testing.expect(a.decodePtr(*const td_mod.TypedInstance).descriptor == aaa);
2465+
const one = [_]Value{Value.initInteger(9)};
2466+
const b = try special_forms.constructInstance(&fix.rt, &fix.env, "Point", "bbb", &one, .{});
2467+
try testing.expect(b.decodePtr(*const td_mod.TypedInstance).descriptor == bbb);
24472468
}
24482469

24492470
test "member access (.x inst) analyses into interop_call_node .instance_member" {

‎src/eval/analyzer/special_forms.zig‎

Lines changed: 40 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -104,9 +104,22 @@ pub fn constructInstance(
104104
rt: *Runtime,
105105
env: *Env,
106106
type_name: []const u8,
107+
/// The namespace the ctor form was analyzed in ("" when none). A user
108+
/// type registered as `<definer_ns>.<type_name>` wins over every other
109+
/// reading of the bare name, as a class name clj compiled there would.
110+
definer_ns: []const u8,
107111
args: []const Value,
108112
loc: error_catalog.SourceLocation,
109113
) anyerror!Value {
114+
if (definer_ns.len != 0) {
115+
var qbuf: [512]u8 = undefined;
116+
if (std.fmt.bufPrint(&qbuf, "{s}.{s}", .{ definer_ns, type_name })) |qualified| {
117+
if (rt.types.get(qualified)) |td| return constructFromDescriptor(rt, env, td, type_name, args, loc);
118+
} else |_| {
119+
// Namespace + name longer than the buffer: no registered key
120+
// could match it either, so fall through to the steps below.
121+
}
122+
}
110123
// D-284: `(MapEntry. k v)` / `(new clojure.lang.MapEntry k v)` constructs an
111124
// entry, so collection libs that build entries explicitly (priority-map,
112125
// ordered, …) work. It used to return a plain 2-vector, from when a map entry
@@ -151,9 +164,23 @@ pub fn constructInstance(
151164
}
152165
const td = resolveJavaSurface(rt, env, type_name) orelse
153166
return error_catalog.raise(.symbol_unresolved, loc, .{ .sym = type_name });
167+
return constructFromDescriptor(rt, env, td, type_name, args, loc);
168+
}
169+
170+
/// The descriptor-driven half of `constructInstance`: a field layout
171+
/// (deftype / record) allocates an instance, a `<init>` entry (java surface)
172+
/// is called through the runtime vtable, anything else takes no constructor.
173+
fn constructFromDescriptor(
174+
rt: *Runtime,
175+
env: *Env,
176+
td: *const type_descriptor_mod.TypeDescriptor,
177+
type_name: []const u8,
178+
args: []const Value,
179+
loc: error_catalog.SourceLocation,
180+
) anyerror!Value {
154181
if (td.field_layout) |fl| {
155182
if (args.len != fl.len)
156-
return error_catalog.raise(.arity_not_expected, loc, .{ .got = args.len, .fn_name = type_name, .expected = fl.len });
183+
return error_catalog.raise(.arity_not_expected, loc, .{ .got = args.len, .fn_name = td.fqcn orelse type_name, .expected = fl.len });
157184
return type_descriptor_mod.allocInstance(rt, td, args);
158185
}
159186
if (td.lookupMethod(null, "<init>")) |me| {
@@ -261,14 +288,20 @@ pub fn analyzeCtorCall(
261288
// lexically correct one. Eval-time resolution would consult the CALLER's
262289
// ns and miss it (hiccup.util's `(URI. s)` inside to-uri, called from
263290
// user code). Guarded on the FQCN actually resolving as a registered
264-
// surface. A USER deftype keeps the bare spelling here and resolves on
265-
// the eval-time path (which also allows forward refs to later deftypes);
266-
// since ADR-0198 that path finds it by qualifying the bare name with the
267-
// current ns inside `host_class_resolve.resolve`, rather than by the
268-
// simple-name registry hit this comment used to describe.
291+
// surface.
292+
//
293+
// A USER type keeps the bare spelling and resolves at eval time, which is
294+
// what lets a `(Name. …)` inside the defrecord's own expansion refer to a
295+
// type registered a moment later. The namespace of THIS analysis rides
296+
// along as `definer_ns` so that eval-time lookup qualifies the name
297+
// lexically (ADR-0198): the current ns at eval time is the CALLER's, and
298+
// a bare `Point` there fell to the last-wins simple-name index, so
299+
// `aaa/->Point` built whichever `Point` any namespace registered last.
269300
var resolved_name = type_name;
301+
var definer_ns: []const u8 = "";
270302
if (std.mem.findScalar(u8, type_name, '.') == null) {
271303
if (env.current_ns) |cur_ns| {
304+
definer_ns = try arena.dupe(u8, cur_ns.name);
272305
if (cur_ns.imports.get(type_name)) |fqcn| {
273306
if (rt.types.get(fqcn) != null) {
274307
resolved_name = fqcn;
@@ -288,6 +321,7 @@ pub fn analyzeCtorCall(
288321
n.* = .{ .interop_call_node = .{
289322
.kind = .constructor,
290323
.type_name = resolved_name,
324+
.definer_ns = if (resolved_name.ptr == type_name.ptr) definer_ns else "",
291325
.args = args,
292326
.loc = form.location,
293327
} };

‎src/eval/backend/tree_walk.zig‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -722,7 +722,7 @@ fn evalConstructorCall(rt: *Runtime, env: *Env, locals: []Value, n: node_mod.Int
722722
buf[i] = try eval(rt, env, locals, &arg_node);
723723
af.publish(i + 1);
724724
}
725-
return special_forms.constructInstance(rt, env, n.type_name, buf[0..n.args.len], n.loc);
725+
return special_forms.constructInstance(rt, env, n.type_name, n.definer_ns, buf[0..n.args.len], n.loc);
726726
}
727727

728728
/// `(.member recv args...)` / `(.-field recv)` unified instance member

0 commit comments

Comments
 (0)