0024: fn lands and the modes finally disagree — defn fib runs on both engines

Closures are subtypes of a per-arity-slot $Fn base, each capture
signature in its own rec group (0009); method funcs declare (type $sigN)
explicitly because a bare func type canonicalizes as its own singleton
group and loses identity with the slot field — 0009's rule, met again.
Free variables are computed on uniquified names (the pass is now
scheduled in analyze-forms); the capture-shadow entry exists because
name-based free vars are wrong without it. recur in a method rebinds
the argument params through the same stack-swap as loop.

The first real mode divergence: :prod direct-links a fn def into an
immutable constant struct.new global and plain calls (^:dynamic/^:redef
excluded, re-def refused), :dev keeps the mutable global and the
cast+slot+call_ref path. The dev-only redef entry introduces :modes,
because a shared corpus could never express what the open world exists
to preserve (0024 review, finding 4).

The same review re-measured the depth bands for this frame shape --
wasmtime ~8-10k, V8 ~10.3-10.7k, oracle JVM ~8.5-9.2k, none of them
0022 D's minimal-function numbers -- so depth entries pin 3000; the
exhaustion entry adds trap row 2 (SOE matches on class alone, arrives
unwrapped, V8 reports RangeError which run.mjs now classifies). It also
verified constant struct.new globals, cross-rec-group subtyping, and
that null-call/cast traps stay distinguishable from every table row.

One regression caught mid-unit: the (def y y) ordering fix had broken
recursive fn defs in dev; a fn init now registers its global before the
method bodies emit, because a body's var reads happen at call time.

45 corpus entries, 220 assertions inside the flake, gate green.

Claude-Session: https://claude.ai/code/session_01XF5Hfq4Ca2N2XYEzWQQuHt

Author: chaploud

corpus/run.mjs

@@ -14,6 +14,9 @@ const instance = new WebAssembly.Instance(
 try {
   console.log(`result ${instance.exports.entry()}`);
 } catch (e) {
-  if (e instanceof WebAssembly.RuntimeError) console.log(`trap ${e.message}`);
+  // V8 reports wasm stack exhaustion as RangeError, not RuntimeError
+  // (0024) — both are classified outcomes, not harness failures.
+  if (e instanceof WebAssembly.RuntimeError || e instanceof RangeError)
+    console.log(`trap ${e.message}`);
   else throw e;
 }

corpus/s3.edn

@@ -53,4 +53,44 @@
  ;; sibling's recorded bug) is unobservable until try/catch exists —
  ;; claimed nowhere until it does.
  {:id "def-redef-throws" :forms [(def x 5) (def x (quot 1 0)) x]}
- {:id "trap-div-zero"   :forms [(quot 1 0)]}]
+ {:id "trap-div-zero"   :forms [(quot 1 0)]}
+ ;; --- fn (0024). An entry carries :modes when only one mode can
+ ;; express it; default is both.
+ {:id "fn-anon-call"    :forms [((fn [x y] (+ x y)) 2 3)]}
+ {:id "fn-zero-arity"   :forms [((fn [] 7))]}
+ {:id "fn-multi-arity"  :forms [(let [f (fn ([x] x) ([x y] (* x y)))]
+                                  (+ (f 5) (f 3 4)))]}
+ {:id "fn-closure"      :forms [(let [a 10] ((fn [x] (+ x a)) 5))]}
+ {:id "fn-closure-shared" :forms [(let [a 2]
+                                    (let [g (fn [x] (* a x))]
+                                      (+ (g 3) (g 4))))]}
+ ;; Partial shadowing: the outer x is captured, the inner x is not —
+ ;; exactly what uniquified free-variable computation exists for.
+ {:id "fn-capture-shadow" :forms [(let [x 1]
+                                    ((fn [y] (+ x (let [x 10] (+ x y)))) 5))]}
+ {:id "fn-named-fact"   :forms [((fn fact [n] (if (< n 2) 1 (* n (fact (- n 1))))) 10)]}
+ {:id "fn-method-recur" :forms [((fn [n acc] (if (< 0 n) (recur (- n 1) (+ acc n)) acc)) 10 0)]}
+ {:id "fn-higher-order" :forms [((fn [f x] (f (f x))) (fn [n] (+ n 3)) 10)]}
+ {:id "fn-def-call"     :forms [(def dbl (fn [x] (* 2 x))) (dbl 21)]}
+ {:id "fn-as-value"     :forms [(def dbl (fn [x] (* 2 x)))
+                                ((fn [g] (+ (g 3) (g 4))) dbl)]}
+ ;; The roadmap's S3 sentence, verbatim shape: defn fib runs everywhere.
+ {:id "fn-defn-fib"     :forms [(defn fib [n] (if (< n 2) n (+ (fib (- n 1)) (fib (- n 2)))))
+                                (fib 20)]}
+ ;; Depth pin: 3000 frames, ~2.8x under the shallowest measured band for
+ ;; this frame shape — wasmtime ~8-10k, V8 ~10.3-10.7k, the oracle JVM
+ ;; ~8.5-9.2k (0024 §9).
+ {:id "fn-depth-safe"   :forms [(defn down [n] (if (< 0 n) (down (- n 1)) 42))
+                                (down 3000)]}
+ ;; Deliberate exhaustion on every lane, including the oracle: trap row 2.
+ {:id "fn-stack-exhaust" :forms [(defn down [n] (if (< 0 n) (down (- n 1)) 42))
+                                 (down 10000000)]}
+ ;; Redefinition taking effect is the open world's point (0009); prod
+ ;; direct-links and refuses, so this entry is dev-only (0024 §10).
+ {:id "fn-redef-dev"    :modes [:dev]
+  :forms [(def f (fn [] 1)) (def f (fn [] 2)) (f)]}
+ ;; ^:redef opts out of direct linking in prod (0004's exclusion rule),
+ ;; so redefinition works in both modes.
+ {:id "fn-redef-marked" :forms [(def ^:redef f (fn [] 1))
+                                (def ^:redef f (fn [] 2))
+                                (f)]}]

corpus/trap_table.edn

@@ -8,8 +8,15 @@
 ;; a division trap. All strings are observed, not recalled: the JVM's is
 ;; what `(quot 1 0)` actually throws, the engine strings verified
 ;; 2026-07-30 on the pinned toolchain (wasmtime 47.0.1, V8/node).
+;; Row 2 matches on class alone (:jvm-message "") — StackOverflowError's
+;; message is null, a recorded weakening of the jvm-message guard (0024).
 [{:row 1
   :class "java.lang.ArithmeticException"
   :jvm-message "/ by zero"
   :wasmtime "integer divide by zero"
-  :v8 "divide by zero"}]
+  :v8 "divide by zero"}
+ {:row 2
+  :class "java.lang.StackOverflowError"
+  :jvm-message ""
+  :wasmtime "call stack exhausted"
+  :v8 "Maximum call stack size exceeded"}]

doc/design/0024-fn-representation.md

@@ -0,0 +1,148 @@
+# 0024 — fn in the slice: closures, arities, and the first real mode divergence
+
+**Status:** proposed · 2026-07-30 · adversarially reviewed the same day,
+before implementation finished. The review verified the mechanics it
+attacked (constant `struct.new` globals, cross-rec-group subtyping,
+distinguishable null/cast traps, `RangeError` on V8 exhaustion, the three
+arity rules enforced by bare tools.analyzer, safe SOE recovery in the
+oracle) and changed four things, marked *(review)*: the depth pin's
+rationale re-measured for the real frame shape, row 2's class-only match
+made explicit, K's definition extended to call sites, and a per-mode
+corpus annotation for behaviour only one mode can express.
+
+## The question
+
+`0022` D contracts what `fn` must do (multi-arity ships with it, varargs
+carry an explicit rest-mode, no tail calls, the depth bands). It does not
+pin how a closure is laid out in WasmGC, how a call site reaches the code,
+or where `:dev` and `:prod` emission first produce different bytes. Those
+are this note's decisions — the first ones `0023`'s harness can check
+mode-divergently, since until now both modes emitted identical modules.
+
+## The decision
+
+1. **Function signatures.** `$sigN` is
+   `(func (param (ref eq)) (param (ref null eq))×N (result (ref null eq)))`
+   — parameter 0 is the closure itself, through which the body reads its
+   captures. Uniform boxing per `0023`; unboxed signatures are a prod
+   optimisation for later, not the baseline (`0022` C).
+2. **`$Fn` is a base struct with one nullable typed-function-ref field per
+   arity**, 0..K where K is the module's maximum arity **over both method
+   definitions and fn-value call sites** *(review)* — a higher-order site
+   calling arity N > any defined method must find a null slot and take
+   the classified out-of-contract trap, not emit a `struct.get` on a
+   field that does not exist. Per-arity slots are `0004`'s measured
+   shape — collapsing them to one funcref was measured there and
+   rejected for 0.11 ns; the structure stands.
+3. **Each capture signature is its own subtype of `$Fn`, in its own rec
+   group** (`0009`'s identity rule: unit-private types must not perturb
+   shared groups). The callee's prologue casts parameter 0 to its leaf
+   type once into a local (B4: casts are cheap by depth), and capture
+   reads are `struct.get` on that local. Captures are by value, which is
+   exact — Clojure has no `set!` on locals, so there is nothing mutable
+   to alias.
+4. **Free variables are computed on uniquified names.**
+   `clojure.tools.analyzer.passes.uniquify` is now scheduled inside
+   `analyze-forms` — the pass-scheduler home `0022` E predicted for
+   closure conversion. Partial shadowing
+   (`(let [x 1] (fn [] (+ x (let [x 2] x))))`) makes name-based free-var
+   computation wrong without it.
+5. **Multi-arity is slots in one struct; a missing arity is a null slot.**
+   Calling it traps on the null `call_ref` — reported by the harness as
+   out-of-contract until an `ArityException` representation exists (the
+   throw representation is open, `0022` C). The three JVM arity rules are
+   enforced at analysis where tools.analyzer sees the fn literal.
+6. **Varargs analyze but refuse to emit.** `:variadic?` methods are a
+   loud out-of-slice error: the rest parameter is a seq, no seq
+   representation exists, and `0022` D's rest-mode contract stays marked
+   untested — its corpus entry enters the day printing does (`0022` B.3).
+7. **`recur` in a fn method rebinds the method's argument parameters**
+   (never parameter 0, the closure) — the method's `:loop-id` registers
+   them exactly as `loop` registers its bindings, same simultaneous
+   stack-swap emission.
+8. **Calls.** The generic path is `0004`'s measured shape — reach the fn
+   value (local, global, or fresh), `ref.cast (ref $Fn)`, `struct.get`
+   the arity-N slot, `call_ref $sigN` — for every call whose target is a
+   fn value. **The first mode divergence**: a top-level
+   `(def f (fn …))` whose var is neither `^:dynamic` nor `^:redef`
+   compiles in `:prod` to an **immutable global holding a constant
+   `struct.new`**, and every call through the var to a **direct `call`**
+   of the method — Clojure's own direct-linking rule and exclusion list
+   (`0004`). In `:dev` the var stays a mutable `(ref null eq)` global
+   assigned at eval, and calls go through the generic path — the var
+   indirection `0009`'s open world requires. Re-`def` of a direct-linked
+   fn in `:prod` is a loud refusal (`^:redef` is how you ask for
+   redefinability, same as on the JVM).
+9. **Stack depth.** No tail calls, for parity (`0022` D). *(review — the
+   bands re-measured for this note's actual frame shape, because `0022`
+   D's 30–40k/10–20k numbers were for a minimal function and it predicted
+   fatter frames itself)*: with the self param, prologue cast, slot load
+   and `call_ref`, **wasmtime exhausts at ~8–10k, V8 at ~10.3–10.7k, and
+   the oracle JVM itself at ~8.5–9.2k** for an eval'd recursive fn on the
+   default test-JVM stack — the oracle is a band like any other lane.
+   Depth-sensitive entries therefore pin **3,000 frames**, ~2.8× under
+   the shallowest measured band. One entry exhausts the stack on purpose
+   and adds **trap-table row 2**: `java.lang.StackOverflowError` — whose
+   message is null, so **this row matches on class alone**, a recorded
+   weakening of the table's jvm-message guard *(review)*; SOE also
+   arrives *unwrapped* by `eval`, so the CompilerException unwrap is
+   correctly a no-op here — ↔ wasmtime's "call stack exhausted" ↔ V8's
+   `RangeError` ("Maximum call stack size exceeded"), which is *not* a
+   `WebAssembly.RuntimeError`, so the V8 runner learns to classify
+   `RangeError` as a trap.
+10. **Per-mode corpus entries** *(review)*. `:prod` refuses fn re-`def`
+    (point 8), so the dev-mode behaviour it exists to preserve —
+    redefinition taking effect — could never be exercised by entries
+    that run in both modes. A corpus entry may carry `:modes [:dev]`
+    (default: both); the fn-redef entry is dev-only by construction,
+    and the `^:redef` entry covers both modes.
+
+## Why
+
+- Every load-bearing shape is inherited from a measured decision: the
+  per-arity slots and the direct-linking exclusion rule from `0004`, the
+  rec-group isolation from `0009`, cheap leaf casts from B4, boxed
+  signatures from `0023`/`0022` C, the depth bands from `0022` D's
+  measurements.
+- The divergence point (calls through vars, not `def` itself) is where
+  Clojure's own direct linking draws it, so no new vocabulary or caveat
+  list is invented — including its known behaviour that a direct-linked
+  fn ignores redefinition, which here is a refusal instead of a silent
+  staleness.
+
+## Alternatives rejected
+
+- **One funcref field plus arity dispatch in the callee.** `0004`
+  measured the per-arity table against this and kept the table; an
+  arity switch on every call buys one word per closure.
+- **Reusing `0004`'s full `$obj`/vtable-array layout now.** That layer
+  exists for protocols and open user types (selector coloring over the
+  whole program). A fn value's callee set is closed at creation; slots
+  suffice. The two converge when protocols land, not before.
+- **A universal closure type with an `(ref null eq)` capture array.**
+  Loses typed capture reads, adds a bounds-checked load per capture, and
+  `0009` already decided per-signature types for sharing reasons.
+- **Implementing varargs against an ad-hoc pair/array type.** Every rest
+  argument would be re-done when seqs land; `0022` D's rest-mode
+  contract is about *marking*, which analysis already preserves.
+- **`return_call` for self-recursion now.** The toolchain supports it,
+  but parity with the JVM's missing TCO is the deliberate baseline
+  (`0022` D); tail-position marking lands with its consumer, post-S3.
+- **Direct-linking in `:dev` too.** Kills redefinition, which is the
+  entire point of the dev mode's open world (`0009`).
+
+## What would falsify this
+
+- A measured prod-vs-dev call-cost gap too small to justify two paths —
+  nothing here is a performance *claim*; the benchmark named by `0010`
+  (coverage/guard precision) arrives with specialisation, and if direct
+  linking buys nothing measurable the prod path simplifies to the
+  generic one.
+- The null-slot arity trap colliding with a future `ArityException`
+  row — expected; resolving it is part of the throw-representation
+  decision (`0022` C), and the trap table gains the distinction then.
+- A capture set the uniquified-name computation gets wrong — one corpus
+  entry exists specifically to catch partial shadowing.
+- `struct.new` with `ref.func` failing as a constant expression on
+  either engine — checked at validation, would force prod fns back to
+  start-time assignment (a smaller divergence, same call-site shape).

doc/status.md

@@ -18,24 +18,25 @@ _Short by design, and printed at every session start — so findings live in
    reports tiered drift (`:stale`/`:modified`/`:missing`/`:orphans`), and
    `check` is the CI verb. What remains of S2 — the nREPL entry point and
    the watcher — waits for a compiler to serve, so:
-3. **S3's first vertical slice is landed** (`0022` decided the shape,
-   `0023` the harness and representation): the differential oracle is in
-   the gate — `corpus/s3.edn` (29 scalar entries) runs through real
-   `clojure` (fresh namespace per entry) and through
+3. **Every S3 form now compiles and is oracle-checked** (`0023` the
+   harness, `0024` fn): `corpus/s3.edn` — 45 entries: literals, `if`,
+   `let`, `loop`/`recur`, `def`, `fn` with multi-arity, closures,
+   method-`recur`, higher-order calls — runs on every `bb check` through
+   real `clojure` (fresh namespace per entry) and through
    `cljwit.analyze` → `cljwit.emit` → `wasm-tools` → **wasmtime and V8,
-   in both modes** (which emit identical bytes until calls to def'd fns
-   land — stated, not hidden), with `corpus/trap_table.edn` row 1
-   (`ArithmeticException` ↔ division trap) exercised. Representation
-   pinned in `0023`: values are `(ref null eq)`, fixnums i31, nil = null,
-   false/true = singleton structs, overflow arm `unreachable`. Still
-   open, each with its benchmark named (`0022`): the dev-loop output
-   format (before the nREPL unit), the boxed-i64 lane (before the
-   emitter grows past i31), the throw representation. `loop`/`recur` and
-   scalar `def` are in the corpus (29 entries); mode divergence has still
-   not started — it begins with *calls to* def'd fns (direct linking,
-   `0004`), so the next unit is **`fn`**: `0022` D's full contract —
-   multi-arity ships with it, varargs carry an explicit rest-mode — and
-   the first depth-sensitive entry brings the trap table its stack row.
+   in both modes**. `(defn fib [n] …) (fib 20)` returns 6765 on both
+   engines — the roadmap's S3 sentence, minus an actual browser page.
+   **The modes now genuinely diverge**: `:prod` direct-links fn defs
+   (immutable constant global + plain `call`, `^:dynamic`/`^:redef`
+   excluded, re-`def` refused), `:dev` keeps var indirection; the
+   dev-only redef entry carries `:modes [:dev]` (`0024` §10). The trap
+   table has two exercised rows (division, stack exhaustion — depth
+   bands re-measured for the real frame shape, pin at 3,000). Varargs
+   analyze but refuse to emit until seqs exist. Still open, each with
+   its benchmark named (`0022`): the dev-loop output format (before the
+   nREPL unit), the **boxed-i64 lane** (before the emitter grows past
+   i31 — fib's n = 46…92 domain), the throw representation. Next unit:
+   **the boxed-i64 benchmark**, predictions first (`0002`'s rule).
 
 Done since the last update: `0016` `own<T>` handles, `0017` host imports
 (A–F), `0018` host-defined resources, `0012`'s `ex-data` contract shrunk to

src/cljwit/analyze.clj

@@ -15,10 +15,17 @@
   (:refer-clojure :exclude [macroexpand-1])
   (:require [clojure.tools.analyzer :as ana]
             [clojure.tools.analyzer.env :refer [with-env]]
+            [clojure.tools.analyzer.passes :refer [schedule]]
+            [clojure.tools.analyzer.passes.uniquify :refer [uniquify-locals]]
             [clojure.tools.analyzer.utils :refer [resolve-sym]]))
 
 (set! *warn-on-reflection* true)
 
+;; Uniquify is the one scheduled pass: emission computes free variables
+;; and lexical scope by name, which partial shadowing would corrupt —
+;; `(let [x 1] (fn [] (+ x (let [x 2] x))))` has two locals both named x.
+(def ^:private run-passes (schedule #{#'uniquify-locals}))
+
 (defn- macroexpand-1
   "Only `:macro` vars expand; specials and locals shield; no `:inline`."
   [form env]
@@ -53,5 +60,5 @@
                   ana/parse         ana/-parse
                   ana/var?          var?]
           (with-env genv
-            (mapv #(ana/analyze % env) forms))))
+            (mapv #(run-passes (ana/analyze % env)) forms))))
       (finally (remove-ns ns-sym)))))