|
| 1 | +# 0022 — S3 opens: the compiler's shape, and the one decision it defers |
| 2 | + |
| 3 | +**Status:** proposed · 2026-07-30 · the entry survey for S3, run against the |
| 4 | +sibling projects' recorded failures, the ClojureScript compiler's source, and |
| 5 | +the toolchain as it exists today. Three decisions are evidenced enough to |
| 6 | +make; the analyzer choice is not, and saying so is the point. |
| 7 | + |
| 8 | +## The question |
| 9 | + |
| 10 | +S0's verdict (`0010`) licenses a compiler whose viability rests on per-site |
| 11 | +guard precision; S1/S2 built the host side. S3 is `def`, `fn`, `if`, `let`, |
| 12 | +`loop`/`recur`, calls and literals, running in both a browser and wasmtime. |
| 13 | +What shape does that compiler take — and what does it refuse to decide |
| 14 | +before verifying? |
| 15 | + |
| 16 | +## The decision |
| 17 | + |
| 18 | +### A. Emit WAT text; assemble and optimize with the pinned toolchain |
| 19 | + |
| 20 | +The compiler prints WAT; `wasm-tools parse` assembles, `wasm-opt` optimizes |
| 21 | +(prod only, per `0009`). Three pieces of evidence, all checked 2026-07-30: |
| 22 | + |
| 23 | +- This repo's whole measured pipeline already is exactly that |
| 24 | + (`bench/s0/run.clj`, the `wat` skill) — S3 inherits validated |
| 25 | + infrastructure, including the GC/tail-call flag gotchas. |
| 26 | +- **J2CL/j2wasm — a JVM-hosted compiler with the same constraints — "emits |
| 27 | + naïve Wasm using the text format and relies on the Binaryen toolchain"** |
| 28 | + (<https://github.com/google/j2cl/blob/master/docs/getting-started-j2wasm.md>). |
| 29 | +- clj.wasm — the prior attempt at this exact project — died building its own |
| 30 | + assembler from the spec grammar (`.ref/clj.wasm/chronology.md`: "the spec |
| 31 | + document is more rendering centric than I hoped"). Never own the assembler. |
| 32 | + |
| 33 | +No off-the-shelf JVM WasmGC-emission library was found (searched |
| 34 | +2026-07-30); every WasmGC producer owns its writer or emits text. Binaryen's |
| 35 | +own lowering guidance shapes the emitter: more types rather than fewer |
| 36 | +(`--type-merging` cleans up), maximally refined and immutable fields, |
| 37 | +vtables as immutable globals — with immutability a **prod-mode** property, |
| 38 | +since dev-mode var globals are mutable by design (`0009`) |
| 39 | +(<https://github.com/WebAssembly/binaryen/wiki/GC-Implementation---Lowering-Tips>). |
| 40 | + |
| 41 | +**Unmeasured and flagged:** dev-mode REPL latency of a JVM → |
| 42 | +`wasm-tools parse` process spawn per form. Benchmark before the nREPL unit |
| 43 | +assumes it is fine. |
| 44 | + |
| 45 | +### B. The differential oracle is CI-mandatory from the first special form |
| 46 | + |
| 47 | +Semantics claims are checked against `clojure` itself — the standing rule, |
| 48 | +mechanized: |
| 49 | + |
| 50 | +1. A committed **corpus** of source forms; every special form lands with its |
| 51 | + corpus lines in the same commit, and any "X works" claim leaves its |
| 52 | + probing expression behind. (ClojureWasm discharged a debt row once by |
| 53 | + listing functions that were not done — `.ref/ClojureWasm/.dev/`; the |
| 54 | + corpus is the countermeasure.) |
| 55 | +2. The oracle lane runs the corpus through real `clojure`, **batched in one |
| 56 | + JVM** (per-form spawning is the sibling's recorded anti-pattern), |
| 57 | + printing `pr-str` results or error *classes*. |
| 58 | +3. Compiled lanes run on **both wasmtime and node, in both modes** (dev |
| 59 | + open-world and prod direct-linked) from day one. ClojureWasm made the |
| 60 | + comparison opt-in and it "consumed weeks" of silent drift |
| 61 | + (`.ref/ClojureWasm/.dev/decisions/0005`, `0036`); mode divergence is the |
| 62 | + same bug class. Early corpus entries are scalar-returning |
| 63 | + (`wasmtime run --invoke`, exactly `bench/s0`'s mechanism, whose driver |
| 64 | + already fails on wrong answers); printing arbitrary values is itself |
| 65 | + corpus-gated later. |
| 66 | +4. Values compare **structurally** (read back with `clojure.edn`) — never |
| 67 | + string-compare set/map printing; errors compare by class. Every diff is |
| 68 | + classified: a bug, or a numbered divergence with a pinned test. |
| 69 | + |
| 70 | +### C. Numerics: fixnum i31, a boxed i64 that throws, and `+` is not `+'` |
| 71 | + |
| 72 | +fib's semantic baseline: Clojure longs. `+` **throws** on 64-bit overflow — |
| 73 | +only `+'` promotes — which the sibling learned the hard way |
| 74 | +(`.ref/ClojureWasm/docs/clojure_vs_clojurewasm.md`). The representation is |
| 75 | +the measured one: i31 fixnums (B3: boxed arithmetic at 0.31× JVM on both |
| 76 | +engines, overflow check free — `0002`), overflowing into a boxed i64 whose |
| 77 | +own overflow throws. Promotion to bigints is `+'`'s job and out of S3 scope. |
| 78 | +Unboxed i64 emission is a prod-mode optimisation for proven types, not the |
| 79 | +baseline. |
| 80 | + |
| 81 | +### D. The S3 forms, on the measured substrate |
| 82 | + |
| 83 | +- **`def`** — dev: a mutable global, calls load it (var indirection); prod: |
| 84 | + direct `call` except `^:dynamic`/`^:redef` (`0004`, Clojure's own |
| 85 | + direct-linking rule). Declare at analysis, assign at eval: `(def x 5) |
| 86 | + (def x (/ 1 0)) x` is 5, and a throwing re-`def` must not wipe the old |
| 87 | + root (`.ref/ClojureWasm/.dev/decisions/0038` — both naive orders are |
| 88 | + recorded bugs). |
| 89 | +- **`fn`** — a closure is the `$obj` header plus captured locals, per-arity |
| 90 | + funcref slots (`0004`, b1's measured prototype); each capture signature |
| 91 | + is its own struct type in its own rec group (`0009`), callee prologue |
| 92 | + casts to leaf (B4: 0.09 ns). **Varargs carry an explicit rest-mode** — |
| 93 | + rest-binding and `apply`-spreading are two operations, and inferring one |
| 94 | + from argument shape is a recorded correctness bug |
| 95 | + (`.ref/ClojureWasm/.dev/decisions/0042`); `((fn [a & xs] xs) 1 '(2 3))` |
| 96 | + ⇒ `((2 3))` goes in the day-one corpus. Multi-arity ships with `fn`, not |
| 97 | + after it, with the three JVM arity rules enforced at analysis (`0041`). |
| 98 | +- **`if`** — truthiness is nil-or-false; the false representation (singleton |
| 99 | + box vs i31 encoding) is an emitter decision the first corpus entry pins. |
| 100 | +- **`let`** — Wasm locals; captured ones become closure fields, which makes |
| 101 | + closure conversion an analysis pass. |
| 102 | +- **`loop`/`recur`** — a block and a `br`; no tail-call feature needed. |
| 103 | + Non-self tail calls (`return_call`) are out of S3 scope, but the analyzer |
| 104 | + marks tail position from the start — cljs's `:context :return` is not a |
| 105 | + real tail-position analysis, and a `call` where `return_call` was needed |
| 106 | + fails only at depth (the structural-defect class |
| 107 | + `.ref/ClojureWasm/.dev/lessons/structural_defect_hunting.md` names: run |
| 108 | + everything at large n). |
| 109 | +- **Calls** — generic three-loads-plus-`call_ref`; specialisation only above |
| 110 | + the measured crossover, conservative 80% threshold by default, per-lane |
| 111 | + builds an open S3 decision (`0010`). The coverage report is a |
| 112 | + deliverable: it is the number S0's verdict rests on. |
| 113 | + |
| 114 | +### E. Deferred, with its survey named: the analyzer |
| 115 | + |
| 116 | +The front end is the load-bearing choice and the evidence is not in yet. |
| 117 | +What is known (read from source, 2026-07-30, paths in `.ref/clojurescript`): |
| 118 | + |
| 119 | +- **cljs.analyzer is a fork, not a dependency.** The pipeline is |
| 120 | + read→analyze→emit one top-level form at a time; the AST contract |
| 121 | + (`:op`/`:children`) and the `parse` methods for exactly S3's forms are |
| 122 | + clean — but `js*` bottoms out in JS strings **71 times in `core.cljc`**, |
| 123 | + 7 of 41 ops are JS interop, name resolution bakes in goog/node munging, |
| 124 | + and the tag lattice is JS-shaped. The macros are the real cost either |
| 125 | + way: every core macro that expands into host idioms needs a cljwit |
| 126 | + version. |
| 127 | +- **tools.analyzer was designed for retargeting** (pluggable |
| 128 | + `macroexpand-1`/`parse`, scheduled passes — which a Wasm backend wants: |
| 129 | + closure conversion, boxing, tail marking). But its pass-scheduling |
| 130 | + specifics were **recalled, not read** in this survey, and it is not in |
| 131 | + `.ref/`. |
| 132 | +- clj.wasm's author — the one person who tried this before — picked the |
| 133 | + cljs side (`.ref/clj.wasm/plan.md`). |
| 134 | + |
| 135 | +**The next unit is that survey**: add `tools.analyzer` (and TeaVM, the |
| 136 | +production JVM-hosted WasmGC emitter, reported browser-ready as of late |
| 137 | +2025 — secondary source only) to `refs.json`, read both, and decide in an |
| 138 | +amendment here. Deciding today on a recalled API would be `0013`'s failure |
| 139 | +shape — a claim about the tool, never run. |
| 140 | + |
| 141 | +## Alternatives rejected |
| 142 | + |
| 143 | +- **Own binary emitter first.** clj.wasm's recorded death, for zero |
| 144 | + measured benefit at S3's scale; binary emission stays available later as |
| 145 | + a measured optimisation. |
| 146 | +- **An opt-in differential oracle.** The sibling measured what opt-in |
| 147 | + costs: weeks of silent drift. |
| 148 | +- **Deciding the analyzer now.** The strongest candidate's advantages are |
| 149 | + recalled rather than read; see E. |
| 150 | +- **`--closed-world` in dev mode, or anywhere heap-sharing units exist** — |
| 151 | + already measured as a trap (`0009`, the `wat` skill). |
| 152 | + |
| 153 | +## What would falsify this |
| 154 | + |
| 155 | +- The JVM→`wasm-tools` per-form latency making WAT-text emission untenable |
| 156 | + for the REPL loop — measured before the nREPL unit, and binary emission |
| 157 | + is the recorded exit. |
| 158 | +- The tools.analyzer read (E's survey) contradicting the recalled API — the |
| 159 | + amendment records whichever way it lands. |
| 160 | +- A corpus entry where structural comparison cannot express the contract |
| 161 | + (print order, hash values) — becomes a numbered out-of-contract |
| 162 | + declaration, the sibling's `0122` pattern. |
0 commit comments