0022 E decided the same day it deferred: tools.analyzer, spike run in-repo

The survey the deferral named ran against the clones, not recall.
tools.analyzer's core is 827 maintained lines whose host contract is four
dynamic vars, whose -parse already handles every S3 form with the arity
and def-ordering rules D requires, and whose pass scheduler is where
closure conversion and tail marking want to live. Its JVM host layer,
read as the worked example, is 58% reflection machinery a Wasm target
does not have.

The measured surprise is the macro layer: with :inline off,
clojure.core's own macros expand S3's forms into pure special forms and
core-var calls -- the leak table is two entries (map destructuring's
kwargs branch, chunked doseq) -- so the cljs fork's day-one cost of
owning destructure/let/loop/fn/defn buys nothing here, and the host ops
double as mechanical leak detectors. bb spike-analyzer commits the
executable half: (defn fib ...) through bare tools.analyzer 1.2.2, zero
host ops, recursion resolved.

TeaVM, read as the third WasmGC emitter, independently lands on 0004's
three-loads-plus-call_ref vtable shape.

Claude-Session: https://claude.ai/code/session_01XF5Hfq4Ca2N2XYEzWQQuHt

Author: chaploud

bb.edn

@@ -146,6 +146,9 @@
                                "-o" "target/imp.component.wasm")
                           (run "clojure" "-M:dev" "-m" "cljwit.spike.host-import"))}
 
+  spike-analyzer {:doc "Can bare tools.analyzer analyze S3's forms without host-op leaks? See 0022 E."
+                  :task (run "clojure" "-M:dev" "-m" "cljwit.spike.analyzer")}
+
   spike-reflect {:doc "Can a component describe itself at run time? See 0014."
                  :task (do (babashka.fs/create-dirs "target")
                            (run "wasm-tools" "parse" "dev/resources/echo.wat" "-o" "target/echo.core.wasm")

deps.edn

@@ -5,7 +5,10 @@
  {;; --- development -------------------------------------------------------
   :dev  {:extra-paths ["test" "dev"]
          :extra-deps  {io.github.cognitect-labs/test-runner
-                       {:git/tag "v0.5.1" :git/sha "dfb30dd"}}}
+                       {:git/tag "v0.5.1" :git/sha "dfb30dd"}
+                       ;; bb spike-analyzer (0022 E): the S3 front-end survey's
+                       ;; executable half.
+                       org.clojure/tools.analyzer {:mvn/version "1.2.2"}}}
 
   ;; `clj -M:test` — the unit lane
   :test {:extra-paths ["test"]

dev/cljwit/spike/analyzer.clj

@@ -0,0 +1,73 @@
+(ns cljwit.spike.analyzer
+  "Can bare tools.analyzer, with a minimal host layer, analyze S3's forms
+   without leaking JVM host ops? (`0022` E's survey ran this and the answer
+   decided the front end.)
+
+   The host layer is four dynamic vars, modeled on tools.analyzer's own
+   core-test. The one decision embedded here: `macroexpand-1` applies only
+   `:macro` vars, never `:inline` — with inlining on, `(+ x 1)` becomes
+   `(. clojure.lang.Numbers (add x 1))` and `+`-vs-`+'` semantics leave
+   cljwit's hands. Without it, `+` stays an `:invoke` of `#'clojure.core/+`,
+   which the emitter maps to the overflow-checking runtime (`0022` C).
+
+   Success is a `:def` AST for fib whose distinct op set contains no
+   `:host-call`/`:host-field`/`:host-interop`/`:maybe-host-form`/`:new` —
+   the host ops double as leak detectors."
+  (:refer-clojure :exclude [macroexpand-1])
+  (:require [clojure.pprint :as pp]
+            [clojure.tools.analyzer :as ana]
+            [clojure.tools.analyzer.ast :as ast]
+            [clojure.tools.analyzer.env :refer [with-env]]
+            [clojure.tools.analyzer.utils :refer [resolve-sym]]))
+
+(set! *warn-on-reflection* true)
+
+(defn macroexpand-1
+  "Only `:macro` vars expand; specials and locals shield; no `:inline`."
+  [form env]
+  (if (seq? form)
+    (let [op (first form)]
+      (if (ana/specials op)
+        form
+        (let [v (resolve-sym op env)]
+          (if (and (not (-> env :locals (get op)))
+                   (:macro (meta v)))
+            (apply v form env (rest form))
+            form))))
+    form))
+
+(def ^:private HOST-OPS #{:host-call :host-field :host-interop :maybe-host-form :new})
+
+(def ^:private fib-form
+  '(defn fib [n]
+     (if (< n 2)
+       n
+       (+ (fib (- n 1)) (fib (- n 2))))))
+
+(defn -main [& _]
+  (let [env  {:context :ctx/expr :locals {} :ns 'user}
+        genv (atom {:namespaces
+                    {'user         {:mappings (ns-map 'clojure.core)
+                                    :aliases {} :ns 'user}
+                     'clojure.core {:mappings (ns-map 'clojure.core)
+                                    :aliases {} :ns 'clojure.core}}})
+        result (binding [ana/macroexpand-1 macroexpand-1
+                         ana/create-var    (fn [sym env]
+                                             (doto (intern (:ns env) sym)
+                                               (reset-meta! (meta sym))))
+                         ana/parse         ana/-parse
+                         ana/var?          var?]
+                 (with-env genv
+                   (ana/analyze fib-form env)))
+        ops    (sort (distinct (map :op (ast/nodes result))))
+        leaks  (filter HOST-OPS ops)]
+    (println "top :op      =>" (:op result))
+    (println "via :raw-forms =>" (pr-str (map first (:raw-forms result))))
+    (println "distinct ops =>" (pr-str ops))
+    (println "host-op leaks =>" (if (seq leaks) (pr-str leaks) "none"))
+    (println "recursive fib resolved =>"
+             (-> result :init :expr :methods first :body :ret :else
+                 :args first :fn :var pr-str))
+    (when (seq leaks)
+      (pp/pprint result)
+      (System/exit 1))))

doc/design/0022-s3-compiler-shape.md

@@ -2,8 +2,11 @@
 
 **Status:** proposed · 2026-07-30 · the entry survey for S3, run against the
 sibling projects' recorded failures, the ClojureScript compiler's source, and
-the toolchain as it exists today. Three decisions are evidenced enough to
-make; the analyzer choice is not, and saying so is the point.
+the toolchain as it exists today. The first draft made three decisions and
+deferred the analyzer to a survey; the survey ran the same day (E, amended)
+and decided it — tools.analyzer core plus our own host layer, validated by
+`bb spike-analyzer`. An adversarial review of the whole note is in flight;
+`proposed` until it lands.
 
 ## The question
 
@@ -111,9 +114,63 @@ baseline.
   builds an open S3 decision (`0010`). The coverage report is a
   deliverable: it is the number S0's verdict rests on.
 
-### E. Deferred, with its survey named: the analyzer
-
-The front end is the load-bearing choice and the evidence is not in yet.
+### E. The analyzer: tools.analyzer core, plus our own host layer
+
+**Amended 2026-07-30, same day: the survey ran, and it decides.** Everything
+below the rule is the original deferral, kept because its reasoning shaped
+the survey. What the survey found (all read from source or executed;
+`bb spike-analyzer` is the committed, re-runnable half):
+
+- **tools.analyzer core is 827 lines, maintained (4466d93, 2026-02-15), and
+  host-agnostic by contract**: four dynamic vars (`macroexpand-1`, `parse`,
+  `create-var`, `var?` — `analyzer.clj:129-148`). Its `-parse` natively
+  handles every S3 form, already enforces the three JVM arity rules and the
+  declare-before-init `def` ordering that D requires, and its pass
+  scheduler (`passes.clj`: `:walk`/`:depends` metadata, same-walk fusion)
+  is the natural home for closure conversion, tail marking and boxing —
+  `collect-closed-overs` already exists.
+- **The host layer's cost is measured by example**: tools.analyzer.jvm is
+  3,172 lines, of which **58% is JVM reflection and class resolution that a
+  Wasm target simply does not have**; ~450 lines are the generic-host
+  pattern to adapt (ns map, macroexpansion, the Gilardi `do`-unrolling) and
+  ~680 are passes cljwit would write under *any* route.
+- **clojure.core's own macros are reusable directly — measured, with a
+  two-entry leak table.** With `:inline` **off** (the one deliberate
+  divergence from t.a.jvm's `macroexpand-1`), `defn`, `and`, `or`, `when`,
+  `->`, `loop` and vector destructuring expand into pure special forms and
+  core-var calls; `(+ x 1)` stays an `:invoke` of `#'clojure.core/+`, which
+  keeps `+`-vs-`+'` in cljwit's hands (C). The leaks: map destructuring's
+  kwargs branch touches `PersistentArrayMap` (needs a shim), and chunked
+  `doseq` (out of S3 scope). The host ops (`:host-call` etc.) double as
+  **free leak detectors**: any future macro that expands into interop fails
+  analysis mechanically.
+- **The spike ran, in this repo**: `bb spike-analyzer` analyzes
+  `(defn fib [n] …)` through bare tools.analyzer 1.2.2 with a ~40-line
+  host binding — top op `:def`, recursive `fib` resolves mid-parse, and
+  the distinct op set contains **zero host ops**.
+- The cljs fork's real cost is now counted: `core.cljc` defines 166
+  macros, and the S3-relevant ones it *re-implements* — `destructure`
+  (~140 lines), `let`, `loop`, `fn`, `defn`, `and`, `or` — are exactly
+  what the tools.analyzer route defers until a corpus line actually leaks.
+  TeaVM, read as the third emitter, independently confirms `0004`'s
+  three-loads-plus-`call_ref` vtable shape
+  (`WasmGCVirtualCallGenerator.java:56-79`).
+
+**Falsified by:** map-based pseudo-vars failing in passes (the spike
+interned real JVM vars; the next spike swaps in a map-returning
+`create-var` — the contract allows it, but it was not run); the leak table
+growing past shim scale once the corpus passes S3 (`case*`'s JVM hashing,
+`binding`, `lazy-seq` — if cljwit ends up owning ~20+ core macros, the
+cljs fork's macro layer stops being a differentiator and this reopens); a
+macro capturing a non-EDN JVM object into an expansion (mechanically
+detectable: a `:const` whose `:val` does not print); the scheduler proving
+unable to express the closure-conversion ordering.
+
+---
+
+*The original deferral, for the record:*
+
+The front end is the load-bearing choice and the evidence was not in yet.
 What is known (read from source, 2026-07-30, paths in `.ref/clojurescript`):
 
 - **cljs.analyzer is a fork, not a dependency.** The pipeline is
@@ -132,11 +189,9 @@ What is known (read from source, 2026-07-30, paths in `.ref/clojurescript`):
 - clj.wasm's author — the one person who tried this before — picked the
   cljs side (`.ref/clj.wasm/plan.md`).
 
-**The next unit is that survey**: add `tools.analyzer` (and TeaVM, the
-production JVM-hosted WasmGC emitter, reported browser-ready as of late
-2025 — secondary source only) to `refs.json`, read both, and decide in an
-amendment here. Deciding today on a recalled API would be `0013`'s failure
-shape — a claim about the tool, never run.
+**The next unit is that survey** — which is the amendment above: read
+both, run the spike, decide. Deciding on a recalled API would have been
+`0013`'s failure shape — a claim about the tool, never run.
 
 ## Alternatives rejected
 
@@ -145,8 +200,14 @@ shape — a claim about the tool, never run.
   a measured optimisation.
 - **An opt-in differential oracle.** The sibling measured what opt-in
   costs: weeks of silent drift.
-- **Deciding the analyzer now.** The strongest candidate's advantages are
-  recalled rather than read; see E.
+- **Deciding the analyzer on recall** (the note's own first draft, for a
+  few hours). The survey replaced recall with reads and a spike; see E.
+- **Forking cljs.analyzer.** Owning `destructure`/`let`/`loop`/`fn`/`defn`
+  from day one, for parse methods tools.analyzer also has; its 71 `js*`
+  bottoms in `core.cljc` are the JS target's, not ours.
+- **A purpose-built analyzer for the S3 subset.** Re-implements 827 lines
+  of subtle, maintained validation against the standing prefer-the-
+  ecosystem constraint.
 - **`--closed-world` in dev mode, or anywhere heap-sharing units exist** —
   already measured as a trap (`0009`, the `wat` skill).
 

doc/status.md

@@ -20,10 +20,13 @@ _Short by design, and printed at every session start — so findings live in
    the watcher — waits for a compiler to serve, so:
 3. **S3 is entered** (`0022`, proposed): WAT-text emission through the
    pinned toolchain, a CI-mandatory differential oracle against `clojure`
-   itself, i31-fixnum numerics with a throwing boxed i64. **The analyzer
-   choice is deliberately deferred** — the next unit is its survey: add
-   `tools.analyzer` and TeaVM to `refs.json`, read them, amend `0022`.
-   The sibling failures that shaped all of this are cited in the note.
+   itself, i31-fixnum numerics with a throwing boxed i64, and — the survey
+   ran same-day — **tools.analyzer core plus our own host layer** as the
+   front end, with clojure.core's macros reused directly (`:inline` off, a
+   two-entry leak table, host ops as free leak detectors).
+   `bb spike-analyzer` is the executable half. Next unit: fold the in-flight
+   adversarial review of `0022`, then the first vertical slice — corpus
+   harness + `(def x 42)` through analyze → emit → both engines.
 
 Done since the last update: `0016` `own<T>` handles, `0017` host imports
 (A–F), `0018` host-defined resources, `0012`'s `ex-data` contract shrunk to