Skip to content

ladder.md

Latest commit

 

History

History
237 lines (215 loc) · 28.6 KB

ladder.md

File metadata and controls

237 lines (215 loc) · 28.6 KB
Raw

Real-world library compatibility ladder

Ranked by pure-Clojure degree (1 = zero host interop, easiest/earliest; 5 = threads / java.io / reflection-heavy). See README.md for the method, the load mechanism (-cp manual classpath), and the status vocabulary.

Probed on cljw built from main (2026-06-06). Rows marked loads / partial / fails were actually require'd; rows marked not-probed are static source inspection only. As of Stage 1.2, deps.edn resolution works (:paths/:local/root/:aliases/:git/url), so rungs are now probed via real deps.edn git coordinates, not just -cp (rung 4 below was the first such probe).

deps.edn :mvn policy (2026-06-07, ADR-0101 amendment 1): a :mvn/version dep is now SKIPPED (not rejected) — nearly every lib's own deps.edn declares org.clojure/clojure {:mvn/version …} (= cw itself), which previously aborted resolution. Satisfaction is decided at require time by namespace availability; org.clojure/clojure is silently provided, other skipped coords get a one-line stderr warning. A dep deps.edn with no :paths defaults to src/. Going- forward probe method = a mini deps.edn project with :git/url+:git/sha, replacing corpus copies + hand-laid -cp. Verified end-to-end: medley loads via git coords (find-first/ index-by correct); priority-map (zero-mvn) already did.

rank lib version pure-degree status first-blocking-gap
1 medley master 1 loads none — find-first/dissoc-in/map-keys/index-by/deep-merge/uuid?/abs all correct
2 clojure.math.combinatorics master 1 loads none on VALUES (27-expr edge sweep byte-identical to clj). Deeper 2026-06-21 re-audit (ns-publics surface, not just happy-path values) caught a GENERIC bug: its private reify-bool macro leaked into ns-publics → defmacro ^:private never set the Var private flag (analyzeDefmacro built its def_node without is_private). Fixed at root (commit 8b834d2e, e2e phase7_defmacro_private). Lesson: audit the META/introspection surface, not only output values.
3 clojure.tools.cli master (cljc) 1 verified none — committed verified_project (parse-opts options/arguments/:default/:errors/:summary, clj-confirmed)
4 clojure.data.priority-map master 1 loads FULLY FUNCTIONAL — (priority-map :a 3 :b 1 :c 2) → peek=[:b 1] (lowest), count=3, keys=(:b :c :a), vals=(1 2 3). Drove the whole deftype host-interface stack: clojure.lang.* family (D-275/D-279/D-280), java.util.Map/Iterable host_inert (D-281/ADR-0103), clojure.core.protocols (D-282), clj-name .method dot-calls (D-283), (MapEntry. …)→2-vec (D-284), keys/vals seq-derivation (D-285).
4b flatland.ordered (amalloy) master 1 verified NOW a committed verified_project (2026-06-14): ordered-map keys/vals/assoc/dissoc + ordered-set insertion-order/dedup all clj-correct. The IEditableCollection/host-interface chain that earlier blocked it (D-286b family) is cleared. History: re-probed 2026-06-07 after D-287 (Java arrays); D-286a cleared bare deftype-supertype names; D-287 the Java-array backing store.
5 clojure.data.generators master 2 loads FULLY FUNCTIONAL 2026-06-07 - loads + bit-for-bit seeded parity with clj: (binding [gen/*rnd* (java.util.Random. 42)] [(gen/long)(gen/long)(gen/boolean)(gen/double)]) => [-5025562857975149833 -5843495416241995736 false 0.9420735430282128] (identical to clj). Drove D-289 (java.util.Random LCG), D-295 (Short/Byte statics), D-296 (syntax-quote machinery clojure.core-qualified). The test.check foundation is now LIVE on cljw.
5b hive-test 0.3.x 2 verified VERIFIED 2026-09-04 as a committed source-coordinate project: its .cljc deftrifecta runs golden, 100-case test.check property, and two mutation checks inside cljw (3 tests / 7 assertions). Both hive-test and test.check are full-SHA source pins; no Maven JAR is used.
6 clojure.tools.reader master 2 parked PARKED 2026-06-07 (Step-0.6 finding): cleared D-288 (mutable field), D-287 (arrays), D-291 (Closeable), D-292 (multi-protocol extend-type) - all general wins. Remaining chain is DEAD interop in cljw: extend-type on java.io.PushbackReader/Reader/InputStream (D-293) + bare extend map-form; cljw uses its OWN reader path so these extensions can never dispatch. Loading them is low-value. Resume if a LIVE host-target consumer appears.
7 clj-commons/clj-yaml-pure n/a 2 not-probed (placeholder: pure EDN/string-shaped; verify a real pure-clj yaml exists)
8 cuerdas master (cljc) 3 partial D-410: java.text.BreakIterator (abbreviate; LOAD still blocked — eager static resolution) — the regex layer is cleared: \p{...} property classes + class-internal \Q/\u/non-ASCII landed 2026-06-13 (D-409)
9 clojure.data.xml master 3 not-probed NEEDS-ROW: javax.xml.stream (StAX) reader/writer interop
10 instaparse master (cljc) 4 not-probed D-238: out dynamic var (print-method Failure binds it; owner-gate dissolved 2026-06-13) — Comparable/IPersistentVector/CharSequence supertypes + auto_flatten_seq all cleared 2026-06-13
11 clojure.data.json master 2 verified NOW a committed verified_project (2026-06-14): write-str/read-str round-trip + :key-fn keyword + scalars (string/number/bool/null/array) all clj-confirmed. The java.io writer/reader chain the earlier row feared is NOT on data.json's core string path — write-str/read-str work directly. Re-graded pure-degree 4→2.
12 cheshire 5.x 5 not-probed NEEDS-ROW: Jackson (com.fasterxml.jackson) JNI/Java class — not pure Clojure
13 clj-time n/a 5 not-probed not pure (joda-time Java dep) — out of pure ladder, kept as a boundary marker
14 core.async master 5 not-probed NEEDS-ROW: threads / executors / go-macro state machine (Campaign Stage 1.7 Phase B)
15 next.jdbc n/a 5 not-probed not pure (java.sql.* JDBC) — out of pure ladder, boundary marker
16 metosin/malli master (cljc) 2 verified NOW a committed verified_project (2026-08-12): validate/explain over predicate, :map, :vector, :maybe, :enum and sequence-regex (:cat) schemas, plus explicit-keys. Nine sequential blockers, every fix general — see the rung note below. First SCHEMA library on the ladder, and the one that unblocks a schema-first consumer stack.

Notes per rung

  • Rungs 1–3 (loads, degree 1): the ladder head is green. medley, math.combinatorics, and tools.cli all require and run correctly on cljw today with only a manual classpath. This is the concrete evidence that cljw's clojure.core surface (protocols, records, transducers, sorted collections, metadata, regex, reduce-kv) covers a real pure-Clojure library end to end.

  • Rung 16 (malli, verified): nine sequential blockers, discovered one per re-probe. Eight were surface gaps — LazilyPersistentVector/createOwning, PersistentArrayMap/createWithCheck, Murmur3/hashLong, java.lang.reflect.Array/newInstance, java.util.ArrayDeque, java.util.concurrent.TimeUnit, HashMap's (int, float) ctor + .putAll, and AbstractList/Vector as opaque classes. Two were deeper: instance? could not resolve a protocol by the interface name defprotocol generates (malli.registry.Registry), and valueToForm dropped symbol metadata on macro expansion's return leg — a general fault that shed type hints, ^:private and mutable-field markers from every macro-produced form, found only because malli's transitive dep borkdude/dynaload happens to set! a ^:volatile-mutable field from a macro-emitted deftype. Two lessons worth carrying: a lib's own transitive deps (dynaload here) are part of its blocker chain and need their coord in the project's deps.edn; and a blocker that appears three layers from its cause is bisected by asking which construct works when written literally and fails when a macro emits it.

  • Rung 8 (cuerdas, partial → Pattern/quote landed): loads after its cuerdas.regexp dependency is laid out, but capitalize (and other fns) hit (java.util.regex.Pattern/quote ...) — a static Java-class method call cljw did not resolve. This was the first real feature gap the ladder found, as opposed to a missing-resolver blocker. Pattern/quote is now wired (Pattern.zig static surface; the regex engine already honored \Q…\E), clj-faithful bit-for-bit (corpus regex_pattern.txt). Re-probe cuerdas when it is re-fetched to find the next blocker (likely more Pattern statics: compile / matches / flag constants).

  • Rungs 5, 9–14 (NEEDS-ROW): static inspection shows each touches a Java surface cljw does not yet provide (seeded RNG, StAX XML, java.io reader/writer, Jackson, threads). These are the honest next gaps; the main loop converts each NEEDS-ROW: into a debt.yaml row classified against the Java-tier SSOT.

  • Rungs 13, 15 (not pure): kept as boundary markers so the ladder makes explicit where the pure-Clojure frontier ends — they will not load without the underlying Java library, regardless of cljw progress.

  • test.check + hive-test (verified 2026-09-04 via source coordinates): test.check 1.1.3 loads from its official Git source and drives hive-test's .cljc deftrifecta end to end in cljw. The committed proof covers golden, 100 generated property cases, and two caught mutants. This supersedes the 2026-06-07 proxy blocker: current live source executes the tested path.

  • Pure deftype/algo libs probed (2026-06-07 via -cp):

    • clojure.data.finger-tree — (extend-type nil …) (nil-punning protocol extension) was unsupported (__extend-type!: expected type_descriptor, got nil). Fixed (extend-type + extend-protocol now accept a nil target → the per-Tag nil descriptor; clj-faithful, e2e phase14_extend_type_nil). Advanced :56 → :138 — (defdigit a) macro hit two deftype-method-lowering gaps, both now fixed (clj-faithful, e2e phase14_deftype_method_lowering): (a) syntax-quote-qualified method params (user/_ from a bare _ inside a backtick — clj's deftype/reify strip the ns; raw fn* still rejects, parity kept); (b) empty method bodies (m [_]) → nil. Advanced :138 → :405 — clojure.lang.Util/… static interop. Now wired (ADR-0108 — new runtime/clojure/lang/ host-surface tree; clojure.lang.Util ships 10 pure statics, oracle-verified, corpus clojure_lang_util.txt; Util/classOf deferred D-303). Advanced :405 → :519 — Counted (a clojure.lang.* interface marker, host_interfaces.yaml territory; next finger-tree blocker, parked). The clojure.lang.Util surface broadly unblocks the ~95 corpus Util call sites; RT/Numbers/APersistentMap are separate units.
    • clojure.algo.generic — (derive Object root-type): bare Object as a class VALUE to derive is unresolved (host-class-value family, D-293).
    • clojure.data.avl — (APersistentMap/mapHash …) + ^AtomicReference/ ^Comparator: deep clojure.lang/java-internal static interop. Park.
    • clojure.core.match — definterface (Tier-D-adjacent). clojure.data.int-map needs clojure.core.reducers (bundled-ns gap, D-273). clojure.algo.monads needs clojure.tools.macro (Compiler-dep, parked).

  • Opaque host-class VALUES (ADR-0109, 2026-06-07): recognised JVM numeric classes cljw collapses away (java.math.BigInteger/Integer/Short/Byte/Float) now resolve as distinct class VALUES — (= (type 5) Integer)/(instance? Integer 5) clj-faithfully false, (extend-type Integer …) a load-only no-op (no crash). Advanced numeric-tower :98/:127 → :162. java.lang.Object is also wired as the universal supertype ((isa? <any> Object)→true, (instance? Object x)→non-nil) — clojure.algo.generic (core ns) now LOADS (its (derive Object root-type) was the blocker). java.lang.Number also wired (narrow members Long/Double/BigInt/Ratio/BigDecimal) — clojure.algo.generic.arithmetic now LOADS. Remainder (D-293 PARTIAL): clojure.lang.IFn (core.contracts) + other markers as class values (each its narrow membership), and a deeper functional gap — (ga/+ 3 4) mis-dispatches on the [Number Number] class-vector defmethod (loads but the generic-fn dispatch machinery, separate). clojure.lang.IFn also wired (value-resolution + isa? callable-members + matchInterface IFn fixed to full ifn?) — clojure.core.contracts now LOADS (with its core.unify dep). The CORE host-class-value markers (opaque-numerics / Object / Number / IFn) are now all landed.

  • clojure.math.numeric-tower (probed 2026-06-07 via -cp): a DEEP java.math interop chain, parked. Advances :79 (D-301 empty-catch) → :98 (java.math.BigInteger class value, D-302) → :127 (Integer class value) → :162 (BigDecimal/ROUND_FLOOR static field), and beyond needs .setScale/.bitLength instance methods + (BigDecimal. …) ctors + extend-type on 8 host classes. Not a single-blocker rung — a full java.math interop unit blocked on D-293 (unified opaque/inert class-value resolver) + a BigDecimal surface. An exploratory opaque-class-VALUE probe (reverted) verified (= (type 5) Integer)/(instance? Integer x) are clj-faithful in isolation but COUPLED to extend-type (D-293); see debt D-293/D-302.

  • Re-probe (2026-06-07, after the ADR-0108/0109 arc) — LOADS now: clojure.algo.generic (core + .arithmetic + .comparison) via the Object/Number/host-class-value markers, clojure.core.contracts (with its core.unify dep) via the IFn marker, plus the earlier set. Frontier is now structural / host-surface / Tier-D (clean single-fix wins harvested): D-075 symbol-metadata layer (core.cache, algo.generic.math-functions — ADR-0037-deferred cross-cutting Symbol/Keyword/Var meta), clojure.lang internals (Util/RT statics, LazilyPersistentVector — host surfaces), reflection (tools.trace — Tier-D), clojure.lang.Compiler (tools.macro — Tier-D), threads (core.async — Phase B), java.io/StAX/Jackson (data.xml/json, instaparse — host surfaces). Each is a sizeable fresh unit; the next clean vein is the D-075 metadata layer or a clojure.lang.RT host surface.

  • Broad re-probe (2026-06-07) after the D-287..D-299 arc found 7 libs now LOAD: clojure.data.csv, clojure.data.codec.base64 (over D-287 byte-arrays), clojure.core.unify, potpuri.core (deep-merge bit-identical to clj), bouncer.core, qbits.ex, and clojure.data.zip (D-299 ns-form leniency). Deferred/parked at that time: symbol metadata = D-075 (interned symbols, structural); test.check = D-298 (later superseded by the 2026-09-04 native trifecta proof above); tools.macro = clojure.lang.Compiler (Tier-D).

  • Symbol-metadata layer LANDED (2026-06-07, D-304 / ADR-0110) — with-meta/meta on a symbol now work (fresh non-interned gc.alloc'd symbol; ns+name-structural identity, meta-ignored; .symbol GC-membrane flip + trace). This was the blocker the frontier listed as "D-075 symbol-metadata layer" for core.cache + algo.generic.math-functions. Var/atom/ns/ref metadata is the remaining sibling (D-239).

  • Ladder re-drive 2026-06-07 (core.cache via -cp) — D-304 cleared cache.clj:70 (with-meta symbol). Next: needs clojure.data.priority-map on cp (present, loads). Then cache.clj:115 = defcache names clojure.lang.Associative/Seqable/IPersistentCollection/Counted DIRECTLY as supertypes → D-306 (these collection-BASE interfaces are now recognised as direct deftype supertypes; count/seq/contains? dispatch). core.cache now advances :115 → :602, where the next gap is defn :pre/:post conditions (the % return-value binding in a :post vector, cache.clj:602) — the next ladder vein, a general defn feature many libs use. var-metadata Slice 1 (synth :name/:ns/:macro) also landed alongside.

  • clojure.core.cache FULLY LOADS (2026-06-07) — now a committed proof at test/conformance/verified_projects/core.cache/ (basic + LRU has?/lookup/miss/evict via deps.edn git coords; data.priority-map supplied as an explicit git-coord dep since core.cache pins it :mvn/version, skipped per ADR-0101 am.1). — defn :pre/:post condition maps landed (lowered at the fn-arity level for fn/defn/defmacro; % binds the return value in :post; a lone map stays a return value, clj parity; e2e phase14_fn_prepost). (clojure.core.cache/basic-cache-factory {:a 1}) → {:a 1} end-to-end. The symbol-meta (D-304) → collection-base-ifaces (D-306) → pre/post chain unblocked core.cache completely. core.memoize next: advances to memoize.clj:36 = clojure.lang.IDeref declared as a DIRECT deftype supertype (host_interfaces.yaml has the IDeref row but recognised: false — deref is a plain fn, not protocol-dispatched; modeling an IDeref protocol + recognising the supertype is the next vein).

  • IDeref/IPending deref-able family LANDED (2026-06-07, D-307) — clojure.lang.IDeref/IPending recognised as direct deftype/reify supertypes; new IDeref/IPending protocols; deref/@/realized? consult them for typed_instance (e2e phase14_deftype_ideref). core.memoize's RetryingDelay deftype now loads → advanced :36 → :67. core.memoize NEXT gaps (3, deeper): (1) (instance? clojure.lang.IDeref v) = host-class-VALUE resolution of a clojure.lang marker (D-293 family); (2) (reify clojure.lang.IDeref (deref [_] v)) = reify protocol_remap (expandReify lacks the rewriteProtocolRemap path deftype has); (3) ^:volatile-mutable fields + set! (D-288, ADR-level). core.cache stays fully loaded.

  • verified_projects via -M:verify (2026-06-07, ADR-0111 run mode) — the committed-proof convention now runs cljw -M:verify (deps.edn :verify alias → verify/-main), exercising the real deps.edn run-mode path. potpuri added as the 5th proof (deep-merge/map-vals/find-first; pure .cljc, no deps.edn → cljw default src). Two functional gaps surfaced while probing further libs (require LOADS but exercising a fn fails — the functional bar catches what bare require misses): clojure.core.unify → (.isArray <class-value>) on a type_descriptor (D-311, java.lang.Class instance-method surface, D-293 sibling, OPEN); clojure.data.zip → with-meta on a typed_instance (D-312, record IObj meta — FIXED same session, ADR-0112; data.zip is now the 6th verified_projects proof). Both were definition-derived root-cause fixes, not per-lib patches. data.generators stays deferred (maven layout, no deps.edn → src/main/clojure unresolvable without the lib's own deps.edn). Record structural-op meta threading ((meta (assoc (with-meta r m) …)) → m) landed in the same ADR-0112 commit (D-313, per the DA's divergence-suppression call).

  • data.generators re-probe (2026-06-25, D-528) — the rung-5 "FULLY FUNCTIONAL" above was for the core generators (the seeded long/boolean/double example, still byte-identical to clj). Running the library's full clojure.test suite this session surfaced a chain its earlier probe missed: (java.util.UUID. msb lsb), the whole java.math.BigInteger Java surface, and (BigDecimal. <BigInteger> scale) were all unwired — now FIXED (commits 0a5c65c4 / 33739d68 / 8400646a, each corpus-backed). One residue: a non-standalone-reproducible "Invalid float literal" from some generated double (D-532, fuzzy). Lesson re-confirmed: a happy-path example is not the full test suite; run the suite.

NEEDS-ROW gap summary (for the main loop)

These are candidate debt.yaml rows — the FIRST real blocker each library hits. Do not edit debt.yaml from this doc; the main loop creates the rows.

  1. NEEDS-ROW: java.util.regex.Pattern static method interop (Pattern/quote) — cuerdas rung 8. LANDED 2026-06-07 (Pattern.zig quote static method; engine already honored \Q…\E; corpus regex_pattern.txt). Re-probe cuerdas for the next Pattern static (compile/matches/flags) when re-fetched.
  2. NEEDS-ROW: java.util.Random seeded RNG — data.generators rung 5. Rowed 2026-06-07 → D-289 (probed: fails at ns-load on the seeded ctor; stateful-native-object capability, sibling to D-288).
  3. NEEDS-ROW: javax.xml.stream StAX reader/writer — data.xml rung 9.
  4. NEEDS-ROW: java.io FileNotFoundException catch + file slurp — instaparse rung 10.
  5. NEEDS-ROW: java.io PrintWriter/PushbackReader/StringWriter + clojure.pprint — data.json rung 11.
  6. NEEDS-ROW: threads/executors/go-macro — core.async rung 14 (already Campaign Stage 1.7 / Phase B).
  7. honeysql: clojure.template / defprotocol options / .. macro — all LANDED 2026-06-07 (811d1f08): bundled clojure.template, defprotocol :keyword value option parsing (:extend-via-metadata dispatch deferred D-314), and the .. member-threading macro (was missing + misparsed). honeysql (honey.sql) now PARKED as a drip-feed well → D-315: it keeps revealing one blocker at a time (java.util.Locale → then regex lookahead (?=…) in dehyphen). Locale was built+reverted this session (speculative without the lib verifying). Resume only when Locale + regex-lookahead land together. qbits.ex is the 7th verified proof (drove the aliased-macro analyzer fix, fa8628ea); core.unify the 8th (drove java.lang.Class methods + java.util.* interfaces, ee1552d6); integrant the 9th (drove ADR-0113 deferred clojure.lang.* host-refs + defmethod-empty-body + defmulti docstring/attr-map, bfa4c514).
  8. PRIORITY (user 2026-06-07): the next two libs are hiccup and honeysql. hiccup LANDED 2026-06-07 (10th verified_projects proof, ADR-0114) — the handover's "java.net.URI only" was a 7-blocker chain, each a general F-013 gap: java.net.URI + URLEncoder + StringBuilder + java.util.Iterator (GC-traced host_instance) + String/valueOf surfaces; java.util.Map extend-TARGET inert (AD-023); IPersistentVector/ISeq/Named extend-TARGET → native-tag distribution; Object-extension universal protocol-dispatch fallback; syntax-quote alias resolution + %N anon-param fix; exception_descriptor method_table leak fix. Finished-form follow-ups deferred: D-317 (ISeq tag table → derive from markers), D-318 (host_instance moving-GC / host_state_shape enum), D-319 (Object as descriptor-chain root). honeysql LANDED 2026-06-07 (11th proof, ADR-0115) — the survey's "Locale + lookahead" (2) was a 5-blocker chain: java.util.Locale/US /ROOT object-valued static fields + String.toUpperCase/toLowerCase 2-arg Locale overload + regex lookahead (?=…)/(?!…) (Pike-NFA zero-width predicate; FULL capture parity, no AD) + clojure.lang.IPersistentMap extend-TARGET → native map tags + .sym keyword method. D-315 discharged; D-320 (lookahead perf) deferred. The library-incorporation campaign is now on STAY (user 2026-06-07); the loop self-selects remaining quality work. For a future re-expansion, the patterns + gap-taxonomy + coverage-raising know-how are in memory 20260911004716-29dac88c (the 12 blocker classes). SSOT = memory 20260911004834-0aebca5a (the SSOT wiring audit).

Cross-cutting blocker (not a single row): no deps.edn / Maven resolver yet (Campaign Stage 1.2). Every transitive dependency must be fetched and laid out by hand, which is what holds the not-probed rows back from being load-verified.