Skip to content

Commit 65e5c0e

Browse files
authored
Merge pull request #20 from BuddhiLW/main
Release v1.15.4 from BuddhiLW/main
2 parents 596ebac + d973510 commit 65e5c0e

231 files changed

Lines changed: 11664 additions & 5525 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.dev/accepted_divergences.yaml‎

Lines changed: 35 additions & 73 deletions
Large diffs are not rendered by default.

‎.dev/debt.yaml‎

Lines changed: 27 additions & 5 deletions
Large diffs are not rendered by default.

‎.dev/optimizations.md‎

Lines changed: 14 additions & 0 deletions
Large diffs are not rendered by default.

‎.dev/wasm_percall_findings.md‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,62 @@ resolution inside `invoke` (zwasm/D-585) and the non-initial-thread stack
6666
query (zwasm/D-584), both upstream. `bench/wasm_percall.sh` is the standing
6767
measurement of this axis from here on.
6868

69+
### After the zwasm/D-585 fix (measured 2026-10-22)
70+
71+
`JitInstance` resolved every by-name invoke with `findExportFunc`, a walk of
72+
the section list plus an export-section decode, and the facade's `invokeJit`
73+
did it twice per call (signature, then invoke). zwasm `22a99bb1`
74+
(`fix/jit-percall` on the fork) serves the name from `CompiledWasm.exports`,
75+
the table compile already builds. The `big-*` cells of `bench/wasm_percall.clj`
76+
call a trivial export of a 47 KB module (`bench/wasm/percall/zigcool.wasm`),
77+
which makes a size-proportional cost visible. Median of 7 trials x 20k calls,
78+
ReleaseSafe `-Dwasm`, same host, load average 11-15, the before and after
79+
binaries run back to back:
80+
81+
| cell | before (d74e63b4) | after (22a99bb1) |
82+
|-------------------|-------------------|------------------|
83+
| JIT, 47 KB module | 5883 | 833 |
84+
| JIT, `add.wasm` | 1031 | 882 |
85+
| interp, 47 KB | 463 | 600 |
86+
| interp, `add.wasm`| 496 | 649 |
87+
88+
The JIT no longer pays by module size: the 47 KB cell now equals the tiny
89+
one. The interpreter rows did not change in kind (its lookup was always a
90+
table loop); their drift is load noise. `perf` was unavailable on the host
91+
(`perf_event_paranoid`), so the attribution rests on the A/B: the fix changes
92+
only the export lookup. The JIT stays above the interpreter on a trivial
93+
callee; what remains is the entry thunk and zwasm/D-584's per-call stack
94+
query, which ADR-0195 only shrinks.
95+
96+
### On the merged fork pin (measured 2026-10-22, zwasm `e9d74934`)
97+
98+
The same fix as merged on the fork's `main` (fork PRs #2 and #3), full
99+
`bench/wasm_percall.clj`, median of 7 x 20k calls, ReleaseSafe `-Dwasm`, load
100+
average 16. With the interp thread-trap fix in the pin, `interp-worker` no
101+
longer traps, so the whole bench runs.
102+
103+
| cell | ns/call | | cell | ns/call |
104+
|-------------|---------|-|---------------|---------|
105+
| big-jit | 736 | | jit-main | 614 |
106+
| big-interp | 527 | | interp-main | 479 |
107+
| big-auto | 742 | | fn-call | 179 |
108+
109+
`:jit` on the 47 KB module is 1.4x `:interp` (the acceptance bound is 2x) and
110+
within noise of `:jit` on `add.wasm`.
111+
112+
`perf record -g` on a loop of 400k `(wasm/call h "popcount" 255)` on the 47 KB
113+
module (`perf_event_paranoid` lowered to 1), stripped ReleaseSafe binaries:
114+
115+
- v1.15.3 (zwasm `d74e63b4`), 7.2 to 9.3 us a call: 88% of samples in cljw,
116+
about 65% in a single ~40-byte loop (`0x13e6910..0x13e6935`), entered from
117+
two distinct call sites per `wasm/call` (7.7% and 4.9% of the hottest
118+
address). That is the export-section walk, run twice per facade call
119+
(signature, then invoke), as zwasm/D-585 describes.
120+
- this branch (zwasm `e9d74934`), 0.7 to 0.9 us a call, the same as
121+
`add.wasm`: no hot loop. What remains is spread out: `pthread_getattr_np` and
122+
`pthread_getaffinity_np` (zwasm/D-584's per-call stack query, about 6%) and
123+
malloc/calloc/free (about 8%), none of which depends on the module.
124+
69125
## Two independent causes, both upstream, both already documented there
70126

71127
zwasm ADR-0209 (`.dev/decisions/0209_percall_latency_bench.md` in the zwasm

‎.gitattributes‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,3 +8,7 @@
88

99
# Keep read-only historical / generated content out of GitHub's language stats.
1010
docs/ja/archive/** linguist-documentation
11+
# Append-only ledgers: parallel branches each add rows at the end, so a union
12+
# merge is the correct resolution (the YAML and ledger gates still check it).
13+
.dev/optimizations.md merge=union
14+
.dev/debt.yaml merge=union

‎.github/workflows/bench-vs-bb.yml‎

Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
# bench-vs-bb: cljw vs babashka vs JVM Clojure on bench/vs_bb/workloads.
2+
#
3+
# Kanban 20261022012233-420c9fea, step 0 of decision 20261022012219-31249feb
4+
# (make cljw faster than bb): every perf step is judged by this table.
5+
#
6+
# This is a REPORT, not a gate. CI has exactly ONE gate configuration
7+
# (ci.yml -> scripts/ci_gate.sh -> test/cljw-test --only full, enforced by
8+
# scripts/check_gate_parity.sh); this workflow adds no tier to it. It never
9+
# fails a PR on a ratio: the job is continue-on-error and run.sh exits 0 on a
10+
# slow or mismatching workload. Its output is the step summary (the table plus
11+
# the YAML history row for this sha), so the ratio for any commit is on its
12+
# run page. A hosted runner is a noisy, shared machine: compare ratios
13+
# (cljw/bb on the same runner, same minute), not absolute milliseconds.
14+
#
15+
# Linux x86_64 only: the ratio is the signal, one host is enough, and the mac
16+
# leg would double the cost of a report.
17+
18+
name: bench-vs-bb
19+
20+
on:
21+
pull_request:
22+
paths-ignore: ['**/*.md', '.dev/**']
23+
push:
24+
branches: [main]
25+
paths-ignore: ['**/*.md', '.dev/**']
26+
workflow_dispatch:
27+
28+
permissions:
29+
contents: read
30+
31+
concurrency:
32+
group: bench-vs-bb-${{ github.ref }}
33+
cancel-in-progress: true
34+
35+
jobs:
36+
bench:
37+
name: cljw vs bb vs JVM (report, non-blocking)
38+
runs-on: ubuntu-22.04
39+
timeout-minutes: 45
40+
continue-on-error: true
41+
steps:
42+
- name: Checkout
43+
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
44+
with:
45+
ref: ${{ github.sha }}
46+
47+
- name: Cache Zig 0.16.0
48+
id: cache-zig
49+
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
50+
with:
51+
path: ~/.local/zig-0.16.0
52+
key: zig-0.16.0-x86_64-linux
53+
54+
- name: Install Zig 0.16.0
55+
if: steps.cache-zig.outputs.cache-hit != 'true'
56+
shell: bash
57+
run: |
58+
set -euo pipefail
59+
source .github/versions.lock
60+
mkdir -p ~/.local
61+
curl -fsSL "https://ziglang.org/download/${ZIG_VERSION}/zig-x86_64-linux-0.16.0.tar.xz" -o /tmp/zig.tar.xz
62+
rm -rf "$HOME/.local/zig-x86_64-linux-0.16.0" "$HOME/.local/zig-0.16.0"
63+
tar -xJf /tmp/zig.tar.xz -C ~/.local
64+
mv "$HOME/.local/zig-x86_64-linux-0.16.0" "$HOME/.local/zig-0.16.0"
65+
66+
- name: Add Zig to PATH
67+
shell: bash
68+
run: |
69+
echo "$HOME/.local/zig-0.16.0" >> "$GITHUB_PATH"
70+
echo "ZIG_GLOBAL_CACHE_DIR=$HOME/.cache/zig" >> "$GITHUB_ENV"
71+
72+
# Read-only restore of the gate's Linux build cache (ci.yml saves it).
73+
# Content-addressed, so a stale restore can only cost a rebuild.
74+
- name: Restore Zig deps + build (read-only)
75+
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
76+
with:
77+
path: |
78+
~/.cache/zig
79+
.zig-cache
80+
key: x86_64-linux-zigbuild-${{ hashFiles('build.zig.zon', 'build.zig') }}-${{ github.run_id }}
81+
restore-keys: |
82+
x86_64-linux-zigbuild-${{ hashFiles('build.zig.zon', 'build.zig') }}-
83+
x86_64-linux-zigbuild-
84+
85+
- name: Install Babashka (pinned, checksummed)
86+
shell: bash
87+
run: |
88+
set -euo pipefail
89+
source .github/versions.lock
90+
archive="/tmp/babashka-${BABASHKA_VERSION}.tar.gz"
91+
curl -fsSL "https://github.com/babashka/babashka/releases/download/v${BABASHKA_VERSION}/babashka-${BABASHKA_VERSION}-linux-amd64.tar.gz" -o "$archive"
92+
actual=$(sha256sum "$archive" | awk '{print $1}')
93+
[ "$actual" = "$BABASHKA_LINUX_AMD64_SHA256" ] || { echo "::error::Babashka checksum mismatch"; exit 1; }
94+
mkdir -p "$HOME/.local/bin"
95+
tar -xzf "$archive" -C "$HOME/.local/bin" bb
96+
echo "$HOME/.local/bin" >> "$GITHUB_PATH"
97+
98+
# JVM column without the clojure CLI: the runner image ships a JDK, the
99+
# jars come straight from Maven Central.
100+
- name: Fetch JVM Clojure jars
101+
shell: bash
102+
run: |
103+
set -euo pipefail
104+
m=https://repo1.maven.org/maven2/org/clojure
105+
d="$HOME/.cache/vs_bb_jars"; mkdir -p "$d"
106+
for j in clojure/1.12.4/clojure-1.12.4 spec.alpha/0.5.238/spec.alpha-0.5.238 \
107+
core.specs.alpha/0.4.74/core.specs.alpha-0.4.74 data.json/2.5.1/data.json-2.5.1; do
108+
curl -fsSL "$m/$j.jar" -o "$d/$(basename "$j").jar"
109+
done
110+
echo "CLJW_BENCH_JVM_CP=$(ls "$d"/*.jar | paste -sd:)" >> "$GITHUB_ENV"
111+
java -version
112+
113+
- name: Build cljw ReleaseSafe -Dwasm
114+
shell: bash
115+
run: zig build -Dwasm -Doptimize=ReleaseSafe -p "$RUNNER_TEMP/cljw-bench"
116+
117+
- name: Run bench/vs_bb
118+
shell: bash
119+
run: |
120+
set -uo pipefail
121+
hist="$RUNNER_TEMP/vs_bb_row.yaml"
122+
bash bench/vs_bb/run.sh --cljw "$RUNNER_TEMP/cljw-bench/bin/cljw" -n 3 --history "$hist" | tee "$RUNNER_TEMP/vs_bb_table.txt"
123+
{
124+
echo "## cljw vs bb vs JVM (${GITHUB_SHA::8})"
125+
echo
126+
echo "Non-blocking report. Ratios < 1.00 mean cljw is faster. Hosted runner: compare ratios, not ms."
127+
echo
128+
echo '```'
129+
cat "$RUNNER_TEMP/vs_bb_table.txt"
130+
echo '```'
131+
echo
132+
echo "<details><summary>YAML history row (append to bench/vs_bb/history.yaml)</summary>"
133+
echo
134+
echo '```yaml'
135+
cat "$hist"
136+
echo '```'
137+
echo "</details>"
138+
} >> "$GITHUB_STEP_SUMMARY"

‎.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,3 +59,6 @@ __pycache__/
5959
# built on demand from jsonsum.go (docs/examples/polyglot/README.md), never
6060
# committed. The C, Zig and Rust guests are a few hundred bytes and are.
6161
docs/examples/polyglot/go/jsonsum.wasm
62+
63+
# Per-worktree ling scratch and resume notes (never committed)
64+
.scratch/

‎CHANGELOG.md‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,87 @@ first stable `1.0.0` tag; pre-1.0 `alpha` / `rc` tags may still change surfaces.
77

88
## [Unreleased]
99

10+
## [1.15.4] - 2026-10-23
11+
12+
### Added
13+
14+
- **`java.time.format`: `DateTimeFormatter`, `DateTimeFormatterBuilder`,
15+
`ChronoField`, `ZoneId` and `ZoneOffset`.** Pattern letters
16+
`G u y M L d D F E a h K k H m s S n A N V X x Z O`, optional sections,
17+
SMART and STRICT resolvers and the 12 `ISO_*` constants. Letters cljw does
18+
not implement raise `not_implemented` instead of formatting wrong text. This
19+
was the last blocker for `malli.transform`.
20+
- **Primitive arrays cross `cljw.ffi` and wasm memory at memcpy speed
21+
(ADR-0204).** `byte`/`short`/`int`/`long`/`float`/`double`/`char`/`boolean`
22+
arrays hold raw native elements. A `:pointer` argument passes the array's
23+
storage with no copy, `:bytes` is one memcpy, `wasm/mem-write!` accepts a
24+
primitive array and the new `wasm/mem-read-array` returns one. `bytes?` is
25+
true only for a byte array.
26+
- **`bench/vs_bb`:** 14 workloads that compare cljw with bb and JVM Clojure,
27+
with a non-blocking CI report.
28+
- **`docs/examples/zig-native`:** one Zig source used both as a wasm guest and
29+
as a `cljw.ffi` library.
30+
31+
### Changed
32+
33+
- **Core sequence functions are much faster (ADR-0205).** The seq of a vector
34+
is now chunked by leaf, as clj's `PersistentVector$ChunkedSeq` is, and
35+
`map`, `filter`, `into`, `mapv` and `reduce` go through one chunked-seq
36+
boundary and the Zig `reduce`. On an 8000-element vector,
37+
`(reduce + (map inc v))` went from 19271 us to 718 us and
38+
`(count (filter even? v))` from 19646 us to 490 us. `(first (map f v))` now
39+
realizes 32 elements, as clj does.
40+
- **Protocol calls hit a per-function dispatch cache (ADR-0206).** Each
41+
protocol function remembers the method it resolved for the last receiver
42+
type; every `extend`, `extend-type`, `extend-protocol` and type redefinition
43+
invalidates it.
44+
- **A Clojure fn call no longer memsets its scratch buffers.** ReleaseSafe
45+
filled about 2 KB of locals with 0xAA on every call. Skipping it cuts 18%
46+
of instructions from `seq_dot`, 38% from `lazy_chain`, 35% from multimethod
47+
dispatch and 10% from startup.
48+
- **String and vector hot paths:** `clojure.string/join`, `upper-case`,
49+
`lower-case` and `split`, `subvec`, `str`, keyword interning and
50+
`dorun`/`doall` take native fast paths. The `string_ops` bench went from
51+
9.3x bb to 2.3x bb.
52+
- **Cold start does less work.** Deserializing the core image no longer maps
53+
a page per quoted list. `cljw -e nil` takes 27.3 ms of CPU, against 38.8 ms
54+
before and 29.1 ms for bb.
55+
- **The GC returns freed memory and reuses it.** Small cells live in 64 KiB
56+
pages that are unmapped once empty, and String, array, transient-vector and
57+
BigInt payload bytes now count toward the collect trigger, so long-lived
58+
processes no longer hold their high-water mark.
59+
60+
### Fixed
61+
62+
- **`clojure.edn` reads the EDN dialect.** It used the permissive Clojure
63+
reader and accepted EDN-invalid input; it now matches JVM Clojure's
64+
`EdnReader` on 59 of 59 cases.
65+
- **wasm `i64` arguments and results cross exactly over the full range.**
66+
Values past 2^47 came back as a lossy Double or were refused, and an
67+
integral float of 2^63 crashed the host.
68+
- **A BigInt, Ratio or BigDecimal past about 1233 digits prints its digits.**
69+
It printed the literal `(BigInt)`.
70+
- **A long-running server that reads stdin now gives memory back.** The
71+
`*in*` and `System/in` readers kept every byte they had ever read; they now
72+
drop what has been consumed. While the main thread waits on stdin, each idle
73+
second runs a collect (when at least 1 MiB was allocated since the last one)
74+
and returns freed memory to the OS, so garbage left by `future`s and worker
75+
threads no longer waits for the next request. Socket and `StringBuilder`
76+
buffers now count toward the collect trigger. A bb-mcp-shaped probe (20
77+
`future`s each building 4 MB of JSON, then idle) settles at 129 MB instead
78+
of holding 955 MB.
79+
80+
- **A `wasm/call` on the JIT no longer costs time proportional to the
81+
module.** zwasm resolved the export by re-parsing the module on every call;
82+
it now reads the export table built at compile. A trivial export of a 47 KB
83+
module went from about 5.9 us to 0.8 us a call, the same as a tiny module.
84+
`bench/wasm_percall.clj` gains `big-*` cells that measure it.
85+
- **A wasm handle called from a `future` no longer traps on the interpreter.**
86+
zwasm's interpreter cached the first calling thread's native-stack limit on
87+
the instance, so a handle warmed on the runtime thread and then called from
88+
another thread whose stack lay below that limit trapped
89+
`CallStackExhausted`. The limit is now per calling thread (zwasm 79f3d63a).
90+
1091
## [1.15.3] - 2026-10-22
1192

1293
### Added

‎bench/ffi_prim_arrays.clj‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
;; SPDX-License-Identifier: EPL-2.0
2+
;; Kanban CLJW-FFI-PRIM-ARRAY-BYTES (ADR-0204): what a 1000-element f64 dot
3+
;; costs through cljw.ffi, call only versus building its arguments.
4+
;;
5+
;; cc -shared -fPIC -O2 -o /tmp/libdot.so test/e2e/fixtures/ffi/ffi_fixture.c
6+
;; cljw bench/ffi_prim_arrays.clj /tmp/libdot.so
7+
;;
8+
;; Prints microseconds per call, best of 5 runs. `:pointer` passes a
9+
;; double-array's storage (only where arrays are typed); `:bytes` packs the
10+
;; doubles into a byte array first, the only route before ADR-0204.
11+
(require '[cljw.ffi :as ffi])
12+
13+
(def lib (ffi/open (first *command-line-args*)))
14+
(def n 1000)
15+
(def xs (vec (range n)))
16+
17+
(defn- best-us [iters f]
18+
(apply min (for [_ (range 5)]
19+
(let [t0 (System/nanoTime)]
20+
(dotimes [_ iters] (f))
21+
(/ (- (System/nanoTime) t0) 1000.0 iters)))))
22+
23+
(defn- f64s->bytes [ds]
24+
(byte-array
25+
(mapcat (fn [d] (let [b (Double/doubleToRawLongBits d)]
26+
(for [i (range 8)] (unchecked-byte (bit-shift-right b (* 8 i))))))
27+
ds)))
28+
29+
(def dot-bytes (ffi/function lib "dot" [:bytes :bytes :long] :double))
30+
(let [b (f64s->bytes xs)]
31+
(println "bytes-call-only-us" (best-us 2000 #(dot-bytes b b n)))
32+
(println "bytes-pack-and-call-us" (best-us 5 #(let [b (f64s->bytes xs)] (dot-bytes b b n)))))
33+
34+
(when (try (ffi/function lib "dot" [:pointer :pointer :long] :double)
35+
((ffi/function lib "dot" [:pointer :pointer :long] :double) (double-array 1) (double-array 1) 1)
36+
true
37+
(catch Exception _ false))
38+
(let [dot (ffi/function lib "dot" [:pointer :pointer :long] :double)
39+
a (double-array xs)]
40+
(println "pointer-call-only-us" (best-us 2000 #(dot a a n)))
41+
(println "pointer-build-and-call-us" (best-us 2000 #(let [a (double-array xs)] (dot a a n))))
42+
(println "check" (dot a a n) (reduce + (map * xs xs)))))
43+
44+
(println "clojure-dot-us" (best-us 20 #(reduce + (map * xs xs))))

‎bench/gc_heap_return.clj‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
;; CLJW-GC-HEAP-RETURN acceptance probe. A bench, not a test: RSS is
2+
;; host-dependent. Steady-state bursts of one size must not grow the heap
3+
;; after the first, and a major collection must hand freed memory back to
4+
;; the OS. Run: cljw bench/gc_heap_return.clj
5+
(require (quote clojure.data.json) (quote clojure.string))
6+
(defn rss-mb [] (let [l (first (filter #(.startsWith ^String % "VmRSS") (clojure.string/split-lines (slurp "/proc/self/status"))))]
7+
(quot (Long/parseLong (re-find #"\d+" l)) 1024)))
8+
(println "start" (rss-mb) "MB")
9+
(def big (atom (vec (range 3000000))))
10+
(println "after 3M vector" (rss-mb) "MB")
11+
(reset! big nil)
12+
(System/gc) (System/gc)
13+
(println "after drop + System/gc" (rss-mb) "MB")
14+
(dotimes [_ 5] (count (vec (range 3000000))))
15+
(println "after 5 more 3M bursts" (rss-mb) "MB")
16+
(dotimes [i 200] (count (clojure.data.json/write-str (vec (range 20000)))))
17+
(System/gc)
18+
(println "after 200 x 20k JSON writes" (rss-mb) "MB")

0 commit comments

Comments
 (0)