Skip to content

Commit 6dab7f9

Browse files
committed
chore: small optimizations and more charts (warpo)
1 parent 280fa84 commit 6dab7f9

19 files changed

Lines changed: 1121 additions & 60 deletions

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,9 @@ as-tral
1111
.vscode/
1212
build/
1313
*.tmp.ts
14+
# Generated, payload-embedding bench sources (gen-warp-bench.mjs / gen-warpo-bench.mjs).
15+
# This dir holds only generated files (*.warp.ts, *.warpo.src.ts, *.tmp.ts).
16+
assembly/__benches__/runtimes/*.ts
1417
.as-test/*
1518
!.as-test/runners/
1619
!.as-test/runners/**

README.md

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -604,6 +604,48 @@ How `json-as` stacks up against other JSON libraries on a ~5 KiB GitHub-repo pay
604604

605605
<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/library-serialize.svg" alt="Library comparison - serialize throughput">
606606

607+
### Runtime Comparison
608+
609+
How fast the **same** `json-as` classic bench deserializes the minified payloads across six WebAssembly runtimes — including [WARP](https://github.com/wasm-ecosystem/wasm-compiler), a single-pass compiler built for embedded targets, alongside the optimizing JITs (Wasmtime, WAVM), the pure-Go wazero, and JS engines (V8, Bun).
610+
611+
Each runtime runs the real `bench()` lib (warm up, time the loop, report MB/s itself) on a `NAIVE`-mode build with a shared feature set (no SIMD / bulk-memory / non-trapping float-to-int) so the executed code is equivalent. The WASI runtimes read the payload over WASI; V8/Bun via an `env`-ABI host. WARP has no WASI and — by design ("no recursions") — can't re-enter the module from a host import, so it runs through a small custom C++ host that links `performance.now`/`console.log`/`writeFile` with the payload embedded. The timed run is split into small frames with a full GC between them (the bench lib's `BENCH_FRAMES`, applied to every runtime so the measurement is identical); this excludes stop-the-world GC pauses and keeps WARP — an embedded, single-shot-oriented compiler — inside its stable envelope. Even so, WARP's single-pass codegen lands within a few percent of the optimizing JITs.
612+
613+
<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/runtimes-deserialize.svg" alt="Deserialization throughput across WebAssembly runtimes">
614+
615+
<details>
616+
<summary>Serialization throughput (click to expand)</summary>
617+
618+
<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/runtimes-serialize.svg" alt="Serialization throughput across WebAssembly runtimes">
619+
</details>
620+
621+
Reproduce locally (the JS engines and standalone runtimes are auto-detected; point `WARP_SRC` at a [wasm-ecosystem/wasm-compiler](https://github.com/wasm-ecosystem/wasm-compiler) checkout with its libs built — see the script header for the cmake flags):
622+
623+
```bash
624+
WARP_SRC=/path/to/wasm-compiler npm run bench:runtimes
625+
npm run charts:runtimes
626+
```
627+
628+
### Compiler Comparison
629+
630+
A different axis: same source, same runtime (V8), different **compiler**. [warpo](https://github.com/wasm-ecosystem/warpo) is a next-generation AssemblyScript compiler; this compares `json-as` when built with stock `asc` vs `warpo`, each before and after a `wasm-opt -O4` pass.
631+
632+
warpo produces consistently faster code on both directions, and the gap widens with number/structure density (Canada). `wasm-opt -O4` barely moves either (both already run binaryen optimizations), so the difference is the compilers' own codegen, not a missing post-pass.
633+
634+
<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/warpo-vs-asc-deserialize.svg" alt="json-as deserialization: asc vs warpo compiler on V8">
635+
636+
<details>
637+
<summary>Serialization (click to expand)</summary>
638+
639+
<img src="https://raw.githubusercontent.com/JairusSW/json-as/refs/heads/docs/charts/v1.5.0/warpo-vs-asc-serialize.svg" alt="json-as serialization: asc vs warpo compiler on V8">
640+
</details>
641+
642+
warpo can't run asc transform plugins, so the bench compiles the post-transform `.tmp.ts` (emitted via `JSON_WRITE`); see `scripts/run-bench.warpo.sh`. Reproduce:
643+
644+
```bash
645+
npm run bench:warpo # builds with asc + warpo, wasm-opt -O4 each, runs on v8
646+
npm run charts:warpo
647+
```
648+
607649
### Lazy Fields
608650

609651
Mark a field `@lazy` (or `JSON.Lazy<T>`, or a whole class with `@json({ lazy: "auto" })`) to defer it: its raw JSON slice is stored at parse time and parsed only on first access. Fields you skip are never parsed, and untouched fields pass through their original bytes on serialize. See the [Lazy Fields guide](https://docs.jairus.dev/json-as/guide/lazy-fields) for the full API and trade-offs.

as-test.config.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66
"coverageDir": "./.as-test/coverage",
77
"snapshotDir": "./.as-test/snapshots",
88
"config": "none",
9+
"features": ["try-as"],
910
"cache": {
1011
"type": "reachable",
1112
"ttl": "1d"

assembly/__benches__/lib/bench.ts

Lines changed: 28 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -184,19 +184,36 @@ export function bench(
184184
routine();
185185
}
186186

187-
const start = performance.now();
188-
189-
let count = ops;
190-
while (count--) {
191-
routine();
192-
if (BENCH_MEMORY) {
193-
const p = u64(memory.size());
194-
if (p > peakPages) peakPages = p;
187+
// Optional framed measurement: split the timed run into BENCH_FRAMES frames
188+
// and run a full __collect() (untimed) between them. Each frame's allocation
189+
// churn stays bounded and the heap is reset between frames, which keeps
190+
// runtimes that destabilize under one long single-shot allocation loop (e.g.
191+
// WARP) inside their safe envelope while still timing the same total ops. The
192+
// between-frame GC pauses are excluded from `elapsed`; the incremental GC that
193+
// runs *within* each frame is still timed, exactly as in the unframed loop.
194+
// Defaults to a single frame (identical to the original behavior).
195+
// @ts-expect-error: BENCH_FRAMES may be undefined.
196+
const frames: u64 = isDefined(BENCH_FRAMES) ? u64(BENCH_FRAMES) : 1;
197+
const perFrame: u64 = frames > 1 ? ops / frames : ops;
198+
199+
let elapsed: f64 = 0;
200+
let remaining = ops;
201+
while (remaining > 0) {
202+
const batch: u64 = remaining < perFrame ? remaining : perFrame;
203+
if (frames > 1) __collect();
204+
const frameStart = performance.now();
205+
let count = batch;
206+
while (count--) {
207+
routine();
208+
if (BENCH_MEMORY) {
209+
const p = u64(memory.size());
210+
if (p > peakPages) peakPages = p;
211+
}
195212
}
213+
elapsed += performance.now() - frameStart;
214+
remaining -= batch;
196215
}
197-
198-
const end = performance.now();
199-
const elapsed = Math.max(1, end - start);
216+
elapsed = Math.max(1, elapsed);
200217

201218
let retainedLiveBytes: u64 = 0;
202219
let inflightLiveBytes: u64 = 0;

assembly/deserialize/naive/staticarray.ts

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,8 @@ function materializeStaticArray<T extends StaticArray<any>>(
2121
src: valueof<T>[],
2222
dst: usize,
2323
): T {
24-
const byteLength = <usize>src.length * sizeof<valueof<T>>();
24+
const len = src.length;
25+
const byteLength = <usize>len * sizeof<valueof<T>>();
2526
let out = dst;
2627

2728
if (!out) {
@@ -31,7 +32,7 @@ function materializeStaticArray<T extends StaticArray<any>>(
3132
}
3233

3334
const typed = changetype<T>(out);
34-
for (let i = 0; i < src.length; i++) {
35+
for (let i = 0; i < len; i++) {
3536
unchecked((typed[i] = unchecked(src[i])));
3637
}
3738
return typed;

assembly/index.ts

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1721,7 +1721,8 @@ export namespace JSON {
17211721

17221722
const keys = value.keys();
17231723
const values = value.values();
1724-
for (let i = 0; i < keys.length; i++) {
1724+
const len = keys.length;
1725+
for (let i = 0; i < len; i++) {
17251726
out.set(unchecked(keys[i]), unchecked(values[i]));
17261727
}
17271728
return out;
@@ -1921,7 +1922,8 @@ export namespace JSON {
19211922
const out = new JSON.Arr();
19221923
// @ts-expect-error: T is JSON.Value[] here
19231924
const arr = changetype<JSON.Value[]>(value);
1924-
for (let i = 0; i < arr.length; i++) {
1925+
const len = arr.length;
1926+
for (let i = 0; i < len; i++) {
19251927
out.pushRawSlot(JSON.Value.bitsFrom<JSON.Value>(unchecked(arr[i])));
19261928
}
19271929
return out;

bench/runners/runtimes-env.mjs

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
// env-ABI host for the runtime comparison, used to run the real json-as classic
2+
// benches under v8 and bun. It supplies the `env` imports the bench lib needs
3+
// (readFile / writeFile / console.log / performance.now / Date.now / abort) and
4+
// calls the exported `start`, so the bench self-measures exactly as it does
5+
// under the WASI runtimes - only the host ABI differs. writeFile is rerouted to
6+
// stdout as an __AS_BENCH_JSON__ line so run-bench.runtimes.sh captures results
7+
// uniformly across every runtime.
8+
//
9+
// v8: v8 --module bench/runners/runtimes-env.mjs -- <wasm>
10+
// bun: bun bench/runners/runtimes-env.mjs <wasm>
11+
const isV8 = typeof readbuffer === "function";
12+
const print =
13+
typeof globalThis.print === "function" ? globalThis.print : console.log;
14+
15+
let wasmPath, fs;
16+
if (isV8) {
17+
wasmPath = arguments[0];
18+
} else {
19+
fs = await import("node:fs");
20+
wasmPath = Bun.argv[2];
21+
}
22+
23+
const bytes = isV8
24+
? new Uint8Array(readbuffer(wasmPath))
25+
: new Uint8Array(fs.readFileSync(wasmPath));
26+
27+
const ARRAYBUFFER_ID = 1;
28+
let memory = null;
29+
30+
const { exports } = new WebAssembly.Instance(new WebAssembly.Module(bytes), {
31+
env: {
32+
abort: (msg, file, line, col) => {
33+
print(`abort: ${liftString(msg)} in ${liftString(file)}:${line}:${col}`);
34+
throw new Error("aborted");
35+
},
36+
"console.log": (ptr) => print(liftString(ptr)),
37+
"Date.now": () => Date.now(),
38+
"performance.now": () => performance.now(),
39+
// The bench's dumpToFile writes results via writeFile; reroute to stdout so
40+
// the shell captures them the same way it does the WASI runtimes' stdout.
41+
writeFile: (namePtr, dataPtr) =>
42+
print(`__AS_BENCH_JSON__${liftString(namePtr)}\t${liftString(dataPtr)}`),
43+
readFile: (pathPtr) => {
44+
const path = liftString(pathPtr);
45+
const data = isV8 ? readbuffer(path) : fs.readFileSync(path);
46+
return lowerBuffer(new Uint8Array(data));
47+
},
48+
},
49+
});
50+
51+
memory = exports.memory;
52+
exports.start();
53+
54+
function liftString(pointer) {
55+
if (!pointer) return null;
56+
const end =
57+
(pointer + new Uint32Array(memory.buffer)[(pointer - 4) >>> 2]) >>> 1;
58+
const memoryU16 = new Uint16Array(memory.buffer);
59+
let start = pointer >>> 1;
60+
let string = "";
61+
while (end - start > 1024)
62+
string += String.fromCharCode(
63+
...memoryU16.subarray(start, (start += 1024)),
64+
);
65+
return string + String.fromCharCode(...memoryU16.subarray(start, end));
66+
}
67+
68+
function lowerBuffer(value) {
69+
if (value == null) return 0;
70+
const pointer = exports.__new(value.byteLength, ARRAYBUFFER_ID) >>> 0;
71+
new Uint8Array(memory.buffer).set(value, pointer);
72+
return pointer;
73+
}

bench/runners/warp_host.cpp

Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,143 @@
1+
// Custom WARP (wasm-ecosystem/wasm-compiler) host for the cross-runtime
2+
// benchmark. Unlike WARP's stock `vb_bench` (which links no imports and times an
3+
// export externally), this host links the `env` functions the json-as bench lib
4+
// needs - performance.now / Date.now / console.log / writeFile / abort - so WARP
5+
// runs the *real* bench() loop and self-measures exactly like every other
6+
// runtime, emitting the same `__AS_BENCH_JSON__` result lines.
7+
//
8+
// WARP has no WASI and - by design ("no recursions", static execution context) -
9+
// cannot re-enter the module from inside a host import, so readFile (which would
10+
// have to call the wasm __new allocator) is impossible. The WARP bench build
11+
// therefore embeds its payload instead of reading a file; the measured
12+
// deserialize/serialize work is identical. WARP also destabilizes under one long
13+
// single-shot allocation loop, so run-bench.runtimes.sh builds with BENCH_FRAMES
14+
// (the bench lib splits the timed run into small GC-separated frames).
15+
//
16+
// Build (links the static libs from a WARP build tree, see run-bench.runtimes.sh):
17+
// g++ -std=gnu++14 -O2 -DJIT_TARGET_X86_64 -DINTERRUPTION_REQUEST=0 \
18+
// -DEAGER_ALLOCATION=1 -I"$WARP_SRC" warp_host.cpp \
19+
// -Wl,--start-group <libWasmModule|libcompiler|libruntime|libutils|lib_core_common>.a \
20+
// -Wl,--end-group -lpthread -o warp_host
21+
//
22+
// Usage: warp_host <module.wasm>
23+
// Prints whatever the bench writes (progress + __AS_BENCH_JSON__<path>\t<json>).
24+
#include <chrono>
25+
#include <cstdint>
26+
#include <cstdio>
27+
#include <cstdlib>
28+
#include <cstring>
29+
#include <string>
30+
#include <vector>
31+
32+
#include "src/WasmModule/WasmModule.hpp"
33+
#include "src/core/common/NativeSymbol.hpp"
34+
#include "src/core/common/function_traits.hpp"
35+
#include "src/utils/STDCompilerLogger.hpp"
36+
37+
using namespace vb;
38+
using Clock = std::chrono::high_resolution_clock;
39+
40+
namespace {
41+
WasmModule *g_module = nullptr;
42+
Clock::time_point g_epoch;
43+
44+
// Reads an AssemblyScript string (UTF-16LE, byte-length stored as the u32 at
45+
// ptr-4) out of linear memory and returns it as UTF-8.
46+
std::string liftString(uint32_t ptr) {
47+
if (ptr == 0U || g_module == nullptr) return std::string();
48+
uint8_t const *lenField = g_module->getLinearMemoryRegion(ptr - 4U, 4U);
49+
uint32_t byteLen = 0U;
50+
std::memcpy(&byteLen, lenField, 4U);
51+
if (byteLen == 0U) return std::string();
52+
uint8_t const *data = g_module->getLinearMemoryRegion(ptr, byteLen);
53+
std::string out;
54+
out.reserve(byteLen / 2U);
55+
for (uint32_t i = 0U; i + 1U < byteLen; i += 2U) {
56+
uint32_t cu = static_cast<uint32_t>(data[i]) | (static_cast<uint32_t>(data[i + 1U]) << 8);
57+
// Minimal UTF-16 -> UTF-8 (the bench's strings are ASCII JSON + log text;
58+
// surrogate pairs are passed through per-unit, which is fine for output).
59+
if (cu < 0x80U) {
60+
out.push_back(static_cast<char>(cu));
61+
} else if (cu < 0x800U) {
62+
out.push_back(static_cast<char>(0xC0U | (cu >> 6)));
63+
out.push_back(static_cast<char>(0x80U | (cu & 0x3FU)));
64+
} else {
65+
out.push_back(static_cast<char>(0xE0U | (cu >> 12)));
66+
out.push_back(static_cast<char>(0x80U | ((cu >> 6) & 0x3FU)));
67+
out.push_back(static_cast<char>(0x80U | (cu & 0x3FU)));
68+
}
69+
}
70+
return out;
71+
}
72+
73+
// --- env imports the bench lib calls (none re-enter the module) -------------
74+
double host_performance_now(void *) noexcept {
75+
return std::chrono::duration<double, std::milli>(Clock::now() - g_epoch).count();
76+
}
77+
double host_date_now(void *) noexcept {
78+
return std::chrono::duration<double, std::milli>(Clock::now().time_since_epoch()).count();
79+
}
80+
void host_console_log(uint32_t ptr, void *) noexcept { printf("%s\n", liftString(ptr).c_str()); }
81+
82+
// dumpToFile() calls writeFile(path, json). The env build's path is
83+
// ./build/logs/as/<mode>/<suite>.<type>.as.json; re-emit it as an
84+
// __AS_BENCH_JSON__ line so run-bench.runtimes.sh routes it to runtimes/warp/.
85+
void host_write_file(uint32_t namePtr, uint32_t dataPtr, void *) noexcept {
86+
printf("__AS_BENCH_JSON__%s\t%s\n", liftString(namePtr).c_str(), liftString(dataPtr).c_str());
87+
}
88+
void host_abort(uint32_t msg, uint32_t file, uint32_t line, uint32_t col, void *) noexcept {
89+
printf("abort: %s in %s:%u:%u\n", liftString(msg).c_str(), liftString(file).c_str(), line, col);
90+
std::exit(1);
91+
}
92+
93+
std::vector<uint8_t> loadFile(char const *path) {
94+
FILE *f = fopen(path, "rb");
95+
if (f == nullptr) {
96+
fprintf(stderr, "warp_host: cannot open %s\n", path);
97+
std::exit(1);
98+
}
99+
fseek(f, 0, SEEK_END);
100+
long n = ftell(f);
101+
rewind(f);
102+
std::vector<uint8_t> buf(static_cast<size_t>(n));
103+
size_t rd = fread(buf.data(), 1U, buf.size(), f);
104+
(void)rd;
105+
fclose(f);
106+
return buf;
107+
}
108+
} // namespace
109+
110+
int main(int argc, char **argv) {
111+
if (argc < 2) {
112+
fprintf(stderr, "usage: warp_host <module.wasm>\n");
113+
return 1;
114+
}
115+
std::vector<uint8_t> bytecode = loadFile(argv[1]);
116+
g_epoch = Clock::now();
117+
118+
WasmModule::initEnvironment(&malloc, &realloc, &free);
119+
STDCompilerLogger logger{};
120+
WasmModule module(UINT64_MAX, logger, false, nullptr, 0U);
121+
g_module = &module;
122+
123+
// V1 imports (statically linked at compile time -> pass to compile(), and an
124+
// EMPTY span to initFromCompiledBinary, which rejects STATIC symbols).
125+
auto imports = make_array(
126+
STATIC_LINK("env", "performance.now", host_performance_now),
127+
STATIC_LINK("env", "Date.now", host_date_now),
128+
STATIC_LINK("env", "console.log", host_console_log),
129+
STATIC_LINK("env", "writeFile", host_write_file),
130+
STATIC_LINK("env", "abort", host_abort));
131+
Span<NativeSymbol const> importSpan(imports.data(), imports.size());
132+
133+
try {
134+
WasmModule::CompileResult compiled{module.compile(
135+
Span<uint8_t const>(bytecode.data(), static_cast<uint32_t>(bytecode.size())), importSpan)};
136+
module.initFromCompiledBinary(compiled.getModule().span(), Span<NativeSymbol const>(), Span<uint8_t const>());
137+
module.start(nullptr); // runs the bench (all work happens in the start section)
138+
} catch (std::exception const &e) {
139+
fprintf(stderr, "warp_host: %s\n", e.what());
140+
return 1;
141+
}
142+
return 0;
143+
}

0 commit comments

Comments
 (0)