You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
elixir/lib/printable_binary.ex = Elixir ~PB compile-time sigil: decodes printable-binary glyphs to raw bytes at compile time (reads character_map.txt via @external_resource); decode/1 runtime helper (whitespace-tolerant)
7
9
elixir/mix.exs = Mix project for the ~PB sigil demo (no deps)
@@ -15,5 +17,6 @@ annotate=[
15
17
test/module_consumer.zig = Test fixture: minimal downstream importer of the printable_binary Zig module (mirrors how difz/blip consume it) for the FFI-symbol-leak test
test/test_module_no_ffi_symbols = Regression test (nm oracle): importing the module must emit 0 pb_* symbols while the FFI static lib keeps all 12 — guards the difz-blocking duplicate-symbol bug
A comprehensive guide to the LuaJIT, C, and Zigimplementations of PrintableBinary - a tool for encoding binary data into human-readable UTF-8 strings and decoding them back.
3
+
A guide to the LuaJIT, C, Cosmopolitan APE, Zig, Rust, JavaScript/Node, and WebAssembly implementations of PrintableBinary—a tool for encoding binary data into human-readable UTF-8 and decoding it back.
4
4
5
5
## Overview
6
6
@@ -18,36 +18,53 @@ PrintableBinary is available in multiple high-performance implementations:
18
18
-**CLI parity**: Same flags, environment variables, and character map behavior as the ELF build
19
19
-**Great for distribution**: Ship one file (`printable-binary-ape.com`) and it just works
-**Memory-safe**: Zig's safety features catch bugs at compile time and runtime
23
29
-**Cross-compilation**: Easy cross-compilation to many platforms from a single host
24
30
-**Fast compilation**: Incremental builds and fast compile times
25
31
-**CLI parity**: Same flags and behavior as C implementation
26
32
33
+
### 🦀 **Rust Implementation** (Embedded Raw Codec)
34
+
-**Byte-identical core**: Compile-time tables generated from `character_map.txt`
35
+
-**Low-allocation API**: `encode_into` and `decode_into` reuse caller-owned buffers
36
+
-**Transport-focused CLI**: Minimal stdin→stdout encode/decode path for Rust↔Zig integration
37
+
-**Deliberately narrow surface**: Formatting, containers, runtime map overrides, and mapping reports remain the job of the full CLIs
38
+
39
+
### 🌐 **JavaScript / Node.js Implementation** (Web and Automation)
40
+
-**Shared core**: One module powers the browser UI and Node CLI
41
+
-**Portable runtime**: Runs in browsers, Node.js, and compatible bundlers
42
+
-**CLI parity**: Node exposes the full user-facing command surface
43
+
27
44
### ⚡ **LuaJIT Implementation** (Original)
28
45
-**Reference implementation**: Easy to modify and extend
29
46
-**Well-tested**: Extensive test suite and battle-tested
30
47
-**Development-friendly**: Rapid prototyping and debugging
31
48
32
49
## Performance Comparison
33
50
34
-
### Benchmark Results (Apple M4 Max)
51
+
Performance depends on input mix, CPU, filesystem, runtime version, and whether a runtime must start. Rather than preserve a partial historical C-vs-Lua table as a project-wide claim, run the reproducible comparison locally:
35
52
36
-
| File Size | Operation | LuaJIT Time | C Time | C Speedup | C Improvement |
| Implementation | Included in comparison runner | Measurement boundary |
61
+
| --- | --- | --- |
62
+
| C (native), Zig, LuaJIT, Node.js | Yes, direct when built | CLI process and file I/O |
63
+
| APE | Yes, through a clean-environment adapter | Cosmopolitan process and file I/O |
64
+
| Rust | Yes, through an stdin adapter | Rust process and stdin/stdout I/O |
65
+
| WebAssembly | Yes, through wazero | WASM runtime startup plus CLI I/O |
46
66
47
-
-**Overall Encoding**: C is 1.14x faster (12.2% improvement)
48
-
-**Overall Decoding**: C is 2.08x faster (52.0% improvement)
49
-
-**Large Files**: Up to **6x speedup** for 1MB+ binary/random data decoding
50
-
-**Memory Usage**: C implementation uses significantly less memory
67
+
`--impls rust,wasm,ape` makes those three targets mandatory, so a missing artifact fails instead of producing a misleading partial result. The runner round-trips a deterministic random input before it invokes `hyperfine`; it also includes an ASCII input set. Rust's `cargo run --release --example bench` is a separate in-process codec microbenchmark and should not be compared directly with CLI timings.
51
68
52
69
## Quick Start
53
70
@@ -80,6 +97,19 @@ make ape
80
97
make test-ape
81
98
```
82
99
100
+
### WebAssembly Implementation
101
+
102
+
```bash
103
+
# Build the standalone WASM CLI from the C core.
104
+
./build wasm
105
+
106
+
# Run it through the WASI runtime supplied by the development shell.
107
+
nix develop -c wazero run bin/printable-binary.wasm < input.bin > encoded.pbt
WASM has the C CLI's feature set. The runtime does not automatically inherit host environment variables; pass `--env=NAME=value` to wazero when a variable such as `PRINTABLE_BINARY_MUTE_STATS` matters.
# Measure the in-process codec alone (not directly comparable to CLI timings).
135
+
nix develop -c cargo run --release --manifest-path rust/Cargo.toml --example bench
136
+
```
137
+
138
+
Use Rust as an embedded codec or a Rust↔Zig transport boundary. Its CLI currently supports only raw stdin encode and `-d`/`--decode`; use a full CLI for file arguments, format options, mapping reports, containers, or runtime map overrides.
All compiled variants (ELF, APE, WASM via wazero) share **identical**command-line interfaces. Use whichever binary suits your platform (`./bin/printable-binary-c`, `./bin/printable-binary-ape.com`, etc.).
197
+
The full compiled variants (native C, APE, Zig, and WASM via wazero) share the user-facing command-line interface. The Rust transport CLI is intentionally narrower: raw stdin encode plus `-d`/`--decode`.
150
198
151
199
### Basic Operations
152
200
@@ -205,13 +253,15 @@ Input/Output:
205
253
All binaries embed the canonical 256-entry map, so the `--mappings*` flags work even when `character_map.txt` is missing. If you place a custom map alongside the executable (or set `PRINTABLE_BINARY_MAP`), these options will reflect the override automatically. The override file should contain **exactly 256 lines**, each a single UTF-8 glyph (line 0 = byte 0x00, line 255 = byte 0xFF).
206
254
```
207
255
208
-
### Environment Variables (All Implementations)
256
+
### Environment Variables (Full CLIs)
209
257
210
258
```
211
259
PRINTABLE_BINARY_MAP – points to an alternate character_map.txt
212
260
PRINTABLE_BINARY_MUTE_STATS – set to 1/true/yes to suppress stderr statistics
213
261
```
214
262
263
+
The Rust crate consumes its map at compile time and has no runtime map/environment configuration. When invoking WASM, pass variables explicitly with wazero's `--env` option because WASI does not inherit the host environment by default.
264
+
215
265
## When to Use Which Implementation
216
266
217
267
### Use C Implementation For:
@@ -228,10 +278,23 @@ PRINTABLE_BINARY_MUTE_STATS – set to 1/true/yes to suppress stderr statistics
228
278
229
279
✅ **Cross-compilation** to other platforms
230
280
✅ **Memory-safe production** where safety is paramount
231
-
✅ **WebAssembly targets**(future capability)
281
+
✅ **A safe native CLI**with the complete option surface
232
282
✅ **Modern tooling** with built-in package manager
233
283
✅ **Environments** where C toolchains are unavailable
234
284
285
+
### Use APE or WebAssembly For:
286
+
287
+
✅ **APE** — a single distributable executable across Linux, macOS, and Windows
288
+
✅ **WebAssembly** — a sandboxed/WASI deployment where a native binary is unsuitable
289
+
✅ **Both** — the familiar full C CLI behavior; factor the runtime/loader into performance expectations
290
+
291
+
### Use Rust Implementation For:
292
+
293
+
✅ **Embedding** the byte↔glyph codec in Rust code
294
+
✅ **High-frequency transport** with reusable encode/decode buffers
295
+
✅ **Rust↔Zig boundaries** that need byte-identical raw codec output
296
+
✅ **A deliberately small raw stdin CLI**, not file/container/format workflows
297
+
235
298
### Use LuaJIT Implementation For:
236
299
237
300
✅ **Quick scripts** and one-off operations
@@ -242,44 +305,36 @@ PRINTABLE_BINARY_MUTE_STATS – set to 1/true/yes to suppress stderr statistics
242
305
243
306
## Feature Comparison
244
307
245
-
| Feature | LuaJIT | C | Zig | Notes |
246
-
|---------|--------|---|-----|-------|
247
-
|**Performance**| Fast |**Faster**|**Faster**| C/Zig are 1.1-6x faster |
248
-
|**Memory Usage**| Good |**Better**|**Better**| Native uses less memory |
@@ -545,12 +600,11 @@ When reporting issues, please include:
545
600
546
601
## Summary
547
602
548
-
PrintableBinary offers three excellent native implementations:
549
-
550
-
-**C Implementation**: Maximum performance for production use
551
-
-**Zig Implementation**: Memory-safe with easy cross-compilation
552
-
-**LuaJIT Implementation**: Maximum flexibility for development
603
+
PrintableBinary offers a byte-identical raw codec through several deliberate deployment choices:
553
604
554
-
All maintain perfect compatibility while offering different trade-offs. Choose based on your specific needs: performance-critical applications benefit from the C or Zig versions, while development and scripting scenarios may prefer the LuaJIT version.
605
+
-**Native C / Zig / LuaJIT / Node**: full CLIs for direct use
606
+
-**Cosmopolitan APE**: the full C CLI in one portable executable
607
+
-**WebAssembly**: the full C CLI under a WASI runtime
608
+
-**Rust**: an embedded, reusable-buffer raw codec and narrow stdin transport CLI
555
609
556
-
**🚀 For most users, we recommend the C or Zig implementation for superior performance and efficiency.**
610
+
Choose a full CLI for files, formatting, mappings, containers, and interactive workflows. Choose Rust when the raw codec belongs inside a Rust process or transport boundary. Use `bm/benchmark-zig-opt` on the deployment hardware before making a performance claim—its adapters make Rust, WASM, and APE first-class comparison targets.
0 commit comments