Re-vendor serialize at v1.15.0 - #336
Conversation
Nine minor versions of the optimization and hardening work land in the vendored header: the inlined write and read spines, word-wise WriteBytes, the precomputed compressed-float entry points, the contraction-invariance barrier that pins the compressed-float roundings in-source, the normative integer clamp, degenerate ranges, and the stricter malformed-string reader. Closes #334. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
serialize 1.15.0 pins the compressed-float roundings in-source with an optimization barrier, so the wire bytes no longer depend on the consumer's -ffp-contract setting. BUILDING.md and the CMake rationale now state that; -ffp-contract=off stays as standing policy (belt and braces, and the certification setting), and -ffast-math / -Ofast remain unsupported. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ct requires serialize 1.15.0 debug-asserts that serialize_string writers supply valid UTF-8 (the writer-trusted model). fuzz_connection_structured drives the real write path, so it must generate conforming content: string bytes are masked to ASCII, exactly as serialize's own fuzzer does. Arbitrary bytes still reach the reader's malformed-string refusal path via fuzz_connection's raw packets. Verified both ways: the unmasked harness reproduces the CI abort on the new assert; the masked one replays the corpus plus 20,000 random inputs under ASan+UBSan with asserts live, clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both version sites carry 1.11.0 (the CMake project version and the YOJIMBO_MAJOR/MINOR/PATCH_VERSION macros in include/yojimbo_config.h). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Two commits follow the differential work. First CI run went red on both per-PR fuzz legs: serialize 1.15.0 debug-asserts that serialize_string writers supply valid UTF-8 (the writer-trusted model), and fuzz_connection_structured was pushing arbitrary fuzzer bytes into the real write path. The fix mirrors serialize's own fuzzer exactly: write-side string bytes masked to ASCII, while arbitrary bytes still reach the reader's malformed-string refusal path via fuzz_connection's raw packets. Verified both directions locally — the unmasked harness reproduces the CI abort on the new assert, and the masked one replays the committed corpus plus 20,000 random inputs under ASan+UBSan with asserts live, clean. The version also moves to 1.11.0 at both sites (the CMake project version and the yojimbo_config.h macros), following the #335 pattern: bump in the PR, tag after merge. |
Enacts #334: the vendored serialize moves from 1.6.0 to v1.15.0, byte-for-byte as upstream ships it (
serialize/serialize.his byte-identical to the file at the upstreamv1.15.0tag — verified withcmp). Follows the existing vendoring mechanism: the single amalgamated header copied in unmodified.What nine minor versions bring
The optimization payload: inlined write and read spines, word-wise
WriteBytes(one flush, one bulk copy, one tail reload), and the precomputed compressed-float entry points. Plus the hardening work: the in-source contraction-invariance barrier (compressed-float wire bytes are now identical under every-ffp-contractsetting), the normative integer clamp, degenerate ranges (min == max), the stricter malformed-string reader, and serialize's own conformance battery riding along in the header's self-tests.Wire verdict: bits identical on every yojimbo path
Measured, not assumed — all differentials run with yojimbo's own flags (
-O2 -ffp-contract=off, arm64):tools/gen_seed_corpus_connectiondrives realConnection::GeneratePacketoutput over every committed seed scenario (both channel types, the full message vocabulary, single- and multi-fragment blocks). Regenerated under 1.6.0 and under 1.15.0: all 13 packets byte-identical to each other and to the committed corpus infuzz/corpus/fuzz_connection.tools/conformance/verify_standard.pystill fully decodes the corpus against STANDARD.md, 0 failures.-ffp-contract=fastagainst the old header produces different bytes on arm64 — proof the harness can discriminate the exact class of change under watch, so the identical result is a finding, not blindness.max_integer_valuein[2^23, 2^24)at the top of the range. Probed with an oddmax_integer_value(8,388,609): 1.6.0 wrote a code its own reader rejects (read_ok=0— broken, unreadable wire), 1.15.0 clamps to a code both old and new readers accept. So this is a fix of previously-unreadable output, not an interop break; matches upstream's changelog exactly. yojimbo itself has no such declaration. Upstream's golden wire vector is byte-identical between v1.6.0 and v1.15.0.Consequence for consumers: none. Old and new interoperate on every path; no wire-compat note is owed.
Tests
Full suite green before and after the swap:
./bin/test(ALL TESTS PASS, including the[serialize]section running serialize 1.15.0's embedded self-tests viaSERIALIZE_ENABLE_TESTS) andcustom_packet_io_test.Docs
BUILDING.mdand the CMake strict-FP rationale now describe the vendored header as it is: the barrier pins the compressed-float roundings in-source,-ffp-contract=offstays as standing policy (belt and braces, and the certification setting),-ffast-math/-Ofastremain unsupported.🤖 Generated with Claude Code