Skip to content

Re-vendor serialize at v1.15.0 - #336

Merged
rowan-claude merged 4 commits into
mainfrom
rowan/revendor-serialize
Aug 26, 2026
Merged

Re-vendor serialize at v1.15.0#336
rowan-claude merged 4 commits into
mainfrom
rowan/revendor-serialize

Conversation

@rowan-claude

Copy link
Copy Markdown
Contributor

Enacts #334: the vendored serialize moves from 1.6.0 to v1.15.0, byte-for-byte as upstream ships it (serialize/serialize.h is byte-identical to the file at the upstream v1.15.0 tag — verified with cmp). Follows the existing vendoring mechanism: the single amalgamated header copied in unmodified.

What nine minor versions bring

The optimization payload: inlined write and read spines, word-wise WriteBytes (one flush, one bulk copy, one tail reload), and the precomputed compressed-float entry points. Plus the hardening work: the in-source contraction-invariance barrier (compressed-float wire bytes are now identical under every -ffp-contract setting), the normative integer clamp, degenerate ranges (min == max), the stricter malformed-string reader, and serialize's own conformance battery riding along in the header's self-tests.

Wire verdict: bits identical on every yojimbo path

Measured, not assumed — all differentials run with yojimbo's own flags (-O2 -ffp-contract=off, arm64):

  • Packet-path differential. tools/gen_seed_corpus_connection drives real Connection::GeneratePacket output over every committed seed scenario (both channel types, the full message vocabulary, single- and multi-fragment blocks). Regenerated under 1.6.0 and under 1.15.0: all 13 packets byte-identical to each other and to the committed corpus in fuzz/corpus/fuzz_connection. tools/conformance/verify_standard.py still fully decodes the corpus against STANDARD.md, 0 failures.
  • Primitive sweep differential. A standalone harness serializing the full yojimbo serialize vocabulary (bits at every width, ranged ints, bool, float incl. denormals/extremes, double, 2,560 compressed-float values across 5 declarations, every relative-int ladder tier, align, bytes, string — 5,379 wire bytes): wire bytes and decoded read-back bit patterns identical between 1.6.0 and 1.15.0.
  • Negative control. The same harness compiled with -ffp-contract=fast against the old header produces different bytes on arm64 — proof the harness can discriminate the exact class of change under watch, so the identical result is a finding, not blindness.
  • The one place bytes move, probed directly. The normative clamp affects only compressed-float declarations with max_integer_value in [2^23, 2^24) at the top of the range. Probed with an odd max_integer_value (8,388,609): 1.6.0 wrote a code its own reader rejects (read_ok=0 — broken, unreadable wire), 1.15.0 clamps to a code both old and new readers accept. So this is a fix of previously-unreadable output, not an interop break; matches upstream's changelog exactly. yojimbo itself has no such declaration. Upstream's golden wire vector is byte-identical between v1.6.0 and v1.15.0.

Consequence for consumers: none. Old and new interoperate on every path; no wire-compat note is owed.

Tests

Full suite green before and after the swap: ./bin/test (ALL TESTS PASS, including the [serialize] section running serialize 1.15.0's embedded self-tests via SERIALIZE_ENABLE_TESTS) and custom_packet_io_test.

Docs

BUILDING.md and the CMake strict-FP rationale now describe the vendored header as it is: the barrier pins the compressed-float roundings in-source, -ffp-contract=off stays as standing policy (belt and braces, and the certification setting), -ffast-math/-Ofast remain unsupported.

🤖 Generated with Claude Code

rowan-claude and others added 4 commits August 26, 2026 13:17
Nine minor versions of the optimization and hardening work land in the
vendored header: the inlined write and read spines, word-wise WriteBytes,
the precomputed compressed-float entry points, the contraction-invariance
barrier that pins the compressed-float roundings in-source, the normative
integer clamp, degenerate ranges, and the stricter malformed-string reader.

Closes #334.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
serialize 1.15.0 pins the compressed-float roundings in-source with an
optimization barrier, so the wire bytes no longer depend on the consumer's
-ffp-contract setting. BUILDING.md and the CMake rationale now state that;
-ffp-contract=off stays as standing policy (belt and braces, and the
certification setting), and -ffast-math / -Ofast remain unsupported.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ct requires

serialize 1.15.0 debug-asserts that serialize_string writers supply valid
UTF-8 (the writer-trusted model). fuzz_connection_structured drives the real
write path, so it must generate conforming content: string bytes are masked
to ASCII, exactly as serialize's own fuzzer does. Arbitrary bytes still
reach the reader's malformed-string refusal path via fuzz_connection's raw
packets. Verified both ways: the unmasked harness reproduces the CI abort on
the new assert; the masked one replays the corpus plus 20,000 random inputs
under ASan+UBSan with asserts live, clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both version sites carry 1.11.0 (the CMake project version and the
YOJIMBO_MAJOR/MINOR/PATCH_VERSION macros in include/yojimbo_config.h).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@rowan-claude

Copy link
Copy Markdown
Contributor Author

Two commits follow the differential work. First CI run went red on both per-PR fuzz legs: serialize 1.15.0 debug-asserts that serialize_string writers supply valid UTF-8 (the writer-trusted model), and fuzz_connection_structured was pushing arbitrary fuzzer bytes into the real write path. The fix mirrors serialize's own fuzzer exactly: write-side string bytes masked to ASCII, while arbitrary bytes still reach the reader's malformed-string refusal path via fuzz_connection's raw packets. Verified both directions locally — the unmasked harness reproduces the CI abort on the new assert, and the masked one replays the committed corpus plus 20,000 random inputs under ASan+UBSan with asserts live, clean.

The version also moves to 1.11.0 at both sites (the CMake project version and the yojimbo_config.h macros), following the #335 pattern: bump in the PR, tag after merge.

@rowan-claude
rowan-claude merged commit 0eb089f into main Aug 26, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant