Skip to content

Commit 0eb089f

Browse files
rowan-claudeclaude
andauthored
Re-vendor serialize at v1.15.0 (#336)
* Re-vendor serialize at v1.15.0, byte-for-byte as upstream ships it Nine minor versions of the optimization and hardening work land in the vendored header: the inlined write and read spines, word-wise WriteBytes, the precomputed compressed-float entry points, the contraction-invariance barrier that pins the compressed-float roundings in-source, the normative integer clamp, degenerate ranges, and the stricter malformed-string reader. Closes #334. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Floating-point docs describe the vendored serialize as it is now serialize 1.15.0 pins the compressed-float roundings in-source with an optimization barrier, so the wire bytes no longer depend on the consumer's -ffp-contract setting. BUILDING.md and the CMake rationale now state that; -ffp-contract=off stays as standing policy (belt and braces, and the certification setting), and -ffast-math / -Ofast remain unsupported. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * The structured fuzzer writes conforming strings, as the writer contract requires serialize 1.15.0 debug-asserts that serialize_string writers supply valid UTF-8 (the writer-trusted model). fuzz_connection_structured drives the real write path, so it must generate conforming content: string bytes are masked to ASCII, exactly as serialize's own fuzzer does. Arbitrary bytes still reach the reader's malformed-string refusal path via fuzz_connection's raw packets. Verified both ways: the unmasked harness reproduces the CI abort on the new assert; the masked one replays the corpus plus 20,000 random inputs under ASan+UBSan with asserts live, clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * yojimbo 1.11.0: the optimized serialization core is vendored Both version sites carry 1.11.0 (the CMake project version and the YOJIMBO_MAJOR/MINOR/PATCH_VERSION macros in include/yojimbo_config.h). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent d983a80 commit 0eb089f

5 files changed

Lines changed: 2930 additions & 204 deletions

File tree

BUILDING.md

Lines changed: 14 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -27,23 +27,20 @@ The default is a debug build. For an optimized build, pass the build type:
2727

2828
yojimbo's wire arithmetic lives in `serialize.h`, whose compressed float quantizes with two
2929
distinct roundings on write and on read: the product must round to float32 *before* the
30-
constant is added. A compiler allowed to contract fuses the multiply and the add into a
31-
single FMA and rounds once, which shifts the encoded integer and the decoded float by one
32-
ulp — a change to what goes on the wire.
33-
34-
This build sets the right flag on every target it compiles, so nothing is required of you to
35-
build yojimbo itself:
36-
37-
- GCC/Clang: `-ffp-contract=off`. Not merely the absence of `-ffast-math` — GCC's default
38-
is `-ffp-contract=fast`, which contracts across statement boundaries, and clang's default
39-
`=on` still fuses within a single expression.
40-
- MSVC: `/fp:precise`.
41-
42-
**If you compile `serialize.h` in your own translation units, set the same flag there.** The
43-
flag is `PRIVATE` to yojimbo's targets and deliberately does not leak into your build, so
44-
this one is yours to set. It is easy to miss because the symptom is architecture-dependent:
45-
FMA is in the aarch64 baseline and absent from the x86-64 one, so an unflagged build is
46-
typically wrong on arm64 and accidentally right on amd64.
30+
constant is added. The vendored serialize pins those roundings in-source with an
31+
optimization barrier, so the compressed-float wire bytes are identical under every
32+
`-ffp-contract` setting. Two rules remain:
33+
34+
- `-ffast-math` (and `-Ofast`) are not supported: they license reciprocal approximation
35+
and reassociation, which change the wire in ways no barrier can pin.
36+
- `-ffp-contract=off` (MSVC: `/fp:precise`) is the standing policy for the network
37+
libraries serialize ships in — belt and braces on top of the in-source barrier, and
38+
the certification setting for golden vectors.
39+
40+
This build sets the policy flag on every target it compiles, so nothing is required of you
41+
to build yojimbo itself. **If you compile `serialize.h` in your own translation units, set
42+
the same flag there** — the flag is `PRIVATE` to yojimbo's targets and deliberately does
43+
not leak into your build, so this one is yours to set.
4744

4845
## Building on Windows
4946

CMakeLists.txt

Lines changed: 11 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
# section below).
1515

1616
cmake_minimum_required(VERSION 3.16)
17-
project(Yojimbo VERSION 1.10.1 LANGUAGES C CXX)
17+
project(Yojimbo VERSION 1.11.0 LANGUAGES C CXX)
1818

1919
option(YOJIMBO_SYSTEM_SODIUM
2020
"Link the system-installed libsodium instead of the bundled subset in sodium/" OFF)
@@ -171,27 +171,21 @@ endif()
171171

172172
# yojimbo's wire arithmetic lives in serialize.h, and serialize's compressed float quantizes
173173
# with two DISTINCT roundings on write and on read -- the product must round to float32 before
174-
# the constant is added. A compiler permitted to contract fuses those into a single FMA and
175-
# rounds once, which moves the encoded integer and the decoded float by one ulp. That is a
176-
# change to yojimbo's traffic, not a test-suite detail.
174+
# the constant is added. The vendored serialize pins those roundings in-source with an
175+
# optimization barrier, so the compressed-float wire bytes are identical under every
176+
# -ffp-contract setting. -ffast-math / -Ofast remain unsupported: they license reciprocal
177+
# approximation and reassociation, which change the wire in ways no barrier can pin.
177178
#
178-
# It is architecture-dependent, which is why it stayed hidden: FMA is in the aarch64 baseline
179-
# and absent from the x86-64 one, so the same source and the same flags emit a fused multiply
180-
# add on arm64 and none on amd64. Debian package builds were green on every amd64 leg and red
181-
# on every arm64 leg (mas-bandwidth/yojimbo#332) -- and the green legs were green because of an
182-
# instruction set rather than because the build was right.
183-
#
184-
# So every target here that compiles serialize.h builds strict, exactly as serialize's own
179+
# Every target here that compiles serialize.h still builds strict, exactly as serialize's own
185180
# CMakeLists.txt does for its executables:
186-
# - GCC/Clang: -ffp-contract=off. Not merely "no -ffast-math" -- GCC's default is
187-
# -ffp-contract=fast, which contracts ACROSS statements, and clang's default =on still
188-
# fuses within a single expression.
181+
# - GCC/Clang: -ffp-contract=off.
189182
# - MSVC: /fp:precise, which since VS 2022 does not imply contraction.
190183
#
191184
# This is Glenn's standing estate policy (2026-08-24): "generally speaking, network libraries
192-
# we work on require -ffp-contract=off". The flag is the floor. It is PRIVATE so nothing leaks
193-
# into consumers' own flags; consumers that compile serialize.h themselves need it too, and
194-
# BUILDING.md states that as a requirement.
185+
# we work on require -ffp-contract=off" -- belt and braces on top of the in-source barrier,
186+
# and the certification setting for golden vectors. The flag is PRIVATE so nothing leaks
187+
# into consumers' own flags; consumers that compile serialize.h themselves set it too, and
188+
# BUILDING.md states that as policy.
195189

196190
if(MSVC)
197191
set(YOJIMBO_STRICT_FP_FLAGS /fp:precise)

fuzz/fuzz_connection_structured.cpp

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,12 @@ static bool fill_message( Message * message, int type, Reader & r, ConnectionCon
7777
int n = r.u8() % FuzzMaxString; // leave room for the terminator
7878
for ( int i = 0; i < n; ++i )
7979
{
80-
uint8_t c = r.u8();
80+
// masked to ASCII: the string payload is well-formed UTF-8 by the writer's
81+
// contract (serialize debug-asserts it, and asserts are live in this
82+
// harness), so this write-path target must generate conforming content —
83+
// exactly as serialize's own fuzzer does. Arbitrary bytes still reach the
84+
// reader's malformed-string refusal path via fuzz_connection's raw packets.
85+
uint8_t c = r.u8() & 0x7F;
8186
m->str[i] = (char) ( c ? c : ' ' ); // no embedded NUL
8287
}
8388
m->str[n] = '\0';

include/yojimbo_config.h

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,8 +32,8 @@
3232
#endif
3333

3434
#define YOJIMBO_MAJOR_VERSION 1
35-
#define YOJIMBO_MINOR_VERSION 10
36-
#define YOJIMBO_PATCH_VERSION 1
35+
#define YOJIMBO_MINOR_VERSION 11
36+
#define YOJIMBO_PATCH_VERSION 0
3737

3838
#if !defined(YOJIMBO_DEBUG) && !defined(YOJIMBO_RELEASE)
3939
#if defined(NDEBUG)

0 commit comments

Comments
 (0)