You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 2b1cf28
Browse filesBrowse the repository at this point in the historyBrowse files
feat: let a stream function choose its own state serialization
TState had to extend ArrowSerializableDataclass. But the framework only ever
needs two things from a state -- turn it into bytes, turn those bytes back
into state -- and that is a protocol, not a base class. Stated as
StreamStateCodec, with the requirement checked structurally.
ArrowSerializableDataclass satisfies it and remains the default: declare a
dataclass and the encoding is written for you. It is just no longer the only
option, which matters in two directions. Small states pay heavily for Arrow
IPC framing, and a Python worker may need to match the state encoding a
sibling VGI implementation in another language already uses. Neither was
expressible before.
The fixture states are the worked example. CountdownState is two integers,
and it was costing 416 bytes and 36us to encode through Arrow against 16
bytes and 0.15us packed with struct -- the framing was the entire cost. It
now implements the two methods itself, which is what the docstring points at
for anyone wanting to do the same.
Rejection is unchanged in kind, only in reason: a plain dataclass still
raises at class-definition time rather than silently falling back to
initial_state() on every HTTP exchange, which would look like a stream that
keeps restarting. The message now names the two methods instead of a base
class.
Also caches the partition-field scan. Which columns are partition columns is
a pure function of the schema's field metadata and the schema is fixed for a
stream, but it walked every field per turn: 0.9us at one column, 6.5us at
twelve. Returns a tuple from the cache so the value cannot be mutated
through a caller's reference, with the public wrapper copying into a list to
keep its signature.
Measured end to end on a loaded server: +25%, 1142 -> 1429 turns/s -- more
than the codec micro-benchmarks predicted, because a 728 -> 26 byte state
also removes zstd, AEAD and base64 work on both sides of every turn.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
0 commit comments