Skip to content

Commit 42ce4ff

Browse files
rustyconoverclaude
andcommitted
Require vgi-rpc 0.43.0 for correct nullability under Annotated
vgi-rpc's schema derivation reported "not optional" for every field wrapped in `Annotated`, because `get_origin(Annotated[X | None, ...])` is `Annotated` and never a union. 54 protocol fields were therefore described as non-null while routinely carrying nulls -- ScanSplit's four position/bounds fields, all eight optional TableInfo columns, six on PlanResponse, FunctionInfo.max_workers, CatalogInfo.releases[].released_at, and the opaque-data / pushdown / secrets fields repeated across a dozen request records. The split is not arbitrary: 139 optional fields WITHOUT an `Annotated` wrapper were always correct. `Annotated[..., ArrowType(...)]` is required exactly when the Arrow type cannot be inferred from the Python type -- a pa.Schema or pa.RecordBatch riding as opaque bytes, a dict serialized to a blob, a narrowed int -- which is to say every optional payload-carrying field in the protocol. Nothing here writes different bytes. What changes is that the schema now describes them, so a peer that compares schemas field by field stops rejecting correct workers. Both matrix legs pass against the rebuilt C++ client (325/325 subprocess, 325/325 HTTP) and the generated schemas for all five languages are regenerated in their own repos. Also fixes java_schemas importing GlobalInitResponse from vgi.protocol, which re-exports it -- mypy flags the implicit re-export; its home is vgi.invocation. And stops test_reproducible_vectors_regenerate_identically leaving the working tree dirty. It regenerates every split-token vector to check the reproducible ones are stable, but a sealed token draws a fresh random nonce, so its bytes differ every run. I nearly committed that churn into this commit; the non-reproducible vectors are now restored afterwards. NOTE: this cannot be installed until vgi-rpc 0.43.0 is published. The local co-development path override is left commented in [tool.uv.sources] beside the existing duckdb one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 23d25ab commit 42ce4ff

3 files changed

Lines changed: 19 additions & 7 deletions

File tree

‎pyproject.toml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ dependencies = [
4040
"pyarrow",
4141
"typer>=0.9",
4242
"platformdirs",
43-
"vgi-rpc>=0.40.0",
43+
"vgi-rpc>=0.43.0",
4444
"httpx2>=2.9.1",
4545
]
4646

@@ -258,6 +258,7 @@ members = ["packages/vgi-fixtures"]
258258

259259
[tool.uv.sources]
260260
# duckdb = { path = "/path/to/duckdb-python/dist/duckdb-*.whl" } # local engine-build override
261+
# vgi-rpc = { path = "../vgi-rpc-python", editable = true } # local vgi-rpc co-development
261262
vgi-fixtures = { workspace = true }
262263

263264
[dependency-groups]

‎tests/test_split_token.py‎

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -192,11 +192,22 @@ def test_reproducible_vectors_regenerate_identically() -> None:
192192

193193
before = {p.name: p.read_bytes() for p in FIXTURES.glob("*.bin")}
194194
subprocess.run([sys.executable, str(FIXTURES / "generate.py")], check=True, cwd=FIXTURES.parents[2])
195-
for case in MANIFEST["cases"]:
196-
if not case["reproducible"]:
197-
continue
198-
name = f"{case['name']}.bin"
199-
assert (FIXTURES / name).read_bytes() == before[name], f"{name} drifted"
195+
try:
196+
for case in MANIFEST["cases"]:
197+
if not case["reproducible"]:
198+
continue
199+
name = f"{case['name']}.bin"
200+
assert (FIXTURES / name).read_bytes() == before[name], f"{name} drifted"
201+
finally:
202+
# Put the NON-reproducible vectors back. A sealed token draws a fresh
203+
# random nonce, so regenerating rewrites its bytes every run and leaves
204+
# the working tree dirty — which is how unrelated nonce churn ends up
205+
# staged in someone else's commit. The reproducible ones are restored
206+
# too: they compared equal, so this is a no-op for them.
207+
for name, data in before.items():
208+
path = FIXTURES / name
209+
if path.read_bytes() != data:
210+
path.write_bytes(data)
200211

201212

202213
# --------------------------------------------------------------------------- #

‎vgi/codegen/java_schemas.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,14 +47,14 @@
4747
CatalogAttachRequest,
4848
CopyFromContext,
4949
CopyToContext,
50-
GlobalInitResponse,
5150
InitRequest,
5251
TableBufferingCombineRequest,
5352
TableBufferingDestructorRequest,
5453
TableBufferingProcessRequest,
5554
TableFunctionCardinalityRequest,
5655
TableFunctionPlanRequest,
5756
)
57+
from vgi.invocation import GlobalInitResponse
5858
from vgi.codegen._common import (
5959
EXTRA_RESPONSE_TYPES,
6060
EmittedSchema,

0 commit comments

Comments
 (0)