Skip to content

fix(client-contract): keep vectors servable and judge every column by same_value - #34

Merged
jm-rivera merged 1 commit into
devfrom
fix/vector-parquet-types
Oct 29, 2026
Merged

jm-rivera merged 1 commit into
devfrom
fix/vector-parquet-types

Conversation

@jm-rivera

Copy link
Copy Markdown
Contributor

Every reading-order vector is now a table a server could serve, and the validator judges every column by one equality rule.

The SDKs replay each vector by writing it to Parquet and reading it back, as a client reads a dataset. In nested-cells-compare-by-value the flags column held [true] in one row and [1] in another, which no Arrow list column can hold, so no server could serve it and Python could not replay it. flags now holds [true] and [false], and _unorderable reports any member column pa.array cannot build (mixed list items, a struct field of mixed types). The bool-against-number case stays in the unit tests of both reference implementations, since no vector can carry it.

served_table judged flat columns with Arrow unique(), which counts 0.0 and -0.0, and two NaN bit patterns, as two values, while same_value counts each pair as one. A vector and its Parquet file could therefore group columns differently. Every column, flat or nested, is now judged by same_value in one pass, the rule table_from_rows uses.

Testing: test_a_vector_and_its_parquet_file_group_columns_alike (signed zeros, NaN payloads) fails on dev; the two new malformed-vector cases are reported. uv run tools/validate_client_contract.py passes (125 scenarios) and the tools tests pass (586).

… same_value

The nested-cells vector's flags column holds one item type, the validator reports a vector column Arrow cannot build, and served_table judges flat columns by same_value as it does nested ones.
@jm-rivera
jm-rivera merged commit ae602f5 into dev Oct 29, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant