Skip to content

fix(client-contract): ask the file selector only for the keys it reads - #33

Merged
jm-rivera merged 2 commits into
devfrom
fix/selector-query-keys
Oct 29, 2026
Merged

jm-rivera merged 2 commits into
devfrom
fix/selector-query-keys

Conversation

@jm-rivera

@jm-rivera jm-rivera commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

The client contract's file-selector fallback now asks dfo-apis only for the keys its selector reads. A default fetch records without_party_totals: true in its ref, and selector_path put the whole rendering into GET /datasets/{id}/{cv}/file, which dfo-apis refuses with 400 request_invalid (its selector reads format, bundle and layout). So a default-fetched ref whose file had expired or gone unlisted raised instead of rebuilding. The two scenarios for that path are synthetic and faked a 302 for the refused URL.

A new file-selector rule (rules/file-selector.json, query_keys: [format, bundle, layout], four vectors) names the keys once. The validator and the capture tool both build the selector query from it, and the two synthetic scenarios now request file?format=parquet&bundle=false, the URL dev answers with a 302 to the stored digest. The published SDK has the same fault in both languages; its fix follows in DevelopmentFinanceObservatory/sdk once this merges.

Other changes:

  • The validator compares values by one rule (same_value): numbers by value, so 1 equals 1.0, booleans apart from numbers, NaN equal to NaN, lists and structs item by item. The reference reading_order in tests/test_rules.py uses the same rule, and the vector integer-and-float-are-one-value pins it.
  • A ragged row, a column mixing JSON types, or a list or struct column is reported as a problem rather than a traceback.
  • Posted renderings are kept per spec as a list, so a scenario that builds one spec under two renderings checks each ref against the right build.
  • The base-URL rule refuses %2e dot segments, backslashes and ports outside 1 to 65535, with ten vectors checked against WHATWG URL. The client-identity rule labels a non-browser runtime such as Cloudflare Workers other.
  • A mint step can state party_totals, and mint-async-build states the option it was recorded with.
  • The capture tool loads its rules before any live call and starts each long-build year scan after the year its record holds. The rules README says a choice names its option by index when two entity_choice groups offer the same labels (choose-refused-label-in-two-groups).
  • CONTRIBUTING.md and tools/README.md name pyarrow, and the spec-kit changelog says the registry records frame_unavailable on presentation.write at 404 only, while dfo-apis also serves it at 503 for an artifact read failure.

Testing: uv run tools/validate_client_contract.py passes; the tools tests (574) and the six package suites pass.

A file-selector rule names format, bundle and layout; the validator, capture tool and selector scenarios derive the query from it. The validator compares values by one rule and reports malformed vectors, and the base-URL and client-identity rules close the cases where Python and WHATWG clients disagreed.
…d columns out of sort keys

A resumed operation's result takes the rendering its build posted, and refs are checked against the build that served their file. List and struct columns never sort, the reference same_value is the validator's, selector queries lowercase booleans only, and the capture's OECD prerequisite keeps party totals.
@jm-rivera
jm-rivera merged commit c7a5569 into dev Oct 29, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant