Skip to content

README.md

Latest commit

 

History

History
233 lines (176 loc) · 67.9 KB

README.md

File metadata and controls

233 lines (176 loc) · 67.9 KB
Raw

dfo-spec-kit-client-contract (Python)

This package holds the contract the DFO clients are built to: the Python client development-finance and the TypeScript client @developmentfinance/client. The clients generate their wire types, strict checks and error maps from these files and replay the scenarios. dfo-apis CI validates the recorded exchanges against the app's current OpenAPI document, strict fields and code registry. The SDK's scheduled live journeys check deployed server behavior; a change that preserves the schema can pass the API gate. The files below are normative; where this README and the files disagree, the files win and the disagreement is a bug. See the repo README for how this package fits with the rest of dfo-spec-kit.

The import name is dfo_client_contract. It depends on the standard library only.

Installation

This package uses uv for dependency management.

uv sync

Development

# Run tests
uv run pytest

# Lint
uv run ruff check .

# Format
uv run ruff format .

# Type check
uv run ty check src/dfo_client_contract

Contents

src/dfo_client_contract/ ships with the package. Each schema's $id is its path under https://raw.githubusercontent.com/DevelopmentFinanceObservatory/spec-kit/main/python/client-contract/src/dfo_client_contract/, as the kit's other schemas are, because the kit has no schema-hosting service.

File Holds
served/openapi.json The composed dfo-apis app's OpenAPI document, which both clients generate types from
served/SOURCE.json The dfo-apis repository and commit served/openapi.json was captured from
strict-fields.json The response members the clients act on
strict-fields.schema.json The shape of strict-fields.json
actions.json The caller's next step for each refusal code, and the gap kind of each no_data reason
actions.schema.json The shape of actions.json
client-codes.json The code, class and condition of each error a client raises itself
client-codes.schema.json The shape of client-codes.json
proxy-codes.json The code and status of each refusal the website proxy answers itself
proxy-codes.schema.json The shape of proxy-codes.json
warning-codes.json The code, class and condition of each warning a client issues itself
warning-codes.schema.json The shape of warning-codes.json
scenario.schema.json One scenario record
scenarios/ The scenario records, and the binary response bodies they name
dataset-ref.schema.json The frozen DatasetRef formats, 1 and 2
refs/ Accepted and rejected DatasetRef examples
rules/ The client, analysis and publication rules, a vector file each, with schema and text
Function Returns
openapi() served/openapi.json, parsed
served_source() served/SOURCE.json, parsed
strict_fields() strict-fields.json, parsed
actions() actions.json, parsed
client_codes() client-codes.json, parsed
proxy_codes() proxy-codes.json, parsed
warning_codes() warning-codes.json, parsed
scenarios() Every scenario record; raises RuntimeError when none load
ref_examples() Every DatasetRef example; raises RuntimeError when none
rule(name) The vector file for one of RULES
contract_dir() The installed contract directory
*_schema() Each schema, parsed

Served document

served/openapi.json is app.openapi() of the composed dfo-apis app as deployed, with the Search, Datasets, Graph and Presentation surfaces mounted and the platform's /codes/{code} route beside them, and no other route. served/SOURCE.json names the commit it came from as {format_version, repository, commit, captured_from}. It names the closed runtime models, Manifest, ArtifactEntryModel, every Search part model and ProblemDetails among them, which the per-surface schema routes leave open. Both clients generate wire types from it and parse tolerantly regardless of its additionalProperties: false: an unknown member or enum value passes through.

Strict fields

strict-fields.json lists the members a client cannot act without. Each pointer names one property of one component schema, /components/schemas/<Schema>/properties/<member>. A required entry stays in its schema's required list and admits no null, so a client raises ContractError when one is missing or null. Every entry is a member some client rule acts on, and a client checks a body only along them: from the body's root it goes down through strict members alone, so a member no client reads goes unchecked, however deep. A Spec where no rule reads it, in resolved.equivalent_readings, no_data.flagged_match or no_data.alternate_match, may lack resolved or its definition's identity and raises nothing, which answer-nested-specs-unchecked pins at all three. A strict member that holds a JSON type its schema does not admit raises ContractError contract_violation at its pointer, as a missing one does, which answer-candidate-specs-not-a-list pins with candidates.specs as an object. An integer is a JSON number with no fractional part that is not a boolean, as JSON Schema's integer reads it, since a JavaScript client cannot tell 1.0 from 1: a strict member such as a manifest definition's schema_version, and a ref's definition.schema_version, admit 1.0 and refuse 1.5 and true. Each item of a strict list, parts, candidates.specs or artifacts, must be an object, and an item of another type raises contract_violation at its index pointer, such as /parts/0/candidates/specs/0. A member a client reads that no strict pointer names, and that holds a JSON type other than the list, object or string its rule reads, reads as absent, and so does every member of a holder that is not an object; an item of another type in such a list reads as absent too. So no_data.options as an object or null offers no option, mentions as a string mentions nothing, and a diagnostics that is not an object holds no warnings. answer-operations-not-a-list pins a definite part's operations as an object. An item of an assumptions or reading_assumptions list reads only when it is an object whose kind, binding_ref and text are strings, and an item of an operations list only when it is an object whose verb and phrase are strings; any other item reads as absent. A read item holds its schema's members alone: an assumption's refs as its string items when it is a list, and an operation's over when it is a string or null and its target_year when it is null or a number other than a boolean with no fractional part, read as that integer, so 2020.0 reads and echoes as 2020 and 2020.5 reads as absent. An optional member of another type reads as absent, and a client reads no member the schema does not name. An enums entry keeps every listed value, the values a client branches on: the answer kinds, the part statuses and the operation statuses. A client still accepts a value no entry lists.

The answer's kind and each part's status are the discriminators of their unions in the served document. A search response with no kind raises ContractError, as a part with no status does, although the served document gives kind a default and leaves it out of required. A part status a client does not know raises nothing and makes the part an unknown gap. An answer kind a client does not know raises ContractError, because the client cannot read the answer. A Search response body that is not an object, a resolution answer whose parts is empty, at pointer /parts, since the served schema gives it at least one item, a part whose status is not a string, at pointer /parts/<i>/status, and a definite or candidate spec whose definition.spec_id differs from its spec_id raise ContractError contract_violation, in a Search response as in a ref; search-body-not-an-object, answer-empty-parts, answer-part-status-not-a-string and answer-definition-spec-id-mismatch pin the four.

The required members are each part's status and its payload (spec, candidates, no_data, unsupported), Resolution.parts, CandidateSet.specs, NoData.reason, Unsupported.code, Spec.spec_id, definition and resolved, SpecDefinition.spec_id and identity, the manifest's definition, definition.schema_version, definition.spec_id and definition.identity, content_version and artifacts, each artifact's artifact_digest, rendering and url, and the operation resource's operation, operation_id and status. The operation's poll is nullable in the served document, so a client builds the poll path from operation_id. A client posts a spec's identity back to Search and reads its resolved.entity_groups, so a Search spec without resolved, or a definition, a Search spec's or a manifest's, without identity raises ContractError contract_violation; answer-spec-without-resolved, answer-spec-without-identity and fetch-spec-manifest-without-identity pin them. A ref holds a manifest's definition as it is, and the ref format requires its schema_version, so a manifest definition without schema_version raises ContractError contract_violation too, which fetch-spec-manifest-without-schema-version pins.

Actions

actions.json has one row per active refusal code in registries/codes.json that reaches a problem document (problem_details) or a no_data part's reason (no_data_reason), sorted by code. action is the caller's next step, and each client maps it to one error class.

action Error class The caller
fix_request InvalidRequestError Changes the request
check_id NotFoundError Checks the spec, version, operation or other id
choose IncompleteAnswerError Chooses among the gap's options
ask_data_question IncompleteAnswerError Asks the question another way
take_newest_version VersionUnavailableError Takes the newest version
log_in AuthenticationError Logs in
request_scope PermissionDeniedError Asks for the permission the call needs
stop PermissionDeniedError Stops, as nothing it can obtain allows the call
wait RateLimitError Waits, then retries
wait_for_reset QuotaExceededError Waits for the quota to reset
report ServerError Reports the trace id, after the client's retries
upgrade ContractError Upgrades the client

A code that maps to no_data_reason also carries gap_kind, the kind of gap a no_data part with that reason becomes: choice, no_data, not_understood, unsupported or timed_out. timed_out is a part Search left unresolved at its time limit. Sending the question again, or a narrower one, may answer it. choose goes with choice and only with it. choose and ask_data_question name a next step inside an answer, so a code that also reaches a problem document takes another action. The website proxy answers its own refusals with problem documents of the same shape, with type, title, status, detail, code, retryable and trace_id, the trace id repeated in the x-dfo-trace-id header. proxy-codes.json lists their codes with their status, rate_limited at 429, which also sends Retry-After, and transient_unavailable at 502, both retryable. Both are also registered on mcp.tools, which reaches no problem document, so the proxy's use of them takes no row. The validator refuses a problem document whose code is neither an active registry code nor a proxy code at that status. No proxy code has a row here, so a client raises a proxy refusal with the class its status maps to: RateLimitError for rate_limited, which search-proxy-rate-limited pins, and ServerError for transient_unavailable, which search-proxy-unavailable pins. capability_unavailable is a well-formed request for a format, reshape or conversion class the deployment does not enable, answered 501. Its action is fix_request, since the caller asks for another one, and fetch-spec-capability-unavailable pins it. write_token_invalid takes stop: a share takes only the write token issued when it was created, and a share created without one takes none, so no scope or new token lets the caller replace or delete it. licence_not_redistributable takes stop too: a source's licence forbids the export for every caller, so no scope or credential serves the file. So does provenance_unavailable: a recipe root that cannot be cited survives a rebuild, so nothing the caller can change names the frame's sources. allowance_exhausted takes wait_for_reset, and a client raises it at once as QuotaExceededError with its Retry-After, the time until the allowance resets, as the retry rules below state.

A client maps a refusal whose code has no row, or that carries no code, on its HTTP status:

Status Error class
400, 422 InvalidRequestError
401 AuthenticationError
403 PermissionDeniedError
404, 410 NotFoundError
408 DfoTimeoutError
429 RateLimitError
any other 4xx InvalidRequestError
any other 5xx ServerError

A 408 is a timeout the caller retries, and a 410 names a resource that is gone, so the caller stops as it does on a 404. A client reads a refusal's body as a problem document when its content type is application/problem+json or application/json; a body of any other type reads as no body, so the refusal carries no code. A no_data reason with no row becomes a no_data gap.

An answer's gaps follow from its parts. A candidates part is a choice gap, its options the candidates. A no_data part is a gap of its reason's gap_kind, its options the no_data options, whatever diagnostics.timed_out says. Search marks a part it left unresolved at its time limit with the reason timed_out. An unsupported part is an unsupported gap. A part whose status a client does not know is an unknown gap.

Client codes

client-codes.json lists, sorted by code, each code a client sets on an error it raises when no problem document carries one: the code, the error class it raises, retryable, and a one-line condition. retryable is the flag both clients generate onto the error: deadline_exceeded, download_failed and network_error are retryable, digest_mismatch and the rest are not, and refusal_without_code is retryable as its status is, which its row gives as by_status. Both clients set an error's code from this list, so every error a client raises carries a code. A refusal keeps its problem document's code, the website proxy's included. A refusal whose body carries no code raises refusal_without_code with the class its HTTP status maps to, which its row gives as by_status. No other row does. A problem document without a code still gives the error its retryable and trace_id, so a 503 whose body says retryable: false raises at once, which search-refused-without-code pins. Every refusal's trace id is the response's trace id, as the trace-id rule under Scenarios gives it. search-refused-without-code pins the body's winning over the header, and search-unauthorized the header's for a refusal with no body. No client code is a code in actions.json or registries/codes.json, and each class is one the scenario schema names. Each reason the key-lookup rule raises, key_empty, profile_not_found, profile_without_key and credentials_file_invalid, is a row of class AuthenticationError, as are key_invalid, a key outside visible ASCII, and credentials_file_unreadable, a credentials file that exists but cannot be read. header_invalid, a header value with a control character or a caller's header named X-DFO-Client, insecure_transport, a key over plain http to a non-loopback host, and base_url_invalid, a base URL that is empty, not absolute http or https, or carries userinfo, are InvalidRequestError rows. response_too_large, a decoded body past its cap, is a ServerError row. None of these six is retryable. The list holds no warning. warning-codes.json lists, sorted by code, each code a client sets on a warning it issues itself, with its class and condition: ref_digest_changed, a DfoWarning, for a ref member whose content_version serves a file with another digest than the ref's, because the server rendered the same data again, where the client returns the data; route_deprecated, a DfoDeprecationWarning, for a Deprecation or Sunset header; server_warning, a DfoWarning, for an x-dfo-warning header; series_left_out, a SeriesLeftOutWarning, for a derived frame that left series out for missing data, where the client returns the frame; and key_checksum_mismatch, key_not_base62 and key_unknown_prefix, each a DfoWarning, for a key that fails the key-format rule with that reason. Both clients generate their warning codes from it and issue no other. No warning code is a code in another table or in registries/codes.json. A SearchWarning carries the code of the one way the readings differ from the question, a departure's or a warning's code or the verb of an operation Search did not run, and a null code when they differ in several ways. A DataWarning carries the manifest warning's code.

Scenarios

A scenario is one client-level call, the HTTP exchanges it makes, and the outcome each step reaches. Each record is one file, scenarios/<name>.json, validated by scenario.schema.json.

Member Holds
format_version 1
name The kebab-case name, equal to the file name's stem
description One or two sentences on what the scenario proves
origin recorded, with the base_url, the deployed dfo_commit and recorded_at; or synthetic, with the recorded scenario it is derived_from and a one-line edit
call The steps, in order: search; choose; fetch of the previous answer, of a spec with window, entities and facets, or of a ref; mint; operation, which resumes a long build by its operation_id; result; newest_version of a spec
exchanges The requests the call makes, in order, each with the response to give
expect One outcome per step: an answer, a dataset, an operation, a newest_version or an error

A step that acts on an earlier result follows one: choose follows search or choose, a fetch with no target follows search, choose or another fetch with no target and acts on the last answer, and result follows mint or operation. A fetch of an answer or a spec, mint and operation take a layout, long by default or frame, a fetch of an answer or a spec takes strict, and that fetch and mint take party_totals. A spec in a step holds spec_id, the server's unmodified definition, the sentence when the client knows it, and the server's resolved.entity_groups as entity_groups when it names any.

An exchange's request names its target in one of three ways. path is a path the client composes under the base URL, such as /search/resolve. origin_path is a root-relative URL the server returned, such as an artifact's url, or the path and query of an absolute one on the API's origin, which resolves against the base URL's origin, whatever path that URL has. url is an absolute URL outside the API, such as the signed storage URL a files route redirects to, which the client fetches with a bare request carrying no client header and no key. json holds the request body members that must match; a recorded scenario lists every member the client sent. The response gives status, the headers a client reads, and a json body or a body_file, a binary body in scenarios/. A scenario's own body files are named <name>.<ext> and <name>-<n>.<ext>; a synthetic scenario may instead name a body file its parent names.

A newest_version outcome gives the content_version the step found, or null. An answer outcome gives kind, status, sentence, complete, the distinct specs of its definite parts, and its gaps, each with the part index, the gap kind and the number of options. A dataset outcome gives rows, columns, complete, its members, each with parts, spec_id, content_version and sentence, and the missing gaps. For one member the columns are the artifact's columns in the order the reading-order rule gives; a dataset of several members adds a part column and gives part_rows, one entry per member in member order with the part value of that member's rows and their number. A fetch of an answer also gives search_warning, whether the fetch issued a SearchWarning. An error outcome gives the error class and its code: a refusal's code, a warning's code, which for a SearchWarning may be null, or for any other error the client raises itself its code in client-codes.json. Every error outcome gives its trace_id: a refusal's, as the refusal rules below give it; a failed operation's, its error document's trace_id, else the x-dfo-trace-id header of the response that carried the failed resource; a VersionUnavailableError's, its first refusal's, or null when nothing refused. An error the client raises itself follows one rule. An error that describes a response carries that response's trace id, and an error that describes the caller's input carries null. The trace id is the one a user quotes to find the fault in the server's logs. The errors that describe a response are unexpected_response, contract_violation on a body, answer_declined, answer_incomplete, and a strict fetch's SearchWarning or DataWarning. A response's trace id is the top-level trace_id member of its body when the body is a JSON object holding one as a non-empty string, a problem document or any other body, else its x-dfo-trace-id header, else null. A body trace_id that is empty or not a string reads as absent, so the header applies. No scenario holds one, and the validator's tests pin it. A Search answer's diagnostics.trace_id is not read, and answer-timed-out pins the header's winning over a diagnostics.trace_id that differs from it. answer_declined, answer_incomplete, a SearchWarning, and a contract_violation a choice raises on an option describe the held answer, so they carry the trace id of the Search response that answered it. The errors that describe the caller's input, choice_invalid, invalid_argument and the key-lookup reasons, carry null, and so do digest_mismatch and download_failed, which describe a storage-host fetch, and deadline_exceeded, network_error and offline, which describe no response. search-no-json-body, fetch-spec-strict-warning, answer-timed-out, answer-empty-parts, fetch-spec-files-route-no-content and choose-option-two-part-query pin a trace id on an error that describes a response, operation-failed-without-error the header of the response that carried a failed resource with no error, and choose-refused-shared-label a null one on a choice whose answer's response carries a trace id. A VersionUnavailableError also gives newest_version, the version the lookup found, or null. A dataset or error outcome of a step that made a long build gives long_builds, one entry per build the server ran on, in order: the operation_id the client polled last, the poll_waits in seconds before each poll, and the number of resubmissions. Any outcome of a step that retried a refused request gives retry_waits, the seconds before each retry in order, and any outcome of a step whose API responses issued warnings gives warnings, their codes from warning-codes.json in order.

Each outcome follows from the exchanges and actions.json, and tools/validate_client_contract.py fails a record whose expect says otherwise. A record whose outcomes it cannot derive, because its exchanges do not fit its steps or a malformed member raises while it reads them, is reported as that record's problem, and the other records are checked. It runs the steps over the exchanges by the rules below: a refusal followed by the same request is a retry where the retry rules allow one, except on a poll, where the long-build rules count retries, and a record that ends a run of retries while the rules allow another does not fit, since the client makes it, a refusal's class is its code's action or else its status, the class of an error the client raises itself is its code's row in client-codes.json, a gap's kind is its reason's gap_kind, option and candidate counts are those the bodies hold, a dataset's members, complete, missing and search_warning follow from the answer and the manifests, each POST carries its body, each build posts the body the rules give for the spec it builds, so the builds' order and spec_ids are checked, each Search request posts the whole body {"schema_version": 1, "input": ...} with the input the rules give: the question, a choice's input, or a member's identity, each poll names the operation the client holds, each files route is the one the artifact's url names, each signed download is the URL the files route's location names, each retry waits the wait the retry rules give, and each response's warning headers issue the warnings the warning rules give. A dataset's columns are derived: the validator reads each downloaded Parquet file with pyarrow, its columns in the order the server wrote them and which of them vary between rows, and orders them as the reading-order rule gives. The outcome holds no rows, so the rule's vectors pin the row order. A dataset's rows and part_rows row counts are the Parquet bodies' and are not derived, except that part_rows is given exactly when the dataset has several members, each entry's part is its member's value and their rows sum to rows.

A recorded scenario's request and response bodies validate against their route's schemas in served/openapi.json. A required member stays required when its schema admits null: the body carries it, as null or a value. A member the schema leaves optional may be absent or null, and a client reads both as null.

A client replays a scenario by serving the exchanges in order:

  • A request matches the next exchange on its method and target, and on each listed json member, compared as parsed JSON values.
  • Each path and origin_path request carries the X-DFO-Client header the client-identity rule gives, and each url request carries none. Records keep no request headers, so the replay checks the header on every request it serves.
  • A request with no exchange left, or one that matches none, fails the scenario, and so does an exchange the call never makes.
  • path resolves against the base URL the replay gives the client, and origin_path against that base URL's origin, so a replay can run under any base URL form.
  • A replay substitutes its own base URL's origin for the origin of the record's origin.base_url, a synthetic record's parent's, wherever a response names it, in its JSON body and its headers. fetch-spec-same-origin-artifact-url names the recorded origin in its manifest's artifact urls, and the validator replays every scenario under its recorded base URL and again under https://replay.example.org with the same path, so a record that holds only under its recorded host fails.
  • Sleeps are not replayed. The replay injects the client's clock and sleep, and each wait the client asks for advances the clock at once. The waits before polls are the outcome's poll_waits, and the waits before retries its retry_waits.
  • A replay runs each scenario as a fresh process, so no warning has been issued when it starts.
  • Every absolute URL a record holds, a url request or a location header, is a signed URL on https://storage.googleapis.com with its path kept and its query X-Goog-Signature=redacted, whatever signed it; a location may also name a route under the base URL. The validator compares hostnames and refuses any other host, a local app's under any name or port included.

The records also fix these client rules:

  • A fetch builds each distinct spec_id once, in part order, and finishes one member (build, the sentence when the client lacks it, files, download) before the next.
  • A fetch of a spec posts its narrowing as the build's input.scope, and posts no scope when it narrows nothing. A mint takes a layout and no narrowing, so it posts no scope. window posts {"kind": "range", "start_year", "end_year"} for a pair of years, or {"kind": "all"} or {"kind": "latest"}, and facets posts each axis as {"ref": ...}, and facets: {} names no axis and posts no facets. The server's scope.entities replaces the spec's whole entity binding, so entities posts a whole map: the roles the spec's definition.identity.entities binds, with each role the caller names replaced by its refs and each role the caller binds to null dropped. Naming origin alone on a spec bound to origin and destination keeps destination. A fetch that names no role, entities: {} included, posts no entities, so a fetch whose narrowing is all empty posts no scope; fetch-spec-empty-narrowing pins it.
  • A built spec_id that differs from the posted one, or a spec with no sentence, takes its sentence from Search: the client posts the built definition's identity as one dataset_resubmit identity entry, with the member's first part as variable_text, or "dataset" when no answer part names it, and checks the answer's spec_id equals the built one. The entry's entity_groups holds the spec's resolved.entity_groups for each role the built identity binds to the same entities as the spec's definition, so a window or facets narrowing keeps every group and a role the narrowing rebound or dropped loses its group. The entry leaves entity_groups out when it keeps no group, as for a resumed operation, whose spec the client does not hold. The server reads a role posted without its group as its members, so a narrowed spec of flows to the least developed countries would read "44 recipients" in place of "the 44 Least Developed Countries".
  • A dataset of several members puts the part column first, then the members' columns, gathered in member order with each column once and ordered by the reading-order rule, which also sorts each member's rows. A column that members hold in different types widens as pyarrow's permissive promotion widens it, and a pair that cannot widen raises ContractError contract_violation. A member file that already holds a part column raises contract_violation when members stack. Rows and columns are not derived, so no scenario pins these. The part value on a member's rows is the member's parts joined by "; ", so a member that answers two folded parts reads "health ODA disbursements; education ODA disbursements", or the member's sentence when parts is empty. A definite part whose variable_text is absent or empty adds nothing to its member's parts, which answer-folded-parts-unnamed pins.
  • A choice names its part by index, by variable text, or by a word that one gap's options mention, and its option, a candidate among them, by index or label. A boolean part or option is neither an index nor a label. A candidate's label is the text of its spec's resolved.sentence_plan tokens in the heading segment, a token whose segment is not a string counted in it, joined and stripped of surrounding white space, or else its resolved.rendered_sentence. A candidate with neither a heading nor a sentence has a null label; no scenario pins it, and the validator's tests do. A no_data option's label is matched among the options that mention the word the part was named by. An entity_ambiguous option carries entity_choice, the index into no_data.ambiguous_entities of the group it settles, and a menu groups the options by it. When one place name is ambiguous at two positions, as in "aid from Congo to Congo", each group offers the same labels, so a label matches an option in each group and the choice names its option by index, which choose-refused-label-in-two-groups pins. With no part named, exactly one gap must await a choice: a choice gap, or any gap with options.
  • A choice raises InvalidRequestError choice_invalid and makes no request when no gap, or more than one, awaits it, when the part is named by a boolean, is unknown, names more than one part or already has a reading, when the gap offers no options, or when the option is a boolean, is out of range or matches no label or several. No scenario can hold a boolean part or option, and the validator's tests pin both. It raises the same when an entry it would post cannot be built: a part with no variable_text, or a part of a status the client does not know that has no restatement. A chosen option whose data_query holds other than one part raises ContractError, and so does a chosen option with no data_query, in a one-part answer as in a several-part one, before any request. choose-refused-shared-label, choose-refused-unknown-candidate-label, choose-refused-shared-candidate-label, choose-refused-part-without-variable-text, choose-refused-unknown-status, choose-option-two-part-query and choose-option-without-data-query pin these, and choose-candidate-by-label pins a candidate chosen by its label.
  • Every pick of a candidate asks Search for the picked answer, and no client builds it locally. In a one-part answer the pick posts dataset_resubmit, without raw_query, with one identity entry: the candidate's identity and entity_groups, and the part's variable_text, separate and operations as below. The picked answer, its sentence with any "Assumed: …" clause, and its assumptions are Search's response. A choice of a no_data option in a one-part answer posts the option's data_query as the input.
  • In a several-part answer a choice posts dataset_resubmit, without raw_query, with one entry per part in part order. The chosen candidate posts an identity entry with its identity and entity_groups, and the chosen option its data_query's one part as a parsed entry. A definite part posts an identity entry with its spec's identity and entity_groups. A candidates part left alone posts a candidates entry: its candidate set in the server's order, each with its identity and entity_groups, and its ordering and truncated echoed. The server cannot recompute that order, because it rests on the first answer's bind election and availability reads, and a parsed entry would cost an LLM bind. Any other part posts its restatement as a parsed entry, or a placeholder entry with its no_data reason or unsupported code when it has none. Every entry but a parsed one carries the part's variable_text, separate when true and operations when any of the part's operations reads. entity_groups is left out when the spec has none. choose-echoes-separate-and-operations pins the echo on a definite part's entry, and choose-posts-unsupported-placeholder an unsupported part's placeholder.
  • Each identity a choice posts echoes its reading's assumptions as assumptions: the chosen candidate's entry carries its part's reading_assumptions under the candidate's spec_id, a definite part's entry carries the part's assumptions, and each candidate of a candidates entry carries its own reading_assumptions entry. Search then states the echoed items that still hold, so the picked answer states what that reading assumed. [] states none. An identity with no assumptions reads as a fresh request, so a client that holds no list for a reading, from a part that carries no assumptions or reading_assumptions, posts none. The identity entry that asks for a built sentence posts no assumptions, since it reads only the spec's own rendered_sentence. An echoed operations or assumptions list posts exactly the items a client reads under Strict fields, in order, each built from the members it read: an optional member that read as absent is left out, and a refs list posts its string items. An assumptions list with no item read posts [], and an operations list with none is left out. The item rule checks JSON types alone, so the echo posts what a client read, and the server still checks it against its own value constraints: the verb enum, the lengths of phrase and text, and at most one item in refs. choose-echo-drops-malformed-items pins both lists.
  • A strict fetch raises the manifest's first warning above info severity as a DataWarning with that warning's code, before the sentence lookup and the download.
  • An identity resubmit whose answer is not one definite part of the built spec_id, with a sentence, raises ContractError.
  • The frame layout posts {"format": "parquet", "bundle": false, "layout": "frame"} as the build's options and downloads the artifact whose rendering's layout is frame; the long layout posts {"format": "parquet", "bundle": false} and downloads the bare Parquet artifact with any other layout. A fetch of an answer or a spec, and a mint, add "without_party_totals": true to either, as the next rule gives. A client matches an artifact on its rendering's format, bundle and frame layout only, and without_party_totals plays no part in the match. A fetch of an answer with several members in the frame layout raises InvalidRequestError before any build.
  • A fetch of an answer or a spec, and a mint, leave out party totals by default. The build posts "without_party_totals": true in options, and the server drops the rows whose party, on a flow party role the identity binds no entity on, the code index declares a group total, before any reshape. A role the caller binds keeps what it binds, so a total named by its dcid still comes back. The client option is party_totals, false by default, in Python, and partyTotals in TypeScript, and a fetch or mint step's party_totals: true stands for it: the build posts no without_party_totals and keeps every total. The wire is the served rendering flag as it was; only the client's default changes. A cut that leaves rows out answers a version of its own, listed by the versions route with the plain build as its base_content_version, and a cut that leaves nothing out answers the plain version and its file. The manifest's artifact rendering carries without_party_totals: true in both cases. A mint posts the options a fetch of its spec posts, so its result answers the table that fetch answers, which mint-async-build pins with party_totals: true. A resumed operation posts no build, and its result answers the rendering the operation's build posted: the layout's rendering, with without_party_totals: true when the done manifest's artifact for that layout carries it. fetch-spec-party-totals-left-out pins the default on a DAC2a spec whose origin the fetch leaves open, where the cut drops the donor group totals, and fetch-spec-party-totals-kept the option.
  • A download asks the route that redirects to the signed URL, an artifact's url or the file selector, and fetches the URL its location names once. A files route answers 302 with the location the client fetches, or 200 with the file as its body, which the client hashes with no further request, which fetch-spec-files-route-serves-file pins. A refusal reads by the refusal rules, and any other status, another 2xx or another 3xx included, raises ContractError unexpected_response with that response's trace id, which fetch-spec-files-route-no-content pins with a 204 and fetch-spec-files-route-temporary-redirect with a 307. The route ask is an API request and follows the retry rules below, so a retryable refusal is retried within the budget and any other refusal, artifact_expired among them, raises at once. A failed fetch gets exactly one more round. A fetch that answers anything but 200, which fetch-spec-signed-fetch-partial pins with a 206, or bytes that do not hash to the artifact's digest ask the route again, under the same retry rules, and fetch again, with no wait between rounds. A second failure raises ServerError with download_failed, or digest_mismatch when the bytes are wrong. A network error or timeout on the fetch is retryable, so it starts the second round too, and when the second round fails the same way the client raises that failure as itself, network_error or deadline_exceeded. The replay holds no network failure, so no scenario pins that case.
  • A manifest artifact's url names its files route on the API's origin. A root-relative url resolves against the base URL's origin, and an absolute one on that origin is asked as its path and query, so both make the same origin_path request, which fetch-spec-same-origin-artifact-url pins. An artifact url on any other origin, the storage host's included, raises ContractError contract_violation before the sentence lookup and any request for the file, which fetch-spec-foreign-artifact-url and fetch-spec-storage-artifact-url pin.
  • A relative location resolves against the files route's URL, and the client fetches it on the API's origin as an origin_path request. A location on an origin other than the API's or https://storage.googleapis.com raises ContractError contract_violation before any fetch. A fetch on the storage host is bare and carries no API key. fetch-spec-relative-location pins the relative case.
  • The redirect rules above hold where a client reads the files route's response itself: the Python client, and the TypeScript client in Node. A browser page or worker cannot, because a fetch with redirect: "manual" answers an opaque redirect with no status and no location. There the TypeScript client asks the files route again with redirect: "follow", sending the API key alone, which the browser drops on the way to another origin, and the followed request is the round's one fetch. The response must end on the API's origin or https://storage.googleapis.com, and any other host raises ContractError contract_violation after the fetch. So in a browser a files route answering 301, 303, 307 or 308 is followed as a 302 is. The refusal rules, the retry rules, the rounds and the digest check hold in every runtime. The scenario format names no runtime, so the scenarios fix the Python and Node behaviour, and the TypeScript client's own tests check the browser's.
  • A declined or catalogue answer is not complete, and fetching it raises DeclinedError.
  • Fetching an answer with a gap raises IncompleteAnswerError before any request, unless the fetch is partial and some part has a reading.
  • A fetch of an answer issues one SearchWarning when a definite part has an operation Search did not run, a departure, or a warning at severity warn or degraded with a message, or the answer's diagnostics.warnings holds such a warning. An info warning, a warning with no message, and an operation that does not read under Strict fields issue nothing. Each way is one line a client shows: Not done: <verb> (<phrase as JSON>) coded by the verb, the phrase quoted as a JSON string that keeps non-ASCII characters as they are, the departure's text coded by its code, or Note: <message> coded by the warning's code, in part order with the diagnostics warnings last, and ways that show the same line count once, under the first one's code. A strict fetch raises the SearchWarning before any build, with the code of its one line, or a null code when there are several lines. A strict fetch raises it before the frame layout's several-member check.
  • A ref's version is found in this order: the snapshot route and the files route at the stored digest; the file selector, GET /datasets/{id}/{content_version}/file with the query the file-selector rule in rules/README.md builds from the ref's rendering, which leaves out without_party_totals; and a build of the ref's spec in the ref's rendering, which must answer the ref's spec_id and the same content_version. Both recovery steps keep the ref's identity, so a recovered member carries the ref's spec_id and content_version: a snapshot manifest whose definition.spec_id or content_version is not the ref's, or a rebuild manifest whose definition.spec_id is not the ref's, raises ContractError contract_violation before any download, which fetch-ref-expired-rebuild-other-spec pins for the rebuild. A snapshot refusal other than artifact_expired ends the lookup, so snapshot_not_found raises NotFoundError with no further request. A files route or selector refusal of artifact_expired, artifact_not_found or not_found takes the next step, and a selector file of another digest is not the stored file. A snapshot manifest whose artifacts list no artifact of the ref member's digest sends the client to the selector for the ref member's own rendering, and a file that hashes to that digest gives the dataset, which fetch-ref-artifact-unlisted pins. A rebuild that answers another version, or that raises InvalidRequestError, NotFoundError or VersionUnavailableError, ends in VersionUnavailableError with the first refusal's code, or version_unavailable when nothing refused. Its newest_version is the version the rebuild answered, or else the newest stored version, as a newest_version step finds it: the first version GET /datasets/{id}/versions?limit=100 lists with a null base_content_version, the newest plain build, read page by page through &cursor= with each page's next_cursor until one is listed. The route lists versions newest first and needs no key. A version with a base_content_version is derived from a plain build, a groupby collapse or a party-totals cut, and is skipped, which fetch-ref-newest-version pins. The newest version is null when the list ends without a plain version or the route refuses with NotFoundError. It names the newest plain build, which a fetch with the default party totals rule answers only when the cut leaves nothing out; when the cut leaves rows out, the fetch answers a version whose base_content_version is that build. So a client does not present newest_version as comparable to the content_version of a cut ref or a cut fetch: two different versions there need not mean newer data. Any other refusal of the versions route raises, and a page whose versions is not a list of versions each with a content_version and a base_content_version that is a string or null, or whose next_cursor is neither a string nor null, raises ContractError contract_violation, which newest-version-unreadable-page pins.
  • A build answers 200 with the manifest, or 202 with an operation the client polls at /datasets/operations/{operation_id}, the id of the operation it holds. The 202 reads pending or running, because a submission starts a build or joins one under way, and a done operation left by an earlier submission of a rejected head is built again. A deterministic failure answers a 202 that reads failed. fetch-spec-async-build pins pending and fetch-spec-accepted-running pins running. A POST for a build identical to one that is pending or running returns that same operation, because the server derives the operation id from the request: dfo-apis src/datasets_api/operation_store.py compute_operation_id hashes the request identity, the origin catalogue build id and the rendering. A client that stops waiting for a build can post the same build again to join it. A mint's build answers 202. A record whose mint answers 200 fits no mint step. Before each poll it waits the Retry-After of the refusal that answered the last poll, whatever its status, as every retry of a refused request waits; else the last operation resource's retry_after_ms, else the Retry-After of the last answered poll, the response that carried that resource, else a backoff of 1, 2, 4 and then 8 seconds that advances one step each time it is used, a poll after a refusal included. fetch-spec-poll-pacing-changes pins a retry_after_ms that changes from one poll to the next, so a client waits each poll's own value.
  • A retryable 429 poll spends no retry, and a 429 whose body says it is not retryable raises at once, as a refusal's retryable outranks its status, which fetch-spec-poll-rate-limited-not-retryable pins. Any other retryable refused poll spends one of the two retries, which a poll that answers restores, and a refusal that is not retryable raises. fetch-spec-poll-refused pins two 503s polled again after an answered poll, fetch-spec-poll-refused-backoff a 503 with no Retry-After among polls no response paces, after which the backoff advances, fetch-spec-poll-refused-retry-after a 503 whose Retry-After paces the next poll, and fetch-spec-poll-not-found a 404 that raises at once. A poll that answers a failed operation whose error is retryable posts the same build once more and polls the operation it starts; a second failure, an error that is not retryable, and a resumed operation, whose build request the client does not hold, raise the operation's error. A refused poll whose code is operation_failed is a failed operation with that error. A failed operation whose resource carries no error raises ServerError with operation_failed, retryable by status 500 as a refusal without a document is, which operation-failed-without-error pins. A 202 or resumed operation that already reads failed raises its error before any poll.
  • Any other refusal is retried when it is retryable: never when its code's action is wait_for_reset, whatever its document and status say, else as its problem document's retryable says, else when its status is 408, 429, 500, 502, 503 or 504. A client makes two retries by default, and the scenarios assume that default. A build's POST shares those two retries with every other refusal of it, and once a 504 cuts the build off the client posts it once more only, whatever that post answers: a build refused 504 then 503 raises the 503, a build refused 503 then 504 is posted once more after the 504, and a second 504 raises. fetch-spec-504-then-503, fetch-spec-503-then-504 and fetch-spec-504-twice pin the three. A refusal that is not retryable, or one past the retries, raises its error.
  • Each retry waits the refusal's Retry-After when it carries one, and otherwise the client's backoff for that attempt: 0.5 seconds before the first retry, doubling to at most 8. The build's 504 cut-off retry follows the same rule. A client waits a Retry-After of at most 60 seconds (RETRY_AFTER_MAX_S). A refusal whose Retry-After is longer raises at once with that wait on the error, a refused poll included, so no call sleeps for minutes unasked. retry_waits pins these waits, fetch-spec-504-retry-after pins a cut-off that carries Retry-After, and search-retry-after-past-cap a retryable 503 whose Retry-After of 61 raises with no retry.
  • A 401 resolves the key again by the key-lookup rule. When the key changed, the client sends the request once more at once, with no wait and spending no retry. When it did not, or the second answer is a 401 too, the 401 raises AuthenticationError. A replay's key never changes, so search-unauthorized pins the second case and no scenario the first.
  • A request(method, path) whose path is not a path on the base URL, because it starts with // or names a scheme, raises InvalidRequestError invalid_argument before any request. No scenario step makes such a request.
  • A 2xx response of a JSON route with no JSON body raises ContractError unexpected_response. search-no-json-body pins it for Search.
  • A Deprecation or Sunset header on an API response, either alone, issues route_deprecated once per process per distinct pair of their values, and an x-dfo-warning header issues server_warning once per process per message. A signed fetch's headers issue nothing. search-sunset-warning pins a Sunset header alone, issued once over two responses that carry the same value, and search-server-warning pins a repeated x-dfo-warning message issued once, a new message issued again, and a Deprecation beside the same Sunset issued as a new pair.
  • A fetch of derived members issues series_left_out, a SeriesLeftOutWarning, once when the recipe anchor of any member lists left-out series (Recipe.anchor.left_out non-empty in the served document): series the rolling anchor dropped because they hold no row at the anchor period. The client returns the frame. A strict fetch raises the warning in place of its data. A recipe with no rolling period has a null anchor and issues nothing. No scenario serves a derived member yet, so the series-left-out rule's vectors in rules/ pin it.
  • Python synchronous platform name resolution can exceed the following timeout and deadline limits. Asynchronous name resolution is cancellable. Timeouts follow the clients' design and are not replayed, since the replay advances the clock only by the waits the client asks for: the client's timeout, 30 s by default, bounds one request attempt; search(timeout=60) bounds the whole search, each attempt reading until that deadline, and None means no limit; fetch waits for a build with no whole-call deadline by default, and fetch(timeout=60) bounds the whole fetch, each download getting the time left on the fetch's deadline; a download that delivers no byte for 30 s raises the retryable network_error; a synchronous build POST, and mint's POST, read for up to 150 s within any outer deadline; result(timeout=1800) bounds the wait, and None means no limit. A poll attempt that hangs past the per-attempt timeout spends a retry, and a deadline that passes during a body read raises DfoTimeoutError deadline_exceeded. In TypeScript an infinite remaining time arms no timer, and a finite one above 2^31-1 ms is clamped. No scenario step takes a timeout.

Recording

tools/capture_client_contract.py refreshes the served document and the scenarios. Its openapi command calls dfo_apis.app.create_app() in process, with the deployed surface flags (Datasets, Graph and Presentation beside Search), in a dfo-apis checkout with no uncommitted changes and no untracked files, over the checkout's own test origin catalogue and test graph release published into a temporary directory, and writes served/openapi.json and served/SOURCE.json. It refuses an environment that uv sync --frozen --check with the five extras below finds out of step with the checkout's uv.lock, so the checkout needs its environment first. From the repository root, with the checkout at ../dfo-apis:

uv sync --directory ../dfo-apis --extra eval --extra api --extra engine --extra datasets --extra graph
uv run tools/capture_client_contract.py openapi --dfo-apis ../dfo-apis

Its record command runs every scenario against a base URL, dev's Cloud Run URL by default, checks each live answer still has the shape its scenario names, derives the synthetic scenarios from the fresh records with no request, and then runs the validator. A recorded scenario holds only exchanges the server made, and the run refuses one that a canned exchange answered, and an API response without the x-dfo-trace-id header the server sends on every response but a shared-cacheable one, marked Cache-Control: public, whose cache-control header the record then keeps. The validator refuses a recorded API response with neither. The record names the commit the server's responses name. It fails when the validator reports a problem or the binary bodies pass 5 MB. --only records the scenarios it names. --offline makes no request. Each recorded scenario's steps run again over its own exchanges and rewrite its expect, and each synthetic scenario is derived again from its parent, so a change to the outcome rules or to client-codes.json reaches the records without a recording:

uv run tools/capture_client_contract.py record --offline --only answer-declined choose-refused-no-choice

Each record names the commit in the responses' x-dfo-commit header, and the run refuses responses that name none or several. Every recorded scenario's origin.dfo_commit equals served/SOURCE.json's commit, and the validator refuses one that differs, so the records and the served document describe one server. To record against a local app, run the composed app, with the same four surfaces mounted, from the checkout the document came from, with DFO_BUILD_COMMIT set to that checkout's commit, then pass its base URL:

uv run tools/capture_client_contract.py record --base-url http://127.0.0.1:8083/api

A record's origin.base_url is the base URL the run was given, a local app's included. A local app's signed URL, which its launcher serves at /__e2e__/blob/{bucket}/{object} on the API's origin, is written as https://storage.googleapis.com/{bucket}/{object} with its query redacted, as a deployment signs it, because a production download is cross-origin and the client sends no key there; the run, like the validator, refuses a record holding any absolute URL other than such a storage URL, or a location under the base URL, so a local app's host under another name or port is refused too. A server answers a version it has built inline, so mint-async-build and operation-resume each need a spec just above the 100,000-row sync budget that the server has not built: ODA disbursements from the 32 DAC countries to the 133 G77 countries to 2024 for the mint, and from the 38 OECD countries to the G77 countries to 2024 for the resumed operation, each one variable over some 3,700 donor and recipient pairs, so the done manifest stays small. The tool chooses each start year, the earliest whose spec is in that row band and has no version on the versions route. Each record builds both specs, so the next record takes later start years, and the run fails when no year in the tool's range is left. A local recording runs with DATASETS_SYNC_BUDGET_ROWS=100000. --scenarios-dir writes records outside the package, unvalidated.

DatasetRef

dataset-ref.schema.json is the frozen format a client writes for data.ref and reads back in any language. Format 1 is format_version: 1 and one or more dataset members, each {parts, sentence, spec: {spec_id, definition}, content_version, artifact: {digest, rendering}}. parts holds the variable text of each answer part the member answers, in query order, and is empty for a spec fetched outside an answer. definition is the server's definition unmodified, so it holds the schema_version, spec_id and identity the served CanonicalDefinition requires and no other member, and a rebuild posts it as it is. The format takes any integer schema_version and leaves the version to the server, which checks it when a rebuild posts the definition. rendering is the selector a client matches an artifact on, {format, bundle} with layout: "frame" for the frame layout and without_party_totals: true for a fetch that left party totals out, never the server's rendering object. A ref records the rendering its fetch posted, so a member fetched with the default records without_party_totals: true whether or not the cut left out a row, and a member fetched with party totals records no such key. The without_party_totals key is written only when true, so a plain rendering omits it and false is not a valid value. Beyond the schema, each member's definition.spec_id equals its spec.spec_id, and members hold distinct spec_ids. ref-bare-definition-rejected, ref-definition-without-identity-rejected and ref-definition-without-schema-version-rejected pin the definition's required members, and ref-definition-extra-member-rejected a member it does not admit. ref-without-party-totals is the cut rendering a default fetch records, ref-one-member the plain rendering a fetch with party totals records, and ref-without-party-totals-false-rejected pins false. refs/ holds accepted examples and rejected ones, each {format_version, name, description, scenario, ref} with expect: "accept" or error, {by: "schema"} or {by: "rule", reason}, the reason definition_spec_id_mismatch or repeated_spec_id. scenario names the recorded scenario that serves the ref's members, and is left out of a ref with no members. Each member's spec_id, parts, content_version and sentence are those of a dataset member of that scenario, and its artifact digest is the digest of a file the scenario downloads, so a re-recording that changes a member also fails the refs that still name the old one. Each member of an example the schema accepts records the rendering the scenario's build of its spec posted as options. Format 2 is format_version: 2 and one or more derived members, each the frame an analysis recipe computed, and holds no dataset member, so a ref of dataset members has one spelling, format 1. A derived member is {parts, sentence, spec: {spec_id, recipe: {normal_form_hash, replay_key}}, content_version, output_digest, roots, execution_identity, resolved_periods, artifact: {digest, rendering}}. spec_id is the recipe's ops_ id and content_version its ocv_ version. normal_form_hash (rnf_) names the computation whatever data it ran on, and replay_key (rpk_) names it on these roots under this execution identity. output_digest (out_) is the digest of the frame's output that rules/README.md defines. roots lists each dataset version the recipe ran on, {spec_id, content_version, rendering} with the rendering the recipe read, a spec_ id with a cv_ version and never a derived frame, since a recipe's normal form replaces each derived input by its own recipe. A root's rendering holds the selector's members and, for a root read summed, such as by a breakdown or a sum over StatVars, groupby, the options.groupby DFO read it with: {by, op: "sum"}, by one or more distinct column names, with levels mapping each kept hierarchical axis it rolls up to a depth from 0 to 99. An artifact's rendering admits no groupby. execution_identity names the engine, frame grammar and reduction policy versions that ran it, joined by |, such as 0.9.0|1.1.0|0.2.0. resolved_periods lists each period the recipe states relative to the latest period, {anchor: "latest", offset, rule: "half1", period}, with the year it resolved to, and is empty for a recipe that states none. DFO computes every one of these members, and a client copies them from the server's answer and computes none. parts is empty for a derived member, which no answer part names. A client that reads format 1 refuses format 2, so clients that read format 2 ship before any service writes it. == and same_data compare refs as rules/README.md states under output comparison. ref-derived-member, ref-derived-member-fixed-window and ref-derived-grouped-root are accepted derived refs, the first with two resolved periods, the second with none, and the third with a root read summed, and ref-derived-in-format-1-rejected, ref-dataset-member-in-format-2-rejected, ref-derived-without-output-digest-rejected, ref-derived-root-without-rendering-rejected, ref-derived-root-groupby-op-rejected, ref-artifact-groupby-rejected and ref-derived-period-after-latest-rejected pin what the schema refuses. repeated_spec_id holds across derived members too, which ref-derived-repeated-spec-id-rejected pins, and definition_spec_id_mismatch concerns dataset members alone. A derived example's scenario serves its roots: each root's spec_id and content_version are a dataset member's of that scenario, and its rendering the one the scenario's build posted, apart from a summed root's groupby, which no recorded build posts. No scenario serves a derived member itself, so its own identifiers and artifact digest are checked by the schema alone.

Rules

rules/README.md states the shared rules for credentials, key lookup and format, the base URL, disk caching, the order a client presents a dataset's columns and rows in, the keys a caller narrows a spec's roles by, the file selector's query, the X-DFO-Client header that identifies the client to the API, and when a fetch of derived members warns that series were left out. It also states the rules DFO applies and a client reads the results of: the output digest of a derived frame, how == and same_data compare refs, the units and shapes of the operations' outputs, how the rolling steps of one recipe share one latest period, and how a share's cards are published, replaced and taken down. A rule applies where the client and runtime implement its capability. Python implements disk caching; TypeScript currently keeps no disk cache, so cache-dir and cache-layout apply to Python. Credentials-file rules apply to runtimes with local credentials support. Each rule's vectors are in rules/<rule>.json.

Validation

# Validate the contract against its schemas, the served document and the code registry,
# without importing this package
uv run ../../tools/validate_client_contract.py

# Replay the rule vectors through a reference implementation of each rule
uv run pytest tests/test_rules.py tests/test_analysis_rules.py