diff --git a/python/client-contract/CHANGELOG.md b/python/client-contract/CHANGELOG.md index 540c799..ea1159e 100644 --- a/python/client-contract/CHANGELOG.md +++ b/python/client-contract/CHANGELOG.md @@ -11,9 +11,9 @@ All notable changes to the `dfo-spec-kit-client-contract` Python package will be - `strict-fields.json` and its schema: 29 members that stay required and non-nullable, each one a client reads, and the answer kinds, part statuses and operation statuses the clients branch on. - `actions.json` and its schema: the caller's next step for each of the 104 active refusal codes on `problem_details` or `no_data_reason`, with the gap kind of each no_data reason. - The status fallback for a refusal whose code has no row: 400 and 422 to `InvalidRequestError`, 401 to `AuthenticationError`, 403 to `PermissionDeniedError`, 404 and 410 to `NotFoundError`, 408 to `DfoTimeoutError`, 429 to `RateLimitError`, and any other 4xx or 5xx to `InvalidRequestError` or `ServerError`. -- `scenario.schema.json`, the scenario record, and 125 scenarios: 34 recorded against the deployed dfo-apis dev app at `d80df9f5f`, and 91 synthetic, each derived from a recorded one by a one-line edit, with 21 binary bodies totalling 1,134,603 bytes and records and bodies together 13,693,847 bytes. `search-place-ambiguous-twice` pins two entity_ambiguous groups of one text, each option and slot token naming its group by `entity_choice`, and `choose-refused-label-in-two-groups` pins that a label both groups offer matches two options, so a choice there names its option by index. The scenarios cover the several-part choice entries and the assumptions each identity entry echoes, a picked reading whose assumption Search states in its answer, the place choice through a no_data part's options, the choice refusals, a part without variable_text, an unknown status without a restatement, a shared label, a one-part option whose data_query holds two parts, the separate and operations echo, an unsupported part's placeholder, the timed_out, unsupported and unknown gaps, an unknown answer kind, the `SearchWarning` rule, the stacked dataset's `part` column, the frame layout, a strict fetch's `DataWarning`, a spec fetch that rebinds one of two entity roles and keeps the other, one that drops a role bound to null, a spec whose destination is a named group fetched with a window or with its origin rebound, whose sentence lookup posts the groups of the roles the build left bound, a twice cut-off build, a cut-off build retried after its Retry-After, a build refused 504 then 503 and one refused 503 then 504, a refused build, a build refused 501 `capability_unavailable`, a digest mismatch, a signed URL refused on both rounds, a relative files-route Location, an artifact url on a foreign origin, on the storage host and absolute on the API's origin, a manifest definition without identity or schema_version, a Search spec without identity or resolved, nested specs no rule reads, an empty parts list, a Search body that is not an object, a codeless 503 problem document that is not retryable, a 401, an identity resubmit answering another spec, a part whose status is not a string, a Search spec whose definition names another spec_id, a Search response with no JSON body, an option with no data_query, empty entities and facets, a Sunset header issued once, `x-dfo-warning` messages and a Deprecation and Sunset pair, an unreadable versions page, a mint and a spec fetch whose build answers 202 and is polled to done, a resumed operation, a rate-limited poll, 503 polls retried within the budget an answered poll restores, a 202 that reads running, polls whose `retry_after_ms` changes from one to the next, a 404 poll that raises, a failed operation posted again, a failed operation with no error, and a ref whose stored file expired, read through the file selector, rebuilt at the same version, at a newer one, or refused with the newest version read from the versions route: the first version it lists with no `base_content_version`, the newest plain build, so a derived version listed before it is skipped. +- `scenario.schema.json`, the scenario record, and 126 scenarios: 34 recorded against the deployed dfo-apis dev app at `d80df9f5f`, and 92 synthetic, each derived from a recorded one by a one-line edit, with 21 binary bodies totalling 1,134,603 bytes and records and bodies together 13,695,441 bytes. `search-place-ambiguous-twice` pins two entity_ambiguous groups of one text, each option and slot token naming its group by `entity_choice`, and `choose-refused-label-in-two-groups` pins that a label both groups offer matches two options, so a choice there names its option by index. The scenarios cover the several-part choice entries and the assumptions each identity entry echoes, a picked reading whose assumption Search states in its answer, the place choice through a no_data part's options, the choice refusals, a part without variable_text, an unknown status without a restatement, a shared label, a one-part option whose data_query holds two parts, the separate and operations echo, an unsupported part's placeholder, the timed_out, unsupported and unknown gaps, an unknown answer kind, the `SearchWarning` rule, the stacked dataset's `part` column, the frame layout, a strict fetch's `DataWarning`, a spec fetch that rebinds one of two entity roles and keeps the other, one that drops a role bound to null, a spec whose destination is a named group fetched with a window or with its origin rebound, whose sentence lookup posts the groups of the roles the build left bound, a twice cut-off build, a cut-off build retried after its Retry-After, a retryable refusal whose Retry-After passes the 60-second cap, a build refused 504 then 503 and one refused 503 then 504, a refused build, a build refused 501 `capability_unavailable`, a digest mismatch, a signed URL refused on both rounds, a relative files-route Location, an artifact url on a foreign origin, on the storage host and absolute on the API's origin, a manifest definition without identity or schema_version, a Search spec without identity or resolved, nested specs no rule reads, an empty parts list, a Search body that is not an object, a codeless 503 problem document that is not retryable, a 401, an identity resubmit answering another spec, a part whose status is not a string, a Search spec whose definition names another spec_id, a Search response with no JSON body, an option with no data_query, empty entities and facets, a Sunset header issued once, `x-dfo-warning` messages and a Deprecation and Sunset pair, an unreadable versions page, a mint and a spec fetch whose build answers 202 and is polled to done, a resumed operation, a rate-limited poll, 503 polls retried within the budget an answered poll restores, a 202 that reads running, polls whose `retry_after_ms` changes from one to the next, a 404 poll that raises, a failed operation posted again, a failed operation with no error, and a ref whose stored file expired, read through the file selector, rebuilt at the same version, at a newer one, or refused with the newest version read from the versions route: the first version it lists with no `base_content_version`, the newest plain build, so a derived version listed before it is skipped. - `client-codes.json` and its schema: the 23 codes a client sets on the errors it raises itself, each with its error class, retryable flag and condition, disjoint from `actions.json` and `registries/codes.json`, and the `client_codes()` and `client_codes_schema()` accessors. Every scenario error the client raises itself carries its code, and `refusal_without_code` names a refusal whose body carries no code. -- `proxy-codes.json` and its schema, the website proxy's `rate_limited` and `transient_unavailable` codes with their statuses, which the proxy sends in problem documents of the shape every refusal has, both retryable, with `Retry-After` on the 429, so a client reads one refusal shape, and `warning-codes.json` and its schema, the 6 codes a client sets on the warnings it issues itself, with the `proxy_codes()` and `warning_codes()` accessors. +- `proxy-codes.json` and its schema, the website proxy's `rate_limited` and `transient_unavailable` codes with their statuses, which the proxy sends in problem documents of the shape every refusal has, both retryable, with `Retry-After` on the 429, so a client reads one refusal shape, and `warning-codes.json` and its schema, the 7 codes a client sets on the warnings it issues itself, with the `proxy_codes()` and `warning_codes()` accessors. - `ref_digest_changed` in `warning-codes.json`, a `DfoWarning` for a ref member whose `content_version` serves a file with another digest than the ref's, because the server rendered the same data again. The client returns the data, with no error under strict. - `timed_out` in `actions.json` and `registries/codes.json`, the no_data reason Search gives a part it left unresolved at the request's time limit, with action `ask_data_question` and gap kind `timed_out`, which `actions.schema.json` now admits. The client-side `timeout` code, a transport timeout, is unchanged. - A no_data part's gap kind comes from its own reason, whatever the answer's `diagnostics.timed_out` says. `answer-timed-out` gives its part the reason `timed_out`, and `answer-timed-out-keeps-reasons` pins a part that keeps `variable_not_resolved` as a `not_understood` gap in a timed-out answer. @@ -31,5 +31,13 @@ All notable changes to the `dfo-spec-kit-client-contract` Python package will be - The `base-url` rule states the whole validity rule, and seven new vectors pin it: no query, fragment, `.` or `..` segment, character outside visible ASCII, empty userinfo or non-decimal IPv4 host; trailing slashes dropped; a relative URL valid only in a TypeScript client inside a browser page. - The `base-url` rule also refuses a `.` or `..` segment written with `%2e`, a backslash, and a port outside 1 to 65535 or written other than in decimal digits, an empty one included, so a client on Python's `urlsplit` and one on the WHATWG URL parser accept the same values. Ten vectors pin them. - The `file-selector` rule and its vectors: the file selector's query holds the `format`, `bundle` and `layout` of a ref member's rendering, the keys `query_keys` lists, in that order, and leaves out `without_party_totals`, which the selector refuses with 400 `request_invalid` because the version records its own cut. `fetch-ref-expired-file-selector` and `fetch-ref-artifact-unlisted` ask that query, and the validator and the capture build it from `query_keys`. +- Ref format 2 in `dataset-ref.schema.json`: `format_version: 2` and one or more derived members, each `{parts, sentence, spec: {spec_id, recipe: {normal_form_hash, replay_key}}, content_version, output_digest, roots, execution_identity, resolved_periods, artifact}`, with an `ops_` id, an `ocv_` version, `rnf_` and `rpk_` recipe hashes, an `out_` output digest, the root dataset versions with the renderings they were read with, and each period stated relative to the latest with the year it resolved to. Format 1 is unchanged and holds dataset members only, and format 2 holds no dataset member. A derived member's `parts` is empty. A scenario's fetch-by-ref step admits a format 1 ref only, and a format 2 ref fails there on its `format_version`. Nine new ref examples, two accepted and seven rejected, each derived identity computed by the dfo-apis analysis runner. The validator checks a derived member's roots against the scenario it names, each root's rendering as the posted one in the frame layout, and that the derived members of the ref examples and the `output-comparison` vectors agree wherever one identity fixes another. +- A derived member's root records its rendering in `root_rendering`, the artifact selector's members and `groupby`, the sum DFO read the root with as `options.groupby` states it: `by`, one or more distinct column names, `op` `sum`, and `levels`, a depth from 0 to 99 for each kept hierarchical axis, absent when none is set. A breakdown or a sum over StatVars reads a root that way. The artifact's `rendering` admits no `groupby`. `ref-derived-grouped-root` is accepted, its `by` sorted as DFO records it and its identities computed by the dfo-apis analysis runner over the scenario's rows summed to their purpose's sector code, and `ref-derived-root-groupby-op-rejected` and `ref-artifact-groupby-rejected` are rejected. The validator checks a summed root's rendering against its scenario's build without its `groupby`, which no recorded build posts. +- The analysis and publication rules DFO applies, each with vectors and a reference implementation in `tests/test_analysis_rules.py`: `output-digest`, the `out2` digest of a derived frame's values with the `stat_var`, members, period and unit each belongs to, without `measurementMethod`, `provenance`, `status` or a unit's `recipe`, reading the variable `compare`, `sum_of`, `share_of` or `quotient_of` writes under an `mv_` name hashed from its verb and its inputs' variables, 35 characters at any depth, so an engine release or upstream rebuild that moves no number keeps the digest, with 22 canonicalisation vectors, each minted name given with its name document and one naming a `quotient_of` over a `sum_of` by the inner step's name; `output-comparison`, how `==` and `same_data` compare two refs, with 12 vectors; `operation-outputs`, `share_of` in per cent under `Percent`, `join` in long format by `stat_var`, `compare` as data minus other, `sum_of` as data plus other, and `growth` as a fraction in `value` with no unit, leaving out a member with no value at an end of its window, a relative change over a zero or sign-crossing base and a change that is not finite, and refusing two rows of one member in one period with `role_ambiguous`, with 10 vectors; `rolling-composition`, one latest period per recipe tree, refusing rolling steps over different populations with `rolling_populations_differ` even when they settle on one year, with 6 vectors; and `publication`, a share's numbered slots that keep their URLs across a live replace, a snapshot that refuses `share_not_replaceable`, and the write token checks, with 8 vectors. +- `actions.json` rows for `allowance_exhausted` (`wait_for_reset`), `rolling_populations_differ`, `several_members`, `share_not_replaceable` and `write_token_required` (`fix_request`), and `write_token_invalid` (`stop`). +- The `stop` action, class `PermissionDeniedError`, for a refusal no permission or credential the caller can obtain clears: `write_token_invalid`, since a share takes only the write token issued when it was created and a share created without one takes none. +- The retry rules: a refusal whose code's action is `wait_for_reset` is never retried and raises `QuotaExceededError` at once, whatever its document and status say, and a refusal whose `Retry-After` is longer than 60 seconds (`RETRY_AFTER_MAX_S`) raises at once with that wait, a refused poll included. `search-retry-after-past-cap` pins the cap. +- `series_left_out` in `warning-codes.json`, class `SeriesLeftOutWarning`, and the `series-left-out` rule with 8 vectors: a fetch of derived members issues it once when any member's derived manifest lists left-out series in `left_out` or says `left_out_complete` false, a null `left_out` issues nothing, and a strict fetch raises it in place of its data. `tools/validate_client_contract.py` takes the code from the rule. +- `actions.json` rows (`fix_request`) for the 34 refusal codes the analysis engine serves on `datasets.analysis`, including `compare_no_key_matched` and `sum_of_no_key_matched`. [unreleased]: https://github.com/DevelopmentFinanceObservatory/spec-kit/commits/main/python diff --git a/python/client-contract/README.md b/python/client-contract/README.md index 312af1a..0a0aa87 100644 --- a/python/client-contract/README.md +++ b/python/client-contract/README.md @@ -48,9 +48,9 @@ uv run ty check src/dfo_client_contract | `warning-codes.schema.json` | The shape of `warning-codes.json` | | `scenario.schema.json` | One scenario record | | `scenarios/` | The scenario records, and the binary response bodies they name | -| `dataset-ref.schema.json` | The frozen `DatasetRef` format | +| `dataset-ref.schema.json` | The frozen `DatasetRef` formats, 1 and 2 | | `refs/` | Accepted and rejected `DatasetRef` examples | -| `rules/` | The client rules, one vector file each, with `rules.schema.json` and the rules' text | +| `rules/` | The client, analysis and publication rules, a vector file each, with schema and text | | Function | Returns | | ----------------- | ----------------------------------------------------------- | @@ -92,12 +92,13 @@ The required members are each part's `status` and its payload (`spec`, `candidat | `take_newest_version` | `VersionUnavailableError` | Takes the newest version | | `log_in` | `AuthenticationError` | Logs in | | `request_scope` | `PermissionDeniedError` | Asks for the permission the call needs | +| `stop` | `PermissionDeniedError` | Stops, as nothing it can obtain allows the call | | `wait` | `RateLimitError` | Waits, then retries | | `wait_for_reset` | `QuotaExceededError` | Waits for the quota to reset | | `report` | `ServerError` | Reports the trace id, after the client's retries | | `upgrade` | `ContractError` | Upgrades the client | -A code that maps to `no_data_reason` also carries `gap_kind`, the kind of gap a no_data part with that reason becomes: `choice`, `no_data`, `not_understood`, `unsupported` or `timed_out`. `timed_out` is a part Search left unresolved at its time limit. Sending the question again, or a narrower one, may answer it. `choose` goes with `choice` and only with it. `choose` and `ask_data_question` name a next step inside an answer, so a code that also reaches a problem document takes another action. The website proxy answers its own refusals with problem documents of the same shape, with `type`, `title`, `status`, `detail`, `code`, `retryable` and `trace_id`, the trace id repeated in the `x-dfo-trace-id` header. `proxy-codes.json` lists their codes with their status, `rate_limited` at 429, which also sends `Retry-After`, and `transient_unavailable` at 502, both retryable. The validator refuses a problem document whose code is neither an active registry code nor a proxy code at that status. No proxy code has a row here, so a client raises a proxy refusal with the class its status maps to: `RateLimitError` for `rate_limited`, which `search-proxy-rate-limited` pins, and `ServerError` for `transient_unavailable`, which `search-proxy-unavailable` pins. `capability_unavailable` is a well-formed request for a format, reshape or conversion class the deployment does not enable, answered 501. Its action is `fix_request`, since the caller asks for another one, and `fetch-spec-capability-unavailable` pins it. +A code that maps to `no_data_reason` also carries `gap_kind`, the kind of gap a no_data part with that reason becomes: `choice`, `no_data`, `not_understood`, `unsupported` or `timed_out`. `timed_out` is a part Search left unresolved at its time limit. Sending the question again, or a narrower one, may answer it. `choose` goes with `choice` and only with it. `choose` and `ask_data_question` name a next step inside an answer, so a code that also reaches a problem document takes another action. The website proxy answers its own refusals with problem documents of the same shape, with `type`, `title`, `status`, `detail`, `code`, `retryable` and `trace_id`, the trace id repeated in the `x-dfo-trace-id` header. `proxy-codes.json` lists their codes with their status, `rate_limited` at 429, which also sends `Retry-After`, and `transient_unavailable` at 502, both retryable. The validator refuses a problem document whose code is neither an active registry code nor a proxy code at that status. No proxy code has a row here, so a client raises a proxy refusal with the class its status maps to: `RateLimitError` for `rate_limited`, which `search-proxy-rate-limited` pins, and `ServerError` for `transient_unavailable`, which `search-proxy-unavailable` pins. `capability_unavailable` is a well-formed request for a format, reshape or conversion class the deployment does not enable, answered 501. Its action is `fix_request`, since the caller asks for another one, and `fetch-spec-capability-unavailable` pins it. `write_token_invalid` takes `stop`: a share takes only the write token issued when it was created, and a share created without one takes none, so no scope or new token lets the caller replace or delete it. `allowance_exhausted` takes `wait_for_reset`, and a client raises it at once as `QuotaExceededError` with its `Retry-After`, the time until the allowance resets, as the retry rules below state. A client maps a refusal whose code has no row, or that carries no code, on its HTTP status: @@ -118,7 +119,7 @@ An answer's gaps follow from its parts. A candidates part is a `choice` gap, its ## Client codes -`client-codes.json` lists, sorted by code, each code a client sets on an error it raises when no problem document carries one: the `code`, the error `class` it raises, `retryable`, and a one-line `condition`. `retryable` is the flag both clients generate onto the error: `deadline_exceeded`, `download_failed` and `network_error` are retryable, `digest_mismatch` and the rest are not, and `refusal_without_code` is retryable as its status is, which its row gives as `by_status`. Both clients set an error's `code` from this list, so every error a client raises carries a code. A refusal keeps its problem document's `code`, the website proxy's included. A refusal whose body carries no code raises `refusal_without_code` with the class its HTTP status maps to, which its row gives as `by_status`. No other row does. A problem document without a code still gives the error its `retryable` and `trace_id`, so a 503 whose body says `retryable: false` raises at once, which `search-refused-without-code` pins. Every refusal's trace id is the response's trace id, as the trace-id rule under Scenarios gives it. `search-refused-without-code` pins the body's winning over the header, and `search-unauthorized` the header's for a refusal with no body. No client code is a code in `actions.json` or `registries/codes.json`, and each class is one the scenario schema names. Each reason the key-lookup rule raises, `key_empty`, `profile_not_found`, `profile_without_key` and `credentials_file_invalid`, is a row of class `AuthenticationError`, as are `key_invalid`, a key outside visible ASCII, and `credentials_file_unreadable`, a credentials file that exists but cannot be read. `header_invalid`, a header value with a control character or a caller's header named `X-DFO-Client`, `insecure_transport`, a key over plain http to a non-loopback host, and `base_url_invalid`, a base URL that is empty, not absolute http or https, or carries userinfo, are `InvalidRequestError` rows. `response_too_large`, a decoded body past its cap, is a `ServerError` row. None of these six is retryable. The list holds no warning. `warning-codes.json` lists, sorted by code, each code a client sets on a warning it issues itself, with its class and condition: `ref_digest_changed`, a `DfoWarning`, for a ref member whose `content_version` serves a file with another digest than the ref's, because the server rendered the same data again, where the client returns the data; `route_deprecated`, a `DfoDeprecationWarning`, for a `Deprecation` or `Sunset` header; `server_warning`, a `DfoWarning`, for an `x-dfo-warning` header; and `key_checksum_mismatch`, `key_not_base62` and `key_unknown_prefix`, each a `DfoWarning`, for a key that fails the key-format rule with that reason. Both clients generate their warning codes from it and issue no other. No warning code is a code in another table or in `registries/codes.json`. A `SearchWarning` carries the code of the one way the readings differ from the question, a departure's or a warning's code or the verb of an operation Search did not run, and a null code when they differ in several ways. A `DataWarning` carries the manifest warning's code. +`client-codes.json` lists, sorted by code, each code a client sets on an error it raises when no problem document carries one: the `code`, the error `class` it raises, `retryable`, and a one-line `condition`. `retryable` is the flag both clients generate onto the error: `deadline_exceeded`, `download_failed` and `network_error` are retryable, `digest_mismatch` and the rest are not, and `refusal_without_code` is retryable as its status is, which its row gives as `by_status`. Both clients set an error's `code` from this list, so every error a client raises carries a code. A refusal keeps its problem document's `code`, the website proxy's included. A refusal whose body carries no code raises `refusal_without_code` with the class its HTTP status maps to, which its row gives as `by_status`. No other row does. A problem document without a code still gives the error its `retryable` and `trace_id`, so a 503 whose body says `retryable: false` raises at once, which `search-refused-without-code` pins. Every refusal's trace id is the response's trace id, as the trace-id rule under Scenarios gives it. `search-refused-without-code` pins the body's winning over the header, and `search-unauthorized` the header's for a refusal with no body. No client code is a code in `actions.json` or `registries/codes.json`, and each class is one the scenario schema names. Each reason the key-lookup rule raises, `key_empty`, `profile_not_found`, `profile_without_key` and `credentials_file_invalid`, is a row of class `AuthenticationError`, as are `key_invalid`, a key outside visible ASCII, and `credentials_file_unreadable`, a credentials file that exists but cannot be read. `header_invalid`, a header value with a control character or a caller's header named `X-DFO-Client`, `insecure_transport`, a key over plain http to a non-loopback host, and `base_url_invalid`, a base URL that is empty, not absolute http or https, or carries userinfo, are `InvalidRequestError` rows. `response_too_large`, a decoded body past its cap, is a `ServerError` row. None of these six is retryable. The list holds no warning. `warning-codes.json` lists, sorted by code, each code a client sets on a warning it issues itself, with its class and condition: `ref_digest_changed`, a `DfoWarning`, for a ref member whose `content_version` serves a file with another digest than the ref's, because the server rendered the same data again, where the client returns the data; `route_deprecated`, a `DfoDeprecationWarning`, for a `Deprecation` or `Sunset` header; `server_warning`, a `DfoWarning`, for an `x-dfo-warning` header; `series_left_out`, a `SeriesLeftOutWarning`, for a derived frame that left series out for missing data, where the client returns the frame; and `key_checksum_mismatch`, `key_not_base62` and `key_unknown_prefix`, each a `DfoWarning`, for a key that fails the key-format rule with that reason. Both clients generate their warning codes from it and issue no other. No warning code is a code in another table or in `registries/codes.json`. A `SearchWarning` carries the code of the one way the readings differ from the question, a departure's or a warning's code or the verb of an operation Search did not run, and a null code when they differ in several ways. A `DataWarning` carries the manifest warning's code. ## Scenarios @@ -180,12 +181,13 @@ The records also fix these client rules: - A ref's version is found in this order: the snapshot route and the files route at the stored digest; the file selector, `GET /datasets/{id}/{content_version}/file` with the query the file-selector rule in `rules/README.md` builds from the ref's rendering, which leaves out `without_party_totals`; and a build of the ref's spec in the ref's rendering, which must answer the ref's `spec_id` and the same `content_version`. Both recovery steps keep the ref's identity, so a recovered member carries the ref's `spec_id` and `content_version`: a snapshot manifest whose `definition.spec_id` or `content_version` is not the ref's, or a rebuild manifest whose `definition.spec_id` is not the ref's, raises `ContractError` `contract_violation` before any download, which `fetch-ref-expired-rebuild-other-spec` pins for the rebuild. A snapshot refusal other than `artifact_expired` ends the lookup, so `snapshot_not_found` raises `NotFoundError` with no further request. A files route or selector refusal of `artifact_expired`, `artifact_not_found` or `not_found` takes the next step, and a selector file of another digest is not the stored file. A snapshot manifest whose artifacts list no artifact of the ref member's digest sends the client to the selector for the ref member's own rendering, and a file that hashes to that digest gives the dataset, which `fetch-ref-artifact-unlisted` pins. A rebuild that answers another version, or that raises `InvalidRequestError`, `NotFoundError` or `VersionUnavailableError`, ends in `VersionUnavailableError` with the first refusal's code, or `version_unavailable` when nothing refused. Its `newest_version` is the version the rebuild answered, or else the newest stored version, as a `newest_version` step finds it: the first version `GET /datasets/{id}/versions?limit=100` lists with a null `base_content_version`, the newest plain build, read page by page through `&cursor=` with each page's `next_cursor` until one is listed. The route lists versions newest first and needs no key. A version with a `base_content_version` is derived from a plain build, a groupby collapse or a party-totals cut, and is skipped, which `fetch-ref-newest-version` pins. The newest version is null when the list ends without a plain version or the route refuses with `NotFoundError`. It names the newest plain build, which a fetch with the default party totals rule answers only when the cut leaves nothing out; when the cut leaves rows out, the fetch answers a version whose `base_content_version` is that build. So a client does not present `newest_version` as comparable to the `content_version` of a cut ref or a cut fetch: two different versions there need not mean newer data. Any other refusal of the versions route raises, and a page whose `versions` is not a list of versions each with a `content_version` and a `base_content_version` that is a string or null, or whose `next_cursor` is neither a string nor null, raises `ContractError` `contract_violation`, which `newest-version-unreadable-page` pins. - A build answers 200 with the manifest, or 202 with an operation the client polls at `/datasets/operations/{operation_id}`, the id of the operation it holds. The 202 reads `pending` or `running`, because a submission starts a build or joins one under way, and a `done` operation left by an earlier submission of a rejected head is built again. A deterministic failure answers a 202 that reads `failed`. `fetch-spec-async-build` pins `pending` and `fetch-spec-accepted-running` pins `running`. A POST for a build identical to one that is pending or running returns that same operation, because the server derives the operation id from the request: dfo-apis `src/datasets_api/operation_store.py` `compute_operation_id` hashes the request identity, the origin catalogue build id and the rendering. A client that stops waiting for a build can post the same build again to join it. A mint's build answers 202. A record whose mint answers 200 fits no mint step. Before each poll it waits the `Retry-After` of the refusal that answered the last poll, whatever its status, as every retry of a refused request waits; else the last operation resource's `retry_after_ms`, else the `Retry-After` of the last answered poll, the response that carried that resource, else a backoff of 1, 2, 4 and then 8 seconds that advances one step each time it is used, a poll after a refusal included. `fetch-spec-poll-pacing-changes` pins a `retry_after_ms` that changes from one poll to the next, so a client waits each poll's own value. - A retryable 429 poll spends no retry, and a 429 whose body says it is not retryable raises at once, as a refusal's `retryable` outranks its status, which `fetch-spec-poll-rate-limited-not-retryable` pins. Any other retryable refused poll spends one of the two retries, which a poll that answers restores, and a refusal that is not retryable raises. `fetch-spec-poll-refused` pins two 503s polled again after an answered poll, `fetch-spec-poll-refused-backoff` a 503 with no `Retry-After` among polls no response paces, after which the backoff advances, `fetch-spec-poll-refused-retry-after` a 503 whose `Retry-After` paces the next poll, and `fetch-spec-poll-not-found` a 404 that raises at once. A poll that answers a failed operation whose error is retryable posts the same build once more and polls the operation it starts; a second failure, an error that is not retryable, and a resumed operation, whose build request the client does not hold, raise the operation's error. A refused poll whose code is `operation_failed` is a failed operation with that error. A failed operation whose resource carries no `error` raises `ServerError` with `operation_failed`, retryable by status 500 as a refusal without a document is, which `operation-failed-without-error` pins. A 202 or resumed operation that already reads `failed` raises its error before any poll. -- Any other refusal is retried when it is retryable: as its problem document's `retryable` says, else when its status is 408, 429, 500, 502, 503 or 504. A client makes two retries by default, and the scenarios assume that default. A build's POST shares those two retries with every other refusal of it, and once a 504 cuts the build off the client posts it once more only, whatever that post answers: a build refused 504 then 503 raises the 503, a build refused 503 then 504 is posted once more after the 504, and a second 504 raises. `fetch-spec-504-then-503`, `fetch-spec-503-then-504` and `fetch-spec-504-twice` pin the three. A refusal that is not retryable, or one past the retries, raises its error. -- Each retry waits the refusal's `Retry-After` when it carries one, and otherwise the client's backoff for that attempt: 0.5 seconds before the first retry, doubling to at most 8. The build's 504 cut-off retry follows the same rule. `retry_waits` pins these waits, and `fetch-spec-504-retry-after` pins a cut-off that carries `Retry-After`. +- Any other refusal is retried when it is retryable: never when its code's action is `wait_for_reset`, whatever its document and status say, else as its problem document's `retryable` says, else when its status is 408, 429, 500, 502, 503 or 504. A client makes two retries by default, and the scenarios assume that default. A build's POST shares those two retries with every other refusal of it, and once a 504 cuts the build off the client posts it once more only, whatever that post answers: a build refused 504 then 503 raises the 503, a build refused 503 then 504 is posted once more after the 504, and a second 504 raises. `fetch-spec-504-then-503`, `fetch-spec-503-then-504` and `fetch-spec-504-twice` pin the three. A refusal that is not retryable, or one past the retries, raises its error. +- Each retry waits the refusal's `Retry-After` when it carries one, and otherwise the client's backoff for that attempt: 0.5 seconds before the first retry, doubling to at most 8. The build's 504 cut-off retry follows the same rule. A client waits a `Retry-After` of at most 60 seconds (`RETRY_AFTER_MAX_S`). A refusal whose `Retry-After` is longer raises at once with that wait on the error, a refused poll included, so no call sleeps for minutes unasked. `retry_waits` pins these waits, `fetch-spec-504-retry-after` pins a cut-off that carries `Retry-After`, and `search-retry-after-past-cap` a retryable 503 whose `Retry-After` of 61 raises with no retry. - A 401 resolves the key again by the key-lookup rule. When the key changed, the client sends the request once more at once, with no wait and spending no retry. When it did not, or the second answer is a 401 too, the 401 raises `AuthenticationError`. A replay's key never changes, so `search-unauthorized` pins the second case and no scenario the first. - A `request(method, path)` whose `path` is not a path on the base URL, because it starts with `//` or names a scheme, raises `InvalidRequestError` `invalid_argument` before any request. No scenario step makes such a request. - A 2xx response of a JSON route with no JSON body raises `ContractError` `unexpected_response`. `search-no-json-body` pins it for Search. - A `Deprecation` or `Sunset` header on an API response, either alone, issues `route_deprecated` once per process per distinct pair of their values, and an `x-dfo-warning` header issues `server_warning` once per process per message. A signed fetch's headers issue nothing. `search-sunset-warning` pins a Sunset header alone, issued once over two responses that carry the same value, and `search-server-warning` pins a repeated `x-dfo-warning` message issued once, a new message issued again, and a Deprecation beside the same Sunset issued as a new pair. +- A fetch of derived members issues `series_left_out`, a `SeriesLeftOutWarning`, once when the recipe anchor of any member lists left-out series (`Recipe.anchor.left_out` non-empty in the served document): series the rolling anchor dropped because they hold no row at the anchor period. The client returns the frame. A strict fetch raises the warning in place of its data. A recipe with no rolling period has a null anchor and issues nothing. No scenario serves a derived member yet, so the `series-left-out` rule's vectors in `rules/` pin it. - Python synchronous platform name resolution can exceed the following timeout and deadline limits. Asynchronous name resolution is cancellable. Timeouts follow the clients' design and are not replayed, since the replay advances the clock only by the waits the client asks for: the client's `timeout`, 30 s by default, bounds one request attempt; `search(timeout=60)` bounds the whole search, each attempt reading until that deadline, and None means no limit; `fetch` waits for a build with no whole-call deadline by default, and `fetch(timeout=60)` bounds the whole fetch, each download getting the time left on the fetch's deadline; a download that delivers no byte for 30 s raises the retryable `network_error`; a synchronous build POST, and mint's POST, read for up to 150 s within any outer deadline; `result(timeout=1800)` bounds the wait, and None means no limit. A poll attempt that hangs past the per-attempt timeout spends a retry, and a deadline that passes during a body read raises `DfoTimeoutError` `deadline_exceeded`. In TypeScript an infinite remaining time arms no timer, and a finite one above 2^31-1 ms is clamped. No scenario step takes a timeout. ## Recording @@ -213,11 +215,11 @@ A record's `origin.base_url` is the base URL the run was given, a local app's in ## DatasetRef -`dataset-ref.schema.json` is the frozen format a client writes for `data.ref` and reads back in any language: `format_version: 1` and one or more `members`, each `{parts, sentence, spec: {spec_id, definition}, content_version, artifact: {digest, rendering}}`. `parts` holds the variable text of each answer part the member answers, in query order, and is empty for a spec fetched outside an answer. `definition` is the server's definition unmodified, so it holds the `schema_version`, `spec_id` and `identity` the served `CanonicalDefinition` requires and no other member, and a rebuild posts it as it is. The format takes any integer `schema_version` and leaves the version to the server, which checks it when a rebuild posts the definition. `rendering` is the selector a client matches an artifact on, `{format, bundle}` with `layout: "frame"` for the frame layout and `without_party_totals: true` for a fetch that left party totals out, never the server's rendering object. A ref records the rendering its fetch posted, so a member fetched with the default records `without_party_totals: true` whether or not the cut left out a row, and a member fetched with party totals records no such key. The `without_party_totals` key is written only when true, so a plain rendering omits it and `false` is not a valid value. Beyond the schema, each member's `definition.spec_id` equals its `spec.spec_id`, and members hold distinct `spec_id`s. `ref-bare-definition-rejected`, `ref-definition-without-identity-rejected` and `ref-definition-without-schema-version-rejected` pin the definition's required members, and `ref-definition-extra-member-rejected` a member it does not admit. `ref-without-party-totals` is the cut rendering a default fetch records, `ref-one-member` the plain rendering a fetch with party totals records, and `ref-without-party-totals-false-rejected` pins `false`. `refs/` holds accepted examples and rejected ones, each `{format_version, name, description, scenario, ref}` with `expect: "accept"` or `error`, `{by: "schema"}` or `{by: "rule", reason}`, the reason `definition_spec_id_mismatch` or `repeated_spec_id`. `scenario` names the recorded scenario that serves the ref's members, and is left out of a ref with no members. Each member's `spec_id`, `parts`, `content_version` and `sentence` are those of a dataset member of that scenario, and its artifact digest is the digest of a file the scenario downloads, so a re-recording that changes a member also fails the refs that still name the old one. Each member of an example the schema accepts records the rendering the scenario's build of its spec posted as `options`. +`dataset-ref.schema.json` is the frozen format a client writes for `data.ref` and reads back in any language. Format 1 is `format_version: 1` and one or more dataset `members`, each `{parts, sentence, spec: {spec_id, definition}, content_version, artifact: {digest, rendering}}`. `parts` holds the variable text of each answer part the member answers, in query order, and is empty for a spec fetched outside an answer. `definition` is the server's definition unmodified, so it holds the `schema_version`, `spec_id` and `identity` the served `CanonicalDefinition` requires and no other member, and a rebuild posts it as it is. The format takes any integer `schema_version` and leaves the version to the server, which checks it when a rebuild posts the definition. `rendering` is the selector a client matches an artifact on, `{format, bundle}` with `layout: "frame"` for the frame layout and `without_party_totals: true` for a fetch that left party totals out, never the server's rendering object. A ref records the rendering its fetch posted, so a member fetched with the default records `without_party_totals: true` whether or not the cut left out a row, and a member fetched with party totals records no such key. The `without_party_totals` key is written only when true, so a plain rendering omits it and `false` is not a valid value. Beyond the schema, each member's `definition.spec_id` equals its `spec.spec_id`, and members hold distinct `spec_id`s. `ref-bare-definition-rejected`, `ref-definition-without-identity-rejected` and `ref-definition-without-schema-version-rejected` pin the definition's required members, and `ref-definition-extra-member-rejected` a member it does not admit. `ref-without-party-totals` is the cut rendering a default fetch records, `ref-one-member` the plain rendering a fetch with party totals records, and `ref-without-party-totals-false-rejected` pins `false`. `refs/` holds accepted examples and rejected ones, each `{format_version, name, description, scenario, ref}` with `expect: "accept"` or `error`, `{by: "schema"}` or `{by: "rule", reason}`, the reason `definition_spec_id_mismatch` or `repeated_spec_id`. `scenario` names the recorded scenario that serves the ref's members, and is left out of a ref with no members. Each member's `spec_id`, `parts`, `content_version` and `sentence` are those of a dataset member of that scenario, and its artifact digest is the digest of a file the scenario downloads, so a re-recording that changes a member also fails the refs that still name the old one. Each member of an example the schema accepts records the rendering the scenario's build of its spec posted as `options`. Format 2 is `format_version: 2` and one or more derived `members`, each the frame an analysis recipe computed, and holds no dataset member, so a ref of dataset members has one spelling, format 1. A derived member is `{parts, sentence, spec: {spec_id, recipe: {normal_form_hash, replay_key}}, content_version, output_digest, roots, execution_identity, resolved_periods, artifact: {digest, rendering}}`. `spec_id` is the recipe's `ops_` id and `content_version` its `ocv_` version. `normal_form_hash` (`rnf_`) names the computation whatever data it ran on, and `replay_key` (`rpk_`) names it on these roots under this execution identity. `output_digest` (`out_`) is the digest of the frame's output that `rules/README.md` defines. `roots` lists each dataset version the recipe ran on, `{spec_id, content_version, rendering}` with the rendering the recipe read, a `spec_` id with a `cv_` version and never a derived frame, since a recipe's normal form replaces each derived input by its own recipe. A root's rendering holds the selector's members and, for a root read summed, such as by a breakdown or a sum over StatVars, `groupby`, the `options.groupby` DFO read it with: `{by, op: "sum"}`, `by` one or more distinct column names, with `levels` mapping each kept hierarchical axis it rolls up to a depth from 0 to 99. An artifact's `rendering` admits no `groupby`. `execution_identity` names the engine, frame grammar and reduction policy versions that ran it, joined by `|`, such as `0.9.0|1.1.0|0.2.0`. `resolved_periods` lists each period the recipe states relative to the latest period, `{anchor: "latest", offset, rule: "half1", period}`, with the year it resolved to, and is empty for a recipe that states none. DFO computes every one of these members, and a client copies them from the server's answer and computes none. `parts` is empty for a derived member, which no answer part names. A client that reads format 1 refuses format 2, so clients that read format 2 ship before any service writes it. `==` and `same_data` compare refs as `rules/README.md` states under output comparison. `ref-derived-member`, `ref-derived-member-fixed-window` and `ref-derived-grouped-root` are accepted derived refs, the first with two resolved periods, the second with none, and the third with a root read summed, and `ref-derived-in-format-1-rejected`, `ref-dataset-member-in-format-2-rejected`, `ref-derived-without-output-digest-rejected`, `ref-derived-root-without-rendering-rejected`, `ref-derived-root-groupby-op-rejected`, `ref-artifact-groupby-rejected` and `ref-derived-period-after-latest-rejected` pin what the schema refuses. `repeated_spec_id` holds across derived members too, which `ref-derived-repeated-spec-id-rejected` pins, and `definition_spec_id_mismatch` concerns dataset members alone. A derived example's `scenario` serves its roots: each root's `spec_id` and `content_version` are a dataset member's of that scenario, and its rendering the one the scenario's build posted, apart from a summed root's `groupby`, which no recorded build posts. No scenario serves a derived member itself, so its own identifiers and artifact digest are checked by the schema alone. ## Rules -`rules/README.md` states the shared rules for credentials, key lookup and format, the base URL, disk caching, the order a client presents a dataset's columns and rows in, the keys a caller narrows a spec's roles by, the file selector's query, and the `X-DFO-Client` header that identifies the client to the API. A rule applies where the client and runtime implement its capability. Python implements disk caching; TypeScript currently keeps no disk cache, so `cache-dir` and `cache-layout` apply to Python. Credentials-file rules apply to runtimes with local credentials support. Each rule's vectors are in `rules/.json`. +`rules/README.md` states the shared rules for credentials, key lookup and format, the base URL, disk caching, the order a client presents a dataset's columns and rows in, the keys a caller narrows a spec's roles by, the file selector's query, the `X-DFO-Client` header that identifies the client to the API, and when a fetch of derived members warns that series were left out. It also states the rules DFO applies and a client reads the results of: the output digest of a derived frame, how `==` and `same_data` compare refs, the units and shapes of the operations' outputs, how the rolling steps of one recipe share one latest period, and how a share's cards are published, replaced and taken down. A rule applies where the client and runtime implement its capability. Python implements disk caching; TypeScript currently keeps no disk cache, so `cache-dir` and `cache-layout` apply to Python. Credentials-file rules apply to runtimes with local credentials support. Each rule's vectors are in `rules/.json`. ## Validation @@ -227,5 +229,5 @@ A record's `origin.base_url` is the base URL the run was given, a local app's in uv run ../../tools/validate_client_contract.py # Replay the rule vectors through a reference implementation of each rule -uv run pytest tests/test_rules.py +uv run pytest tests/test_rules.py tests/test_analysis_rules.py ``` diff --git a/python/client-contract/pyproject.toml b/python/client-contract/pyproject.toml index d613df2..a301906 100644 --- a/python/client-contract/pyproject.toml +++ b/python/client-contract/pyproject.toml @@ -45,6 +45,7 @@ dev = [ "pre-commit>=4.6.0", "jsonschema>=4.23", "referencing>=0.36", + "rfc8785>=0.1.4", ] [tool.ruff] diff --git a/python/client-contract/src/dfo_client_contract/actions.json b/python/client-contract/src/dfo_client_contract/actions.json index 8474193..cb28717 100644 --- a/python/client-contract/src/dfo_client_contract/actions.json +++ b/python/client-contract/src/dfo_client_contract/actions.json @@ -1,6 +1,10 @@ { "format_version": 1, "codes": [ + { + "code": "allowance_exhausted", + "action": "wait_for_reset" + }, { "code": "artifact_expired", "action": "take_newest_version" @@ -46,6 +50,18 @@ "code": "column_unknown", "action": "fix_request" }, + { + "code": "compare_no_key_matched", + "action": "fix_request" + }, + { + "code": "comparison_periods_disjoint", + "action": "fix_request" + }, + { + "code": "composed_unit_sum_unsupported", + "action": "fix_request" + }, { "code": "conflicting_requested_options", "action": "fix_request" @@ -54,6 +70,22 @@ "code": "constraint_violation", "action": "fix_request" }, + { + "code": "coverage_insufficient", + "action": "fix_request" + }, + { + "code": "cross_frame_facet_mismatch", + "action": "fix_request" + }, + { + "code": "cross_frame_unit_mismatch", + "action": "fix_request" + }, + { + "code": "cross_frame_unit_unbound", + "action": "fix_request" + }, { "code": "default_facet_undefined", "action": "fix_request" @@ -67,6 +99,14 @@ "action": "ask_data_question", "gap_kind": "no_data" }, + { + "code": "derived_input_ordinal", + "action": "fix_request" + }, + { + "code": "derived_input_unusable", + "action": "fix_request" + }, { "code": "directional_children_unsupported", "action": "fix_request" @@ -123,6 +163,10 @@ "code": "facet_binding_unsupported", "action": "fix_request" }, + { + "code": "facet_heterogeneous", + "action": "fix_request" + }, { "code": "facet_not_available", "action": "fix_request", @@ -132,14 +176,30 @@ "code": "facet_probe_ceiling", "action": "fix_request" }, + { + "code": "filter_predicate_type_mismatch", + "action": "fix_request" + }, + { + "code": "filter_unit_conversion_unsupported", + "action": "fix_request" + }, { "code": "frame_not_refreshable", "action": "fix_request" }, + { + "code": "frame_too_large", + "action": "fix_request" + }, { "code": "frame_unavailable", "action": "check_id" }, + { + "code": "growth_gap_year", + "action": "fix_request" + }, { "code": "identity_collision", "action": "report" @@ -156,6 +216,14 @@ "code": "invalid_role_ref", "action": "fix_request" }, + { + "code": "join_no_common_key_columns", + "action": "fix_request" + }, + { + "code": "join_output_collision", + "action": "fix_request" + }, { "code": "member_not_resolvable", "action": "fix_request" @@ -172,6 +240,10 @@ "code": "mixed_peers_anchor_types", "action": "fix_request" }, + { + "code": "mixed_time_grain", + "action": "fix_request" + }, { "code": "no_matching_slice", "action": "ask_data_question", @@ -186,6 +258,10 @@ "code": "node_not_held", "action": "check_id" }, + { + "code": "non_finite_result", + "action": "fix_request" + }, { "code": "not_found", "action": "check_id" @@ -194,6 +270,10 @@ "code": "observations_unavailable", "action": "fix_request" }, + { + "code": "op_spec_invalid", + "action": "fix_request" + }, { "code": "open_role_unavailable", "action": "fix_request" @@ -206,6 +286,10 @@ "code": "operation_not_found", "action": "check_id" }, + { + "code": "operation_uncovered", + "action": "fix_request" + }, { "code": "option_target_invalid", "action": "fix_request" @@ -219,8 +303,8 @@ "action": "fix_request" }, { - "code": "pin_not_permitted", - "action": "request_scope" + "code": "payload_too_large", + "action": "fix_request" }, { "code": "pivot_not_homogeneous", @@ -238,6 +322,10 @@ "code": "pivot_over_column_cap", "action": "fix_request" }, + { + "code": "place_levels_mixed", + "action": "fix_request" + }, { "code": "population_empty", "action": "fix_request" @@ -263,6 +351,18 @@ "code": "query_too_long", "action": "fix_request" }, + { + "code": "quotient_of_no_key_matched", + "action": "fix_request" + }, + { + "code": "quotient_of_side_unit_unsupported", + "action": "fix_request" + }, + { + "code": "quotient_of_zero_denominator", + "action": "fix_request" + }, { "code": "rate_coverage_gap", "action": "fix_request", @@ -308,6 +408,10 @@ "code": "resource_exhausted", "action": "fix_request" }, + { + "code": "role_ambiguous", + "action": "fix_request" + }, { "code": "role_incompatible", "action": "fix_request", @@ -323,6 +427,18 @@ "action": "fix_request", "gap_kind": "unsupported" }, + { + "code": "rolling_anchor_unresolved", + "action": "fix_request" + }, + { + "code": "rolling_grain_unsupported", + "action": "fix_request" + }, + { + "code": "rolling_populations_differ", + "action": "fix_request" + }, { "code": "rollup_not_summable", "action": "fix_request" @@ -339,10 +455,34 @@ "code": "service_unavailable", "action": "report" }, + { + "code": "several_members", + "action": "fix_request" + }, { "code": "shape_mismatch", "action": "fix_request" }, + { + "code": "share_not_replaceable", + "action": "fix_request" + }, + { + "code": "share_of_grain_mismatch", + "action": "fix_request" + }, + { + "code": "share_of_key_unknown", + "action": "fix_request" + }, + { + "code": "share_of_key_unmatched", + "action": "fix_request" + }, + { + "code": "share_of_zero_denominator", + "action": "fix_request" + }, { "code": "skill_not_found", "action": "check_id" @@ -396,6 +536,14 @@ "code": "spec_not_found", "action": "check_id" }, + { + "code": "stat_var_unresolved", + "action": "fix_request" + }, + { + "code": "sum_of_no_key_matched", + "action": "fix_request" + }, { "code": "timed_out", "action": "ask_data_question", @@ -405,6 +553,10 @@ "code": "timeout", "action": "report" }, + { + "code": "unit_heterogeneous", + "action": "fix_request" + }, { "code": "unit_not_resolved", "action": "ask_data_question", @@ -438,6 +590,14 @@ { "code": "write_refused", "action": "request_scope" + }, + { + "code": "write_token_invalid", + "action": "stop" + }, + { + "code": "write_token_required", + "action": "fix_request" } ] } diff --git a/python/client-contract/src/dfo_client_contract/actions.schema.json b/python/client-contract/src/dfo_client_contract/actions.schema.json index a102481..884937d 100644 --- a/python/client-contract/src/dfo_client_contract/actions.schema.json +++ b/python/client-contract/src/dfo_client_contract/actions.schema.json @@ -37,6 +37,7 @@ "take_newest_version", "log_in", "request_scope", + "stop", "wait", "wait_for_reset", "report", diff --git a/python/client-contract/src/dfo_client_contract/contract.py b/python/client-contract/src/dfo_client_contract/contract.py index e21412b..a8df709 100644 --- a/python/client-contract/src/dfo_client_contract/contract.py +++ b/python/client-contract/src/dfo_client_contract/contract.py @@ -7,7 +7,8 @@ from pathlib import Path from typing import Any -# The rules under `rules/`, each a file of vectors named after the rule. +# The rules under `rules/`, each a file of vectors named after the rule: the +# client rules, then the analysis and publication rules DFO applies. RULES = ( "key-format", "credentials-path", @@ -20,6 +21,12 @@ "client-identity", "role-keys", "file-selector", + "series-left-out", + "output-digest", + "output-comparison", + "operation-outputs", + "rolling-composition", + "publication", ) diff --git a/python/client-contract/src/dfo_client_contract/dataset-ref.schema.json b/python/client-contract/src/dfo_client_contract/dataset-ref.schema.json index 4e59f21..dbf129d 100644 --- a/python/client-contract/src/dfo_client_contract/dataset-ref.schema.json +++ b/python/client-contract/src/dfo_client_contract/dataset-ref.schema.json @@ -2,26 +2,65 @@ "$schema": "https://json-schema.org/draft/2020-12/schema", "$id": "https://raw.githubusercontent.com/DevelopmentFinanceObservatory/spec-kit/main/python/client-contract/src/dfo_client_contract/dataset-ref.schema.json", "title": "DatasetRef", - "description": "The frozen reference to the exact versions a dataset holds. Every DFO client writes and reads this format, so a ref saved by one language loads in another.", - "type": "object", - "additionalProperties": false, - "required": [ - "format_version", - "members" - ], - "properties": { - "format_version": { - "const": 1 - }, - "members": { - "type": "array", - "minItems": 1, - "items": { - "$ref": "#/$defs/member" + "description": "The frozen reference to the exact versions a dataset holds. Every DFO client writes and reads this format, so a ref saved by one language loads in another. Format 1 holds dataset members, format 2 derived members, each the result of an analysis recipe.", + "if": { + "type": "object", + "required": [ + "format_version" + ], + "properties": { + "format_version": { + "const": 2 } } }, + "then": { + "$ref": "#/$defs/format_2" + }, + "else": { + "$ref": "#/$defs/format_1" + }, "$defs": { + "format_1": { + "type": "object", + "additionalProperties": false, + "required": [ + "format_version", + "members" + ], + "properties": { + "format_version": { + "const": 1 + }, + "members": { + "type": "array", + "minItems": 1, + "items": { + "$ref": "#/$defs/member" + } + } + } + }, + "format_2": { + "type": "object", + "additionalProperties": false, + "required": [ + "format_version", + "members" + ], + "properties": { + "format_version": { + "const": 2 + }, + "members": { + "type": "array", + "minItems": 1, + "items": { + "$ref": "#/$defs/derived_member" + } + } + } + }, "member": { "type": "object", "additionalProperties": false, @@ -97,40 +136,244 @@ "pattern": "^sha256:[0-9a-f]{64}$" }, "rendering": { - "description": "The selector a client matches an artifact's rendering on. Never the server's rendering object.", + "$ref": "#/$defs/rendering" + } + } + } + } + }, + "derived_member": { + "description": "The frame an analysis recipe computed. DFO computes every member below; a client copies them from the server's answer and computes none of them.", + "type": "object", + "additionalProperties": false, + "required": [ + "parts", + "sentence", + "spec", + "content_version", + "output_digest", + "roots", + "execution_identity", + "resolved_periods", + "artifact" + ], + "properties": { + "parts": { + "description": "Empty, since no answer part names a derived frame.", + "type": "array", + "maxItems": 0 + }, + "sentence": { + "description": "The server's sentence for the derived frame.", + "type": "string", + "minLength": 1 + }, + "spec": { + "type": "object", + "additionalProperties": false, + "required": [ + "spec_id", + "recipe" + ], + "properties": { + "spec_id": { + "$ref": "#/$defs/ops_id" + }, + "recipe": { + "description": "The recipe's two hashes. `normal_form_hash` names the computation whatever data it ran on; `replay_key` names it on these roots under this execution identity.", "type": "object", "additionalProperties": false, "required": [ - "format", - "bundle" + "normal_form_hash", + "replay_key" ], "properties": { - "format": { - "enum": [ - "csv", - "xlsx", - "parquet" - ] - }, - "bundle": { - "type": "boolean" + "normal_form_hash": { + "type": "string", + "pattern": "^rnf_[0-9a-f]{32}$" }, - "layout": { - "const": "frame" - }, - "without_party_totals": { - "description": "Selects the artifact rendered with party totals left out. The key is absent for a plain rendering, so `false` is not a valid value.", - "const": true + "replay_key": { + "type": "string", + "pattern": "^rpk_[0-9a-f]{32}$" } } } } + }, + "content_version": { + "type": "string", + "pattern": "^ocv_[0-9a-f]{32}$" + }, + "output_digest": { + "description": "The digest of the frame's output alone, as rules/README.md defines it. `same_data` compares it.", + "type": "string", + "pattern": "^out_[0-9a-f]{32}$" + }, + "roots": { + "description": "The dataset versions the recipe ran on, each with the rendering it read.", + "type": "array", + "minItems": 1, + "uniqueItems": true, + "items": { + "$ref": "#/$defs/root" + } + }, + "execution_identity": { + "description": "The engine, frame grammar and reduction policy versions that ran the recipe, joined by `|`.", + "type": "string", + "pattern": "^[0-9A-Za-z.+-]+\\|[0-9A-Za-z.+-]+\\|[0-9A-Za-z.+-]+$" + }, + "resolved_periods": { + "description": "Each period the recipe states relative to the latest period, with the period it resolved to. Empty for a recipe that states none.", + "type": "array", + "uniqueItems": true, + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "anchor", + "offset", + "rule", + "period" + ], + "properties": { + "anchor": { + "const": "latest" + }, + "offset": { + "type": "integer", + "maximum": 0 + }, + "rule": { + "const": "half1" + }, + "period": { + "type": "string", + "pattern": "^[0-9]{4}$" + } + } + } + }, + "artifact": { + "type": "object", + "additionalProperties": false, + "required": [ + "digest", + "rendering" + ], + "properties": { + "digest": { + "type": "string", + "pattern": "^sha256:[0-9a-f]{64}$" + }, + "rendering": { + "$ref": "#/$defs/rendering" + } + } } } }, + "root": { + "type": "object", + "additionalProperties": false, + "required": [ + "spec_id", + "content_version", + "rendering" + ], + "properties": { + "spec_id": { + "$ref": "#/$defs/spec_id" + }, + "content_version": { + "$ref": "#/$defs/member/properties/content_version" + }, + "rendering": { + "$ref": "#/$defs/root_rendering" + } + } + }, + "root_rendering": { + "description": "The rendering an analysis read a root in: the members of the artifact selector, and `groupby` when the analysis read the root summed. `rendering`, the artifact selector, has no `groupby`.", + "$ref": "#/$defs/selector", + "properties": { + "groupby": { + "$ref": "#/$defs/groupby" + } + }, + "unevaluatedProperties": false + }, + "groupby": { + "description": "The sum the root was read with, the mint option `options.groupby` as DFO records it: one row per distinct combination of the columns `by` keeps, every other summable axis summed away. `levels` rolls each kept hierarchical axis it names up to the ancestor at that depth before the sum, 0 being the root codes. The key is absent when no level is set.", + "type": "object", + "additionalProperties": false, + "required": [ + "by", + "op" + ], + "properties": { + "by": { + "type": "array", + "minItems": 1, + "uniqueItems": true, + "items": { + "type": "string", + "minLength": 1 + } + }, + "op": { + "const": "sum" + }, + "levels": { + "type": "object", + "minProperties": 1, + "additionalProperties": { + "type": "integer", + "minimum": 0, + "maximum": 99 + } + } + } + }, + "selector": { + "description": "The members of the selector a client matches an artifact's rendering on, shared by an artifact's rendering and a derived member's root rendering.", + "type": "object", + "required": [ + "format", + "bundle" + ], + "properties": { + "format": { + "enum": [ + "csv", + "xlsx", + "parquet" + ] + }, + "bundle": { + "type": "boolean" + }, + "layout": { + "const": "frame" + }, + "without_party_totals": { + "description": "Selects the artifact rendered with party totals left out. The key is absent for a plain rendering, so `false` is not a valid value.", + "const": true + } + } + }, + "rendering": { + "description": "The selector a client matches an artifact's rendering on. Never the server's rendering object.", + "$ref": "#/$defs/selector", + "unevaluatedProperties": false + }, "spec_id": { "type": "string", "pattern": "^spec_[0-9a-f]{32}$" + }, + "ops_id": { + "type": "string", + "pattern": "^ops_[0-9a-f]{32}$" } } } diff --git a/python/client-contract/src/dfo_client_contract/refs/ref-artifact-groupby-rejected.json b/python/client-contract/src/dfo_client_contract/refs/ref-artifact-groupby-rejected.json new file mode 100644 index 0000000..c280c1a --- /dev/null +++ b/python/client-contract/src/dfo_client_contract/refs/ref-artifact-groupby-rejected.json @@ -0,0 +1,82 @@ +{ + "format_version": 1, + "name": "ref-artifact-groupby-rejected", + "description": "An artifact's rendering selects a file and names no groupby: only a derived member's root records the sum it was read with.", + "scenario": "fetch-spec-known-sentence", + "ref": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria by top-level purpose, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_40511987433085c4f2dadc6b7aae2755", + "recipe": { + "normal_form_hash": "rnf_920ac02001d7529fd9c17ebadb452c99", + "replay_key": "rpk_a0a019687e913a1204cbcb089a58492f" + } + }, + "content_version": "ocv_6b4b9814c24f8d0c94624826c0c7773e", + "output_digest": "out_6927b7d353696180a34110d8dde723d6", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true, + "groupby": { + "by": [ + "crsPurpose", + "time_period" + ], + "op": "sum", + "levels": { + "crsPurpose": 0 + } + } + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2020" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2023" + } + ], + "artifact": { + "digest": "sha256:2f052058736db19e262bb8496f0da6c90f9deef851cb0224cff42dfb04bd0bd1", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "groupby": { + "by": [ + "crsPurpose", + "time_period" + ], + "op": "sum", + "levels": { + "crsPurpose": 0 + } + } + } + } + } + ] + }, + "error": { + "by": "schema" + } +} diff --git a/python/client-contract/src/dfo_client_contract/refs/ref-dataset-member-in-format-2-rejected.json b/python/client-contract/src/dfo_client_contract/refs/ref-dataset-member-in-format-2-rejected.json new file mode 100644 index 0000000..bda6c56 --- /dev/null +++ b/python/client-contract/src/dfo_client_contract/refs/ref-dataset-member-in-format-2-rejected.json @@ -0,0 +1,99 @@ +{ + "format_version": 1, + "name": "ref-dataset-member-in-format-2-rejected", + "description": "Format 2 holds derived members only. A ref of dataset members is written as format 1.", + "scenario": "fetch-spec-known-sentence", + "ref": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Current US dollars · OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "definition": { + "schema_version": 1, + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "identity": { + "identity_version": "spec1", + "shape": { + "kind": "statvar_schema", + "population_type": "FinancialFlow", + "measured_property": "amount", + "stat_type": "measuredValue", + "measurement_qualifier": "Nominal" + }, + "slots": [ + { + "property": "crsCooperationModality", + "binding": { + "state": "unbound" + } + }, + { + "property": "crsPurpose", + "binding": { + "state": "value", + "ref": "CRSPurposeCode12220" + } + }, + { + "property": "crsTypeOfFinance", + "binding": { + "state": "unbound" + } + }, + { + "property": "crsTypeOfFlow", + "binding": { + "state": "value", + "ref": "CRSTypeOfFlow10" + } + }, + { + "property": "financialFlowType", + "binding": { + "state": "value", + "ref": "Disbursement" + } + }, + { + "property": "nettingBasis", + "binding": { + "state": "value", + "ref": "Gross" + } + } + ], + "entities": { + "destination": [ + "country/NGA" + ], + "origin": [ + "country/DEU" + ] + }, + "requested_window": { + "kind": "range", + "start_year": 2020, + "end_year": 2023 + } + } + } + }, + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "artifact": { + "digest": "sha256:c44200969f19d73104d1bc24ebca905e59f749c2939234cdde10ba6f5a131170", + "rendering": { + "format": "parquet", + "bundle": false, + "without_party_totals": true + } + } + } + ] + }, + "error": { + "by": "schema" + } +} diff --git a/python/client-contract/src/dfo_client_contract/refs/ref-derived-grouped-root.json b/python/client-contract/src/dfo_client_contract/refs/ref-derived-grouped-root.json new file mode 100644 index 0000000..8b27887 --- /dev/null +++ b/python/client-contract/src/dfo_client_contract/refs/ref-derived-grouped-root.json @@ -0,0 +1,70 @@ +{ + "format_version": 1, + "name": "ref-derived-grouped-root", + "description": "A format 2 ref naming a growth over a root of fetch-spec-known-sentence read summed, as a breakdown by purpose reads it: the root's rendering carries the groupby the analysis read it with, which keeps the time period and the purpose rolled up to its root codes and sums every other axis away.", + "scenario": "fetch-spec-known-sentence", + "ref": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria by top-level purpose, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_40511987433085c4f2dadc6b7aae2755", + "recipe": { + "normal_form_hash": "rnf_920ac02001d7529fd9c17ebadb452c99", + "replay_key": "rpk_a0a019687e913a1204cbcb089a58492f" + } + }, + "content_version": "ocv_6b4b9814c24f8d0c94624826c0c7773e", + "output_digest": "out_6927b7d353696180a34110d8dde723d6", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true, + "groupby": { + "by": [ + "crsPurpose", + "time_period" + ], + "op": "sum", + "levels": { + "crsPurpose": 0 + } + } + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2020" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2023" + } + ], + "artifact": { + "digest": "sha256:2f052058736db19e262bb8496f0da6c90f9deef851cb0224cff42dfb04bd0bd1", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "expect": "accept" +} diff --git a/python/client-contract/src/dfo_client_contract/refs/ref-derived-in-format-1-rejected.json b/python/client-contract/src/dfo_client_contract/refs/ref-derived-in-format-1-rejected.json new file mode 100644 index 0000000..82dc189 --- /dev/null +++ b/python/client-contract/src/dfo_client_contract/refs/ref-derived-in-format-1-rejected.json @@ -0,0 +1,62 @@ +{ + "format_version": 1, + "name": "ref-derived-in-format-1-rejected", + "description": "Format 1 holds dataset members only, so a derived member needs format 2.", + "scenario": "fetch-spec-known-sentence", + "ref": { + "format_version": 1, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_e1edc6acb97615bdd9e813730378ce19", + "recipe": { + "normal_form_hash": "rnf_4e2298354b4323699033912586859aa7", + "replay_key": "rpk_1185a76368dca1b0301c899170c7dff3" + } + }, + "content_version": "ocv_3187efa2294f182bfe7cab296659b74f", + "output_digest": "out_82b81c6278499b468aa3d6a9c5a6aa6b", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2020" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2023" + } + ], + "artifact": { + "digest": "sha256:4c8fec27543c7bedbe9d2c436f42c5a65f721106c128d0a55d2962c2c67de033", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "error": { + "by": "schema" + } +} diff --git a/python/client-contract/src/dfo_client_contract/refs/ref-derived-member-fixed-window.json b/python/client-contract/src/dfo_client_contract/refs/ref-derived-member-fixed-window.json new file mode 100644 index 0000000..808c33f --- /dev/null +++ b/python/client-contract/src/dfo_client_contract/refs/ref-derived-member-fixed-window.json @@ -0,0 +1,47 @@ +{ + "format_version": 1, + "name": "ref-derived-member-fixed-window", + "description": "A format 2 ref naming a growth over a fixed window, which states no period relative to the latest and so resolves none.", + "scenario": "fetch-spec-known-sentence", + "ref": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_a41f297b3f33f2a7c2ac6b3cedd4d3ad", + "recipe": { + "normal_form_hash": "rnf_5f5a142beef739694b06899a7f0426a7", + "replay_key": "rpk_4f697e3235da4b9ae1404f112c2c78b6" + } + }, + "content_version": "ocv_62a72043403b8a2497cc184623df87df", + "output_digest": "out_82b81c6278499b468aa3d6a9c5a6aa6b", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [], + "artifact": { + "digest": "sha256:4c8fec27543c7bedbe9d2c436f42c5a65f721106c128d0a55d2962c2c67de033", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "expect": "accept" +} diff --git a/python/client-contract/src/dfo_client_contract/refs/ref-derived-member.json b/python/client-contract/src/dfo_client_contract/refs/ref-derived-member.json new file mode 100644 index 0000000..87a84f3 --- /dev/null +++ b/python/client-contract/src/dfo_client_contract/refs/ref-derived-member.json @@ -0,0 +1,60 @@ +{ + "format_version": 1, + "name": "ref-derived-member", + "description": "A format 2 ref naming a growth over the latest four years of the member of fetch-spec-known-sentence: its ops_ id and recipe hashes, its ocv_ version and output digest, its root with the rendering it read, the execution identity, and the two periods the recipe states relative to the latest, resolved.", + "scenario": "fetch-spec-known-sentence", + "ref": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_e1edc6acb97615bdd9e813730378ce19", + "recipe": { + "normal_form_hash": "rnf_4e2298354b4323699033912586859aa7", + "replay_key": "rpk_1185a76368dca1b0301c899170c7dff3" + } + }, + "content_version": "ocv_3187efa2294f182bfe7cab296659b74f", + "output_digest": "out_82b81c6278499b468aa3d6a9c5a6aa6b", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2020" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2023" + } + ], + "artifact": { + "digest": "sha256:4c8fec27543c7bedbe9d2c436f42c5a65f721106c128d0a55d2962c2c67de033", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "expect": "accept" +} diff --git a/python/client-contract/src/dfo_client_contract/refs/ref-derived-parts-rejected.json b/python/client-contract/src/dfo_client_contract/refs/ref-derived-parts-rejected.json new file mode 100644 index 0000000..830d6ea --- /dev/null +++ b/python/client-contract/src/dfo_client_contract/refs/ref-derived-parts-rejected.json @@ -0,0 +1,64 @@ +{ + "format_version": 1, + "name": "ref-derived-parts-rejected", + "description": "A derived member answers no answer part, so its parts are empty.", + "scenario": "fetch-spec-known-sentence", + "ref": { + "format_version": 2, + "members": [ + { + "parts": [ + "ODA from Germany to Nigeria for basic health care" + ], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_e1edc6acb97615bdd9e813730378ce19", + "recipe": { + "normal_form_hash": "rnf_4e2298354b4323699033912586859aa7", + "replay_key": "rpk_1185a76368dca1b0301c899170c7dff3" + } + }, + "content_version": "ocv_3187efa2294f182bfe7cab296659b74f", + "output_digest": "out_82b81c6278499b468aa3d6a9c5a6aa6b", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2020" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2023" + } + ], + "artifact": { + "digest": "sha256:4c8fec27543c7bedbe9d2c436f42c5a65f721106c128d0a55d2962c2c67de033", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "error": { + "by": "schema" + } +} diff --git a/python/client-contract/src/dfo_client_contract/refs/ref-derived-period-after-latest-rejected.json b/python/client-contract/src/dfo_client_contract/refs/ref-derived-period-after-latest-rejected.json new file mode 100644 index 0000000..f02467c --- /dev/null +++ b/python/client-contract/src/dfo_client_contract/refs/ref-derived-period-after-latest-rejected.json @@ -0,0 +1,56 @@ +{ + "format_version": 1, + "name": "ref-derived-period-after-latest-rejected", + "description": "A resolved period counts back from the latest period, so its offset is 0 or below.", + "scenario": "fetch-spec-known-sentence", + "ref": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_e1edc6acb97615bdd9e813730378ce19", + "recipe": { + "normal_form_hash": "rnf_4e2298354b4323699033912586859aa7", + "replay_key": "rpk_1185a76368dca1b0301c899170c7dff3" + } + }, + "content_version": "ocv_3187efa2294f182bfe7cab296659b74f", + "output_digest": "out_82b81c6278499b468aa3d6a9c5a6aa6b", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": 1, + "rule": "half1", + "period": "2024" + } + ], + "artifact": { + "digest": "sha256:4c8fec27543c7bedbe9d2c436f42c5a65f721106c128d0a55d2962c2c67de033", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "error": { + "by": "schema" + } +} diff --git a/python/client-contract/src/dfo_client_contract/refs/ref-derived-repeated-spec-id-rejected.json b/python/client-contract/src/dfo_client_contract/refs/ref-derived-repeated-spec-id-rejected.json new file mode 100644 index 0000000..e42fff8 --- /dev/null +++ b/python/client-contract/src/dfo_client_contract/refs/ref-derived-repeated-spec-id-rejected.json @@ -0,0 +1,111 @@ +{ + "format_version": 1, + "name": "ref-derived-repeated-spec-id-rejected", + "description": "Each member holds a distinct spec_id, an ops_ id included.", + "scenario": "fetch-spec-known-sentence", + "ref": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_e1edc6acb97615bdd9e813730378ce19", + "recipe": { + "normal_form_hash": "rnf_4e2298354b4323699033912586859aa7", + "replay_key": "rpk_1185a76368dca1b0301c899170c7dff3" + } + }, + "content_version": "ocv_3187efa2294f182bfe7cab296659b74f", + "output_digest": "out_82b81c6278499b468aa3d6a9c5a6aa6b", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2020" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2023" + } + ], + "artifact": { + "digest": "sha256:4c8fec27543c7bedbe9d2c436f42c5a65f721106c128d0a55d2962c2c67de033", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + }, + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_e1edc6acb97615bdd9e813730378ce19", + "recipe": { + "normal_form_hash": "rnf_4e2298354b4323699033912586859aa7", + "replay_key": "rpk_1185a76368dca1b0301c899170c7dff3" + } + }, + "content_version": "ocv_3187efa2294f182bfe7cab296659b74f", + "output_digest": "out_82b81c6278499b468aa3d6a9c5a6aa6b", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2020" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2023" + } + ], + "artifact": { + "digest": "sha256:4c8fec27543c7bedbe9d2c436f42c5a65f721106c128d0a55d2962c2c67de033", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "error": { + "by": "rule", + "reason": "repeated_spec_id" + } +} diff --git a/python/client-contract/src/dfo_client_contract/refs/ref-derived-root-groupby-op-rejected.json b/python/client-contract/src/dfo_client_contract/refs/ref-derived-root-groupby-op-rejected.json new file mode 100644 index 0000000..1ed2bb3 --- /dev/null +++ b/python/client-contract/src/dfo_client_contract/refs/ref-derived-root-groupby-op-rejected.json @@ -0,0 +1,72 @@ +{ + "format_version": 1, + "name": "ref-derived-root-groupby-op-rejected", + "description": "A root's groupby sums, so any other op is not a ref.", + "scenario": "fetch-spec-known-sentence", + "ref": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria by top-level purpose, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_40511987433085c4f2dadc6b7aae2755", + "recipe": { + "normal_form_hash": "rnf_920ac02001d7529fd9c17ebadb452c99", + "replay_key": "rpk_a0a019687e913a1204cbcb089a58492f" + } + }, + "content_version": "ocv_6b4b9814c24f8d0c94624826c0c7773e", + "output_digest": "out_6927b7d353696180a34110d8dde723d6", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true, + "groupby": { + "by": [ + "crsPurpose", + "time_period" + ], + "op": "mean", + "levels": { + "crsPurpose": 0 + } + } + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2020" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2023" + } + ], + "artifact": { + "digest": "sha256:2f052058736db19e262bb8496f0da6c90f9deef851cb0224cff42dfb04bd0bd1", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "error": { + "by": "schema" + } +} diff --git a/python/client-contract/src/dfo_client_contract/refs/ref-derived-root-without-rendering-rejected.json b/python/client-contract/src/dfo_client_contract/refs/ref-derived-root-without-rendering-rejected.json new file mode 100644 index 0000000..b038de9 --- /dev/null +++ b/python/client-contract/src/dfo_client_contract/refs/ref-derived-root-without-rendering-rejected.json @@ -0,0 +1,56 @@ +{ + "format_version": 1, + "name": "ref-derived-root-without-rendering-rejected", + "description": "Each root records the rendering the recipe read, since a cut with party totals left out and a plain one are two inputs.", + "scenario": "fetch-spec-known-sentence", + "ref": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_e1edc6acb97615bdd9e813730378ce19", + "recipe": { + "normal_form_hash": "rnf_4e2298354b4323699033912586859aa7", + "replay_key": "rpk_1185a76368dca1b0301c899170c7dff3" + } + }, + "content_version": "ocv_3187efa2294f182bfe7cab296659b74f", + "output_digest": "out_82b81c6278499b468aa3d6a9c5a6aa6b", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4" + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2020" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2023" + } + ], + "artifact": { + "digest": "sha256:4c8fec27543c7bedbe9d2c436f42c5a65f721106c128d0a55d2962c2c67de033", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "error": { + "by": "schema" + } +} diff --git a/python/client-contract/src/dfo_client_contract/refs/ref-derived-without-output-digest-rejected.json b/python/client-contract/src/dfo_client_contract/refs/ref-derived-without-output-digest-rejected.json new file mode 100644 index 0000000..6947fcf --- /dev/null +++ b/python/client-contract/src/dfo_client_contract/refs/ref-derived-without-output-digest-rejected.json @@ -0,0 +1,61 @@ +{ + "format_version": 1, + "name": "ref-derived-without-output-digest-rejected", + "description": "A derived member carries the output digest same_data compares.", + "scenario": "fetch-spec-known-sentence", + "ref": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_e1edc6acb97615bdd9e813730378ce19", + "recipe": { + "normal_form_hash": "rnf_4e2298354b4323699033912586859aa7", + "replay_key": "rpk_1185a76368dca1b0301c899170c7dff3" + } + }, + "content_version": "ocv_3187efa2294f182bfe7cab296659b74f", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2020" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2023" + } + ], + "artifact": { + "digest": "sha256:4c8fec27543c7bedbe9d2c436f42c5a65f721106c128d0a55d2962c2c67de033", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "error": { + "by": "schema" + } +} diff --git a/python/client-contract/src/dfo_client_contract/rules/README.md b/python/client-contract/src/dfo_client_contract/rules/README.md index 7eb40e4..48d3a49 100644 --- a/python/client-contract/src/dfo_client_contract/rules/README.md +++ b/python/client-contract/src/dfo_client_contract/rules/README.md @@ -1,6 +1,6 @@ # Client rules -Clients implement these rules for the capabilities available in their runtime. Python implements every rule. TypeScript implements the key, credentials and base-URL rules in Node, and the reading-order, client-identity, role-keys and file-selector rules in every runtime; browser runtimes use the key sources available there. TypeScript currently keeps no disk cache, so `cache-dir` and `cache-layout` apply to Python. Clients implementing the same capability use the same rules and formats. Each rule's vectors are in `.json`, validated by `rules.schema.json`. In a vector, `env` lists the environment variables set, and an empty value counts as unset everywhere below. +Clients implement these rules for the capabilities available in their runtime. The first twelve rules, from credentials path to series left out, are client rules, and Python implements all twelve. TypeScript implements the key, credentials and base-URL rules in Node, and the reading-order, client-identity, role-keys, file-selector and series-left-out rules in every runtime; browser runtimes use the key sources available there. TypeScript currently keeps no disk cache, so `cache-dir` and `cache-layout` apply to Python. Clients implementing the same capability use the same rules and formats. The last five rules, from output digest to publication, are DFO's: dfo-apis applies them where it computes, stores and publishes, and a client reads their results. Of these, a client implements only the output comparison, over the digests DFO gives it, as `==` and `same_data` in Python and `equals` and `sameData` in TypeScript. Each rule's vectors are in `.json`, validated by `rules.schema.json`. In a vector, `env` lists the environment variables set, and an empty value counts as unset everywhere below. ## Credentials path (`credentials-path`) @@ -120,3 +120,33 @@ The server replaces a spec's whole binding with the posted one, so a client post ## File selector (`file-selector`) The file selector, `GET /datasets/{spec_id}/{content_version}/file`, redirects to a stored version's file in a rendering. A client asks it for a ref member's version when the files route at the stored digest refuses, or when the snapshot manifest lists no artifact of that digest. Its query holds the keys `query_keys` in `file-selector.json` lists, `format`, `bundle` and `layout`, in that order, each one the member's rendering holds, as `key=value` pairs joined by `&`, with booleans written `true` and `false`. The query leaves out `without_party_totals`: the version records the cut it was minted with, and the server lays that over the query. The server refuses any other key with 400 `request_invalid`, so a selector query that carried the cut would end the lookup. A vector gives the `rendering` and the `query` a client sends, without the leading `?`. + +## Series left out (`series-left-out`) + +A derived member's manifest, from `GET /datasets/derived/{ops_id}/{ocv_version}`, lists in `left_out` every series a step of its recipe dropped for a data gap, such as a growth endpoint the series lacks or a key the other side holds no partner for. Each entry names the series' `coordinates`, its `step` as the verb and node index, the `side` it came from, the engine's `reason` and its `rows`. `left_out_complete` is false when the steps dropped more series than `left_out` lists. Both are null for a recipe stored without them. + +A fetch of derived members issues `series_left_out`, as `SeriesLeftOutWarning`, once when any member's manifest lists a left-out series or says `left_out_complete` false, and returns the frame. A strict fetch raises the warning in place of the data. A null `left_out` issues nothing. A vector gives each member's `left_out` and `left_out_complete` in member order, and `strict`, then the `warnings` the fetch issues or the warning it `raises`. + +## Output digest (`output-digest`) + +A derived member's `output_digest` names the output of its frame: its values, with the variable, members, period and unit each belongs to, and nothing about how they were computed. DFO computes it when it stores a derived frame, and a client copies it from the server's answer. + +The digest is taken over a frame as `docs/frame-grammar-v1.md` defines it. Its document is the RFC 8785 (JCS) serialisation of `{"digest_version": "out2", "columns": [...], "rows": [...]}`. `columns` lists `stat_var`, then the frame's role columns in the frame's order, then `time_period`, `value` and `unit`. Each row holds the frame row's cells in those columns, with `recipe` dropped from every unit-request object, in a `unit` cell and in either part of a composed unit. The rows are sorted by the UTF-8 bytes of each row's own JCS serialisation, ascending, and a row that repeats another stays. The digest is `out_` followed by the first 32 lowercase hexadecimal characters of the SHA-256 of the document's UTF-8 bytes. + +The document leaves out `measurementMethod`, `provenance` and `status`, and a unit's `recipe`. These name the source an output came from and how its unit was converted, so they move when a source is imported again or a conversion recipe is revised while no number moves. `stat_var` stays in, so two frames that hold the same values under different variables hold different outputs. A join writes each side's rows under that side's variable, so a join with its sides swapped holds the same rows and the same output, while a frame holding each side's values under the other side's variable holds another. `compare`, `sum_of`, `share_of` and `quotient_of` write rows under a variable of their own (`operation-outputs`), which a frame spells `dcid:` and the step's `ops_` id. That id hashes the versions the step read and, through their `ocv_` versions, the engine that computed them, so the document reads that variable under a name built from what was computed. The name is `mv_` and the first 32 hex digits of the SHA-256 of a name document, the JCS serialisation of an object holding `verb`, the step's verb, and one member per frame slot the step fills, keyed by the slot's name in the recipe tree (`frame_ref`, `right_frame_ref`, `denominator_frame_ref`), each the names of the variables that slot's input carries, sorted by their UTF-8 bytes. A root dataset carries its frame's `stat_var` values, `filter`, `aggregate`, `rank` and `growth` carry their input's variables, `join` the union of its inputs', and a step that writes its own variable carries that variable's `mv_` name. A name document thus holds the names of its inputs and not their documents, so every name is 35 characters however deep the recipe, and a name document grows only with the variables the roots under it hold. The prefix marks the name as computed, apart from a dataset's `dcid:` variable and an operation's `ops_` id, in the style of the `out_` digest. A one-step `quotient_of` of ODA by population has the name document `{"denominator_frame_ref":["dcid:Count_Person"],"frame_ref":["dcid:Amount_ODA"],"verb":"quotient_of"}` and names its variable `mv_d355e59c50ff8cf07e86a4da12b64297`. Equal computations over equal variables thus share a digest across recipes, engine releases and upstream rebuilds that move no number, and a step with its sides swapped holds another. A `filter` that narrows a root holding several variables leaves the name listing all of them, which errs towards two outputs differing. A vector gives the `frame`, the `variables` its operations wrote where it holds any, each name keyed by its `ops_` id, the `name_documents` of those names and of every `mv_` name their documents hold, each keyed by its name, the `document` as text and the `digest`. `quotient-of-a-sum-of` divides a `sum_of` by population, so its name document lists the `sum_of` step's `mv_` name. Vectors whose frames differ only in what the document leaves out, in row order, or in the `ops_` id of a variable named alike, share a digest. + +## Output comparison (`output-comparison`) + +Two refs answer two questions. `==` in Python and `equals` in TypeScript say whether two refs name the same versions, recipes and renderings: they hold as many members, and each member matches the other ref's member at its position on its `spec_id`, its `content_version`, its artifact's `rendering` and, for a derived member, its `recipe`. A rolling recipe and a fixed one therefore stay apart in a cache even when their rows match. `same_data` in Python and `sameData` in TypeScript say whether two refs hold the same output: they hold as many members, and at each position both members are derived members with the same `output_digest`, or both are dataset members with the same `content_version` and the same `without_party_totals`. A dataset version's `content_version` already hashes its frame alone, so a dataset member needs no digest of its own, and its layout and file format hold the same rows and do not enter. A dataset member and a derived member never hold the same output. Both comparisons ignore `format_version`, `parts`, `sentence`, artifact digests, roots, execution identities and resolved periods. So an engine release or an upstream rebuild that moves no number keeps `same_data` true, and a pipeline that rebuilds on `not same_data` rebuilds only when an output moved. This holds for an output under a variable its operation wrote, since the digest names that variable by its verb and its inputs' variables (`output-digest`), so a one-step `quotient_of` and a `filter` that keeps every row followed by the same `quotient_of` hold the same data. A vector gives two refs, `left` and `right`, and the two verdicts, which hold either way round. + +## Operation outputs (`operation-outputs`) + +DFO's operations keep the units and shapes Studio's have. `share_of` gives a share in per cent, 100 times the ratio, under the unit `Percent`. With a second reading, each row of `data` is divided by the row of `other` that `on` pairs it with in the same period. With none, each member of `by` is divided by the total across every member in the same period, after its own rows add up. `join` returns the rows of both readings that `on` pairs, in long format: one row per side for each key and period both hold, told apart by `stat_var`, each keeping its own unit. `compare` gives `data` minus `other`, and `sum_of` gives `data` plus `other`, for each paired member and period both hold, in the unit both share. A relative `growth` is a fraction in the `value` column, 0.25 for 25 per cent, with no unit, and an absolute one is the change in the input's unit. A growth row's period is the window as a span, such as `2018/2022`, and a member without a value at both ends has no row. A relative growth also leaves out a member whose start or end value is zero or whose values change sign (`zero_or_sign_crossing_base`), and either growth leaves out a change that is not finite (`non_finite`). Two rows of one member in one period of the window refuse the step with `role_ambiguous`. An output keeps `data`'s names for the roles `on` pairs. `share_of`, `compare` and `sum_of` write rows under a variable of their own, whose spelling is the operation's; `join` and `growth` keep their inputs' variables. A vector gives the `verb`, its `arguments`, the rows of `data` and of `other`, and either the output `rows`, in any order and leaving `stat_var` out of a row an operation writes under its own variable, or the `error`. + +## Rolling composition (`rolling-composition`) + +A recipe tree has one latest period. Each rolling step of the tree states the populations it counts the latest period over, one for each root dataset it reads with the columns it narrows that dataset's series by. Every rolling step of one tree states the same populations, compared in their one spelling, whatever order they are listed in. A tree whose rolling steps state different populations is refused with `rolling_populations_differ` before anything runs, whether they would settle on different years or on the same one. The latest period is then the newest period at least half of the series of every population hold, the `half1` rule of `dfo_frame_grammar.latest_anchor`. A population no series falls in, or populations no period satisfies, refuse `rolling_anchor_unresolved`. A vector gives every `series` of the roots with the periods it holds, the rolling `steps` with their populations, and the `period` or the `error`. + +## Publication (`publication`) + +A share publishes cards under a stable URL, live with `latest` true or as a snapshot with it false. It holds numbered slots, the first card in slot 1, and each slot's embed and data URLs are named by its number. Creating a share returns a write token. Replacing a share's cards and taking it down each need that token: without one they are refused with `write_token_required`, and with another token, or on a share created without one, with `write_token_invalid`. The token is checked first. Only a live share can be replaced, and a snapshot refuses `share_not_replaceable`, so a cited snapshot URL draws the same figure for good. A replacement puts its cards in slots 1 onwards, so every URL a share has published keeps working: a slot past the replacement's last card holds no card until a later replacement fills it again. A refused step changes nothing. A snapshot is taken down like a live share. A vector gives the `steps`, each naming a share, and one outcome per step: the share's `latest` and slots, `removed`, or the `error`. A step's `write_token` is `held` for the token the share's creation returned and `other` for another one. diff --git a/python/client-contract/src/dfo_client_contract/rules/operation-outputs.json b/python/client-contract/src/dfo_client_contract/rules/operation-outputs.json new file mode 100644 index 0000000..556fc17 --- /dev/null +++ b/python/client-contract/src/dfo_client_contract/rules/operation-outputs.json @@ -0,0 +1,684 @@ +{ + "format_version": 1, + "rule": "operation-outputs", + "vectors": [ + { + "name": "share-of-in-percent", + "description": "ODA as a share of GNI comes back in per cent, 100 times the ratio, under the Percent unit, keyed by data's role. Ghana has no GNI row and Nigeria no ODA row, so neither has a share.", + "verb": "share_of", + "arguments": { + "on": { + "destination": "observationAbout" + } + }, + "data": [ + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/KEN", + "time_period": "2022", + "value": 300.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/KEN", + "time_period": "2023", + "value": 250.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/GHA", + "time_period": "2022", + "value": 90.0, + "unit": "USDollar" + } + ], + "other": [ + { + "stat_var": "dcid:Amount_EconomicActivity_GrossNationalIncome", + "observationAbout": "country/KEN", + "time_period": "2022", + "value": 120000.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_EconomicActivity_GrossNationalIncome", + "observationAbout": "country/KEN", + "time_period": "2023", + "value": 125000.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_EconomicActivity_GrossNationalIncome", + "observationAbout": "country/NGA", + "time_period": "2022", + "value": 400000.0, + "unit": "USDollar" + } + ], + "rows": [ + { + "destination": "country/KEN", + "time_period": "2022", + "value": 0.25, + "unit": "Percent" + }, + { + "destination": "country/KEN", + "time_period": "2023", + "value": 0.2, + "unit": "Percent" + } + ] + }, + { + "name": "share-of-across-members", + "description": "With no second reading, each member of by gets its share of the total across every member in the same year: donors giving 40 and 60 get 40 and 60 per cent. A donor's rows to several recipients add up first.", + "verb": "share_of", + "arguments": { + "by": [ + "origin" + ] + }, + "data": [ + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "origin": "country/DEU", + "destination": "country/ETH", + "time_period": "2024", + "value": 40.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "origin": "country/FRA", + "destination": "country/ETH", + "time_period": "2024", + "value": 60.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "origin": "country/DEU", + "destination": "country/ETH", + "time_period": "2023", + "value": 30.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "origin": "country/FRA", + "destination": "country/ETH", + "time_period": "2023", + "value": 10.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "origin": "country/FRA", + "destination": "country/SOM", + "time_period": "2023", + "value": 10.0, + "unit": "USDollar" + } + ], + "rows": [ + { + "origin": "country/DEU", + "time_period": "2024", + "value": 40.0, + "unit": "Percent" + }, + { + "origin": "country/FRA", + "time_period": "2024", + "value": 60.0, + "unit": "Percent" + }, + { + "origin": "country/DEU", + "time_period": "2023", + "value": 60.0, + "unit": "Percent" + }, + { + "origin": "country/FRA", + "time_period": "2023", + "value": 40.0, + "unit": "Percent" + } + ] + }, + { + "name": "join-long-format", + "description": "join returns both readings' matched rows in long format, one row per side, told apart by stat_var, each side keeping its own unit, the key named as data names it.", + "verb": "join", + "arguments": { + "on": { + "destination": "observationAbout" + } + }, + "data": [ + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/KEN", + "time_period": "2022", + "value": 300.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/KEN", + "time_period": "2023", + "value": 250.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/GHA", + "time_period": "2022", + "value": 90.0, + "unit": "USDollar" + } + ], + "other": [ + { + "stat_var": "dcid:Amount_EconomicActivity_GrossNationalIncome", + "observationAbout": "country/KEN", + "time_period": "2022", + "value": 120000.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_EconomicActivity_GrossNationalIncome", + "observationAbout": "country/KEN", + "time_period": "2023", + "value": 125000.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_EconomicActivity_GrossNationalIncome", + "observationAbout": "country/NGA", + "time_period": "2022", + "value": 400000.0, + "unit": "USDollar" + } + ], + "rows": [ + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/KEN", + "time_period": "2022", + "value": 300.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_EconomicActivity_GrossNationalIncome", + "destination": "country/KEN", + "time_period": "2022", + "value": 120000.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/KEN", + "time_period": "2023", + "value": 250.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_EconomicActivity_GrossNationalIncome", + "destination": "country/KEN", + "time_period": "2023", + "value": 125000.0, + "unit": "USDollar" + } + ] + }, + { + "name": "compare-data-minus-other", + "description": "compare gives data minus other for each paired member and period both hold, in the unit both share.", + "verb": "compare", + "arguments": { + "on": { + "destination": "observationAbout" + } + }, + "data": [ + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/KEN", + "time_period": "2022", + "value": 300.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/KEN", + "time_period": "2023", + "value": 250.0, + "unit": "USDollar" + } + ], + "other": [ + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_NetDisbursement", + "observationAbout": "country/KEN", + "time_period": "2022", + "value": 280.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_NetDisbursement", + "observationAbout": "country/KEN", + "time_period": "2023", + "value": 265.5, + "unit": "USDollar" + } + ], + "rows": [ + { + "destination": "country/KEN", + "time_period": "2022", + "value": 20.0, + "unit": "USDollar" + }, + { + "destination": "country/KEN", + "time_period": "2023", + "value": -15.5, + "unit": "USDollar" + } + ] + }, + { + "name": "sum-of-paired", + "description": "sum_of adds the paired rows of two readings over the periods both hold, in the unit both share.", + "verb": "sum_of", + "arguments": { + "on": { + "destination": "observationAbout" + } + }, + "data": [ + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/KEN", + "time_period": "2022", + "value": 300.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/KEN", + "time_period": "2023", + "value": 250.0, + "unit": "USDollar" + } + ], + "other": [ + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_NetDisbursement", + "observationAbout": "country/KEN", + "time_period": "2022", + "value": 280.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_NetDisbursement", + "observationAbout": "country/KEN", + "time_period": "2023", + "value": 265.5, + "unit": "USDollar" + } + ], + "rows": [ + { + "destination": "country/KEN", + "time_period": "2022", + "value": 580.0, + "unit": "USDollar" + }, + { + "destination": "country/KEN", + "time_period": "2023", + "value": 515.5, + "unit": "USDollar" + } + ] + }, + { + "name": "growth-as-a-fraction", + "description": "A relative growth is a fraction in the value column, 0.25 for 25 per cent, with no unit, its period the window as a span. Nigeria has no 2018 value, so it has no growth.", + "verb": "growth", + "arguments": { + "by": [ + "destination" + ], + "window": [ + "2018", + "2022" + ] + }, + "data": [ + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/KEN", + "time_period": "2018", + "value": 200.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/KEN", + "time_period": "2022", + "value": 250.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/GHA", + "time_period": "2018", + "value": 80.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/GHA", + "time_period": "2022", + "value": 70.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/NGA", + "time_period": "2022", + "value": 50.0, + "unit": "USDollar" + } + ], + "rows": [ + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/KEN", + "time_period": "2018/2022", + "value": 0.25, + "unit": null + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/GHA", + "time_period": "2018/2022", + "value": -0.125, + "unit": null + } + ] + }, + { + "name": "growth-as-an-amount", + "description": "An absolute growth is the change in the input's own unit.", + "verb": "growth", + "arguments": { + "by": [ + "destination" + ], + "window": [ + "2018", + "2022" + ], + "change": "absolute" + }, + "data": [ + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/KEN", + "time_period": "2018", + "value": 200.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/KEN", + "time_period": "2022", + "value": 250.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/GHA", + "time_period": "2018", + "value": 80.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/GHA", + "time_period": "2022", + "value": 70.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/NGA", + "time_period": "2022", + "value": 50.0, + "unit": "USDollar" + } + ], + "rows": [ + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/KEN", + "time_period": "2018/2022", + "value": 50.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/GHA", + "time_period": "2018/2022", + "value": -10.0, + "unit": "USDollar" + } + ] + }, + { + "name": "growth-leaves-out-undefined-bases", + "description": "A relative growth leaves out a member whose start or end value is zero or whose values change sign, as zero_or_sign_crossing_base. Kenya grows by half; Ghana starts at zero, Nigeria ends at zero and Senegal crosses from positive to negative, so none of the three has a row.", + "verb": "growth", + "arguments": { + "by": [ + "destination" + ], + "window": [ + "2018", + "2022" + ] + }, + "data": [ + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/KEN", + "time_period": "2018", + "value": 200.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/KEN", + "time_period": "2022", + "value": 300.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/GHA", + "time_period": "2018", + "value": 0.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/GHA", + "time_period": "2022", + "value": 70.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/NGA", + "time_period": "2018", + "value": 50.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/NGA", + "time_period": "2022", + "value": 0.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/SEN", + "time_period": "2018", + "value": 40.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/SEN", + "time_period": "2022", + "value": -10.0, + "unit": "USDollar" + } + ], + "rows": [ + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/KEN", + "time_period": "2018/2022", + "value": 0.5, + "unit": null + } + ] + }, + { + "name": "growth-amount-keeps-zero-bases", + "description": "An absolute growth is defined over a zero or sign-crossing base, so Ghana and Senegal keep their rows; a change that overflows to a non-finite value is left out as non_finite, so Nigeria has none.", + "verb": "growth", + "arguments": { + "by": [ + "destination" + ], + "window": [ + "2018", + "2022" + ], + "change": "absolute" + }, + "data": [ + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/GHA", + "time_period": "2018", + "value": 0.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/GHA", + "time_period": "2022", + "value": 70.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/SEN", + "time_period": "2018", + "value": 40.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/SEN", + "time_period": "2022", + "value": -10.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/NGA", + "time_period": "2018", + "value": -1e+308, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/NGA", + "time_period": "2022", + "value": 1e+308, + "unit": "USDollar" + } + ], + "rows": [ + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/GHA", + "time_period": "2018/2022", + "value": 70.0, + "unit": "USDollar" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/SEN", + "time_period": "2018/2022", + "value": -50.0, + "unit": "USDollar" + } + ] + }, + { + "name": "growth-refuses-two-rows-of-one-member", + "description": "Two rows of Kenya in 2018, one per provider, leave its start value ambiguous under by destination alone, so growth refuses with role_ambiguous.", + "verb": "growth", + "arguments": { + "by": [ + "destination" + ], + "window": [ + "2018", + "2022" + ] + }, + "data": [ + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/KEN", + "time_period": "2018", + "value": 120.0, + "unit": "USDollar", + "provider": "country/GBR" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/KEN", + "time_period": "2018", + "value": 80.0, + "unit": "USDollar", + "provider": "country/FRA" + }, + { + "stat_var": "dcid:Amount_FinancialFlow_ODA_Disbursement", + "destination": "country/KEN", + "time_period": "2022", + "value": 250.0, + "unit": "USDollar", + "provider": "country/GBR" + } + ], + "error": "role_ambiguous" + } + ] +} diff --git a/python/client-contract/src/dfo_client_contract/rules/output-comparison.json b/python/client-contract/src/dfo_client_contract/rules/output-comparison.json new file mode 100644 index 0000000..93b12b1 --- /dev/null +++ b/python/client-contract/src/dfo_client_contract/rules/output-comparison.json @@ -0,0 +1,1618 @@ +{ + "format_version": 1, + "rule": "output-comparison", + "vectors": [ + { + "name": "same-derived-ref", + "left": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_e1edc6acb97615bdd9e813730378ce19", + "recipe": { + "normal_form_hash": "rnf_4e2298354b4323699033912586859aa7", + "replay_key": "rpk_1185a76368dca1b0301c899170c7dff3" + } + }, + "content_version": "ocv_3187efa2294f182bfe7cab296659b74f", + "output_digest": "out_82b81c6278499b468aa3d6a9c5a6aa6b", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2020" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2023" + } + ], + "artifact": { + "digest": "sha256:4c8fec27543c7bedbe9d2c436f42c5a65f721106c128d0a55d2962c2c67de033", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "right": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_e1edc6acb97615bdd9e813730378ce19", + "recipe": { + "normal_form_hash": "rnf_4e2298354b4323699033912586859aa7", + "replay_key": "rpk_1185a76368dca1b0301c899170c7dff3" + } + }, + "content_version": "ocv_3187efa2294f182bfe7cab296659b74f", + "output_digest": "out_82b81c6278499b468aa3d6a9c5a6aa6b", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2020" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2023" + } + ], + "artifact": { + "digest": "sha256:4c8fec27543c7bedbe9d2c436f42c5a65f721106c128d0a55d2962c2c67de033", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "equal": true, + "same_data": true + }, + { + "name": "sentence-and-file-differ", + "left": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_e1edc6acb97615bdd9e813730378ce19", + "recipe": { + "normal_form_hash": "rnf_4e2298354b4323699033912586859aa7", + "replay_key": "rpk_1185a76368dca1b0301c899170c7dff3" + } + }, + "content_version": "ocv_3187efa2294f182bfe7cab296659b74f", + "output_digest": "out_82b81c6278499b468aa3d6a9c5a6aa6b", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2020" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2023" + } + ], + "artifact": { + "digest": "sha256:4c8fec27543c7bedbe9d2c436f42c5a65f721106c128d0a55d2962c2c67de033", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "right": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Another sentence for the same frame.", + "spec": { + "spec_id": "ops_e1edc6acb97615bdd9e813730378ce19", + "recipe": { + "normal_form_hash": "rnf_4e2298354b4323699033912586859aa7", + "replay_key": "rpk_1185a76368dca1b0301c899170c7dff3" + } + }, + "content_version": "ocv_3187efa2294f182bfe7cab296659b74f", + "output_digest": "out_82b81c6278499b468aa3d6a9c5a6aa6b", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2020" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2023" + } + ], + "artifact": { + "digest": "sha256:fe0f5e3c97d3d02c67422f55b887768edee4670f82971fce48579e3cb326e509", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "equal": true, + "same_data": true + }, + { + "name": "rolling-and-fixed-recipe-same-output", + "left": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_e1edc6acb97615bdd9e813730378ce19", + "recipe": { + "normal_form_hash": "rnf_4e2298354b4323699033912586859aa7", + "replay_key": "rpk_1185a76368dca1b0301c899170c7dff3" + } + }, + "content_version": "ocv_3187efa2294f182bfe7cab296659b74f", + "output_digest": "out_82b81c6278499b468aa3d6a9c5a6aa6b", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2020" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2023" + } + ], + "artifact": { + "digest": "sha256:4c8fec27543c7bedbe9d2c436f42c5a65f721106c128d0a55d2962c2c67de033", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "right": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_a41f297b3f33f2a7c2ac6b3cedd4d3ad", + "recipe": { + "normal_form_hash": "rnf_5f5a142beef739694b06899a7f0426a7", + "replay_key": "rpk_4f697e3235da4b9ae1404f112c2c78b6" + } + }, + "content_version": "ocv_62a72043403b8a2497cc184623df87df", + "output_digest": "out_82b81c6278499b468aa3d6a9c5a6aa6b", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [], + "artifact": { + "digest": "sha256:4c8fec27543c7bedbe9d2c436f42c5a65f721106c128d0a55d2962c2c67de033", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "equal": false, + "same_data": true + }, + { + "name": "engine-release-moves-no-number", + "left": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_e1edc6acb97615bdd9e813730378ce19", + "recipe": { + "normal_form_hash": "rnf_4e2298354b4323699033912586859aa7", + "replay_key": "rpk_1185a76368dca1b0301c899170c7dff3" + } + }, + "content_version": "ocv_3187efa2294f182bfe7cab296659b74f", + "output_digest": "out_82b81c6278499b468aa3d6a9c5a6aa6b", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2020" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2023" + } + ], + "artifact": { + "digest": "sha256:4c8fec27543c7bedbe9d2c436f42c5a65f721106c128d0a55d2962c2c67de033", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "right": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_e1edc6acb97615bdd9e813730378ce19", + "recipe": { + "normal_form_hash": "rnf_4e2298354b4323699033912586859aa7", + "replay_key": "rpk_f0b5865b2d90ff6a89f196b5d046bce6" + } + }, + "content_version": "ocv_8f0cd00ac4727431f3a3cbae4abc35e3", + "output_digest": "out_82b81c6278499b468aa3d6a9c5a6aa6b", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.1|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2020" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2023" + } + ], + "artifact": { + "digest": "sha256:4c8fec27543c7bedbe9d2c436f42c5a65f721106c128d0a55d2962c2c67de033", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "equal": false, + "same_data": true + }, + { + "name": "newer-data-moves-the-output", + "left": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_e1edc6acb97615bdd9e813730378ce19", + "recipe": { + "normal_form_hash": "rnf_4e2298354b4323699033912586859aa7", + "replay_key": "rpk_1185a76368dca1b0301c899170c7dff3" + } + }, + "content_version": "ocv_3187efa2294f182bfe7cab296659b74f", + "output_digest": "out_82b81c6278499b468aa3d6a9c5a6aa6b", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2020" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2023" + } + ], + "artifact": { + "digest": "sha256:4c8fec27543c7bedbe9d2c436f42c5a65f721106c128d0a55d2962c2c67de033", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "right": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2021 to 2024. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_16602b92e22d0aaaf8f7e2cf41bcc925", + "recipe": { + "normal_form_hash": "rnf_4e2298354b4323699033912586859aa7", + "replay_key": "rpk_1ea0c543030b65a5680e7c653d6865a0" + } + }, + "content_version": "ocv_846c87c1b91ea260e927b34ab0f96775", + "output_digest": "out_a258ff67f0d7e61a18b8105da50821a0", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_06fe2b28793f70d868d047201166c31d", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2021" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2024" + } + ], + "artifact": { + "digest": "sha256:79fef3b30a845b187dfb0363282ffe64546fa77f72d75ca2dec2647da17f9304", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "equal": false, + "same_data": false + }, + { + "name": "upstream-rebuild-moves-no-number", + "left": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_e1edc6acb97615bdd9e813730378ce19", + "recipe": { + "normal_form_hash": "rnf_4e2298354b4323699033912586859aa7", + "replay_key": "rpk_1185a76368dca1b0301c899170c7dff3" + } + }, + "content_version": "ocv_3187efa2294f182bfe7cab296659b74f", + "output_digest": "out_82b81c6278499b468aa3d6a9c5a6aa6b", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2020" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2023" + } + ], + "artifact": { + "digest": "sha256:4c8fec27543c7bedbe9d2c436f42c5a65f721106c128d0a55d2962c2c67de033", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "right": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_b4b38a6defb8a169442aada645297e5c", + "recipe": { + "normal_form_hash": "rnf_4e2298354b4323699033912586859aa7", + "replay_key": "rpk_7ff8c2c2aacbc373e07f60dc51d614bf" + } + }, + "content_version": "ocv_845384eedabdd3f0ff6c637694981faf", + "output_digest": "out_82b81c6278499b468aa3d6a9c5a6aa6b", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_ddc47df6c1e8526631c61fbe12affc09", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2020" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2023" + } + ], + "artifact": { + "digest": "sha256:4c8fec27543c7bedbe9d2c436f42c5a65f721106c128d0a55d2962c2c67de033", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "equal": false, + "same_data": true + }, + { + "name": "derived-file-format-differs", + "left": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_e1edc6acb97615bdd9e813730378ce19", + "recipe": { + "normal_form_hash": "rnf_4e2298354b4323699033912586859aa7", + "replay_key": "rpk_1185a76368dca1b0301c899170c7dff3" + } + }, + "content_version": "ocv_3187efa2294f182bfe7cab296659b74f", + "output_digest": "out_82b81c6278499b468aa3d6a9c5a6aa6b", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2020" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2023" + } + ], + "artifact": { + "digest": "sha256:4c8fec27543c7bedbe9d2c436f42c5a65f721106c128d0a55d2962c2c67de033", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "right": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_e1edc6acb97615bdd9e813730378ce19", + "recipe": { + "normal_form_hash": "rnf_4e2298354b4323699033912586859aa7", + "replay_key": "rpk_1185a76368dca1b0301c899170c7dff3" + } + }, + "content_version": "ocv_3187efa2294f182bfe7cab296659b74f", + "output_digest": "out_82b81c6278499b468aa3d6a9c5a6aa6b", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2020" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2023" + } + ], + "artifact": { + "digest": "sha256:78892fc7549d08ac2af6f4be5362f9b232db89a2482980df97204655ba6ef97a", + "rendering": { + "format": "csv", + "bundle": false + } + } + } + ] + }, + "equal": false, + "same_data": true + }, + { + "name": "same-dataset-ref", + "left": { + "format_version": 1, + "members": [ + { + "parts": [], + "sentence": "Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Current US dollars · OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "definition": { + "schema_version": 1, + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "identity": { + "identity_version": "spec1", + "shape": { + "kind": "statvar_schema", + "population_type": "FinancialFlow", + "measured_property": "amount", + "stat_type": "measuredValue", + "measurement_qualifier": "Nominal" + }, + "slots": [ + { + "property": "crsCooperationModality", + "binding": { + "state": "unbound" + } + }, + { + "property": "crsPurpose", + "binding": { + "state": "value", + "ref": "CRSPurposeCode12220" + } + }, + { + "property": "crsTypeOfFinance", + "binding": { + "state": "unbound" + } + }, + { + "property": "crsTypeOfFlow", + "binding": { + "state": "value", + "ref": "CRSTypeOfFlow10" + } + }, + { + "property": "financialFlowType", + "binding": { + "state": "value", + "ref": "Disbursement" + } + }, + { + "property": "nettingBasis", + "binding": { + "state": "value", + "ref": "Gross" + } + } + ], + "entities": { + "destination": [ + "country/NGA" + ], + "origin": [ + "country/DEU" + ] + }, + "requested_window": { + "kind": "range", + "start_year": 2020, + "end_year": 2023 + } + } + } + }, + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "artifact": { + "digest": "sha256:c44200969f19d73104d1bc24ebca905e59f749c2939234cdde10ba6f5a131170", + "rendering": { + "format": "parquet", + "bundle": false, + "without_party_totals": true + } + } + } + ] + }, + "right": { + "format_version": 1, + "members": [ + { + "parts": [], + "sentence": "Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Current US dollars · OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "definition": { + "schema_version": 1, + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "identity": { + "identity_version": "spec1", + "shape": { + "kind": "statvar_schema", + "population_type": "FinancialFlow", + "measured_property": "amount", + "stat_type": "measuredValue", + "measurement_qualifier": "Nominal" + }, + "slots": [ + { + "property": "crsCooperationModality", + "binding": { + "state": "unbound" + } + }, + { + "property": "crsPurpose", + "binding": { + "state": "value", + "ref": "CRSPurposeCode12220" + } + }, + { + "property": "crsTypeOfFinance", + "binding": { + "state": "unbound" + } + }, + { + "property": "crsTypeOfFlow", + "binding": { + "state": "value", + "ref": "CRSTypeOfFlow10" + } + }, + { + "property": "financialFlowType", + "binding": { + "state": "value", + "ref": "Disbursement" + } + }, + { + "property": "nettingBasis", + "binding": { + "state": "value", + "ref": "Gross" + } + } + ], + "entities": { + "destination": [ + "country/NGA" + ], + "origin": [ + "country/DEU" + ] + }, + "requested_window": { + "kind": "range", + "start_year": 2020, + "end_year": 2023 + } + } + } + }, + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "artifact": { + "digest": "sha256:c44200969f19d73104d1bc24ebca905e59f749c2939234cdde10ba6f5a131170", + "rendering": { + "format": "parquet", + "bundle": false, + "without_party_totals": true + } + } + } + ] + }, + "equal": true, + "same_data": true + }, + { + "name": "dataset-layout-differs", + "left": { + "format_version": 1, + "members": [ + { + "parts": [], + "sentence": "Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Current US dollars · OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "definition": { + "schema_version": 1, + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "identity": { + "identity_version": "spec1", + "shape": { + "kind": "statvar_schema", + "population_type": "FinancialFlow", + "measured_property": "amount", + "stat_type": "measuredValue", + "measurement_qualifier": "Nominal" + }, + "slots": [ + { + "property": "crsCooperationModality", + "binding": { + "state": "unbound" + } + }, + { + "property": "crsPurpose", + "binding": { + "state": "value", + "ref": "CRSPurposeCode12220" + } + }, + { + "property": "crsTypeOfFinance", + "binding": { + "state": "unbound" + } + }, + { + "property": "crsTypeOfFlow", + "binding": { + "state": "value", + "ref": "CRSTypeOfFlow10" + } + }, + { + "property": "financialFlowType", + "binding": { + "state": "value", + "ref": "Disbursement" + } + }, + { + "property": "nettingBasis", + "binding": { + "state": "value", + "ref": "Gross" + } + } + ], + "entities": { + "destination": [ + "country/NGA" + ], + "origin": [ + "country/DEU" + ] + }, + "requested_window": { + "kind": "range", + "start_year": 2020, + "end_year": 2023 + } + } + } + }, + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "artifact": { + "digest": "sha256:c44200969f19d73104d1bc24ebca905e59f749c2939234cdde10ba6f5a131170", + "rendering": { + "format": "parquet", + "bundle": false, + "without_party_totals": true + } + } + } + ] + }, + "right": { + "format_version": 1, + "members": [ + { + "parts": [], + "sentence": "Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Current US dollars · OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "definition": { + "schema_version": 1, + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "identity": { + "identity_version": "spec1", + "shape": { + "kind": "statvar_schema", + "population_type": "FinancialFlow", + "measured_property": "amount", + "stat_type": "measuredValue", + "measurement_qualifier": "Nominal" + }, + "slots": [ + { + "property": "crsCooperationModality", + "binding": { + "state": "unbound" + } + }, + { + "property": "crsPurpose", + "binding": { + "state": "value", + "ref": "CRSPurposeCode12220" + } + }, + { + "property": "crsTypeOfFinance", + "binding": { + "state": "unbound" + } + }, + { + "property": "crsTypeOfFlow", + "binding": { + "state": "value", + "ref": "CRSTypeOfFlow10" + } + }, + { + "property": "financialFlowType", + "binding": { + "state": "value", + "ref": "Disbursement" + } + }, + { + "property": "nettingBasis", + "binding": { + "state": "value", + "ref": "Gross" + } + } + ], + "entities": { + "destination": [ + "country/NGA" + ], + "origin": [ + "country/DEU" + ] + }, + "requested_window": { + "kind": "range", + "start_year": 2020, + "end_year": 2023 + } + } + } + }, + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "artifact": { + "digest": "sha256:c44200969f19d73104d1bc24ebca905e59f749c2939234cdde10ba6f5a131170", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + } + ] + }, + "equal": false, + "same_data": true + }, + { + "name": "dataset-party-totals-differ", + "left": { + "format_version": 1, + "members": [ + { + "parts": [], + "sentence": "Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Current US dollars · OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "definition": { + "schema_version": 1, + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "identity": { + "identity_version": "spec1", + "shape": { + "kind": "statvar_schema", + "population_type": "FinancialFlow", + "measured_property": "amount", + "stat_type": "measuredValue", + "measurement_qualifier": "Nominal" + }, + "slots": [ + { + "property": "crsCooperationModality", + "binding": { + "state": "unbound" + } + }, + { + "property": "crsPurpose", + "binding": { + "state": "value", + "ref": "CRSPurposeCode12220" + } + }, + { + "property": "crsTypeOfFinance", + "binding": { + "state": "unbound" + } + }, + { + "property": "crsTypeOfFlow", + "binding": { + "state": "value", + "ref": "CRSTypeOfFlow10" + } + }, + { + "property": "financialFlowType", + "binding": { + "state": "value", + "ref": "Disbursement" + } + }, + { + "property": "nettingBasis", + "binding": { + "state": "value", + "ref": "Gross" + } + } + ], + "entities": { + "destination": [ + "country/NGA" + ], + "origin": [ + "country/DEU" + ] + }, + "requested_window": { + "kind": "range", + "start_year": 2020, + "end_year": 2023 + } + } + } + }, + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "artifact": { + "digest": "sha256:c44200969f19d73104d1bc24ebca905e59f749c2939234cdde10ba6f5a131170", + "rendering": { + "format": "parquet", + "bundle": false, + "without_party_totals": true + } + } + } + ] + }, + "right": { + "format_version": 1, + "members": [ + { + "parts": [], + "sentence": "Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Current US dollars · OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "definition": { + "schema_version": 1, + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "identity": { + "identity_version": "spec1", + "shape": { + "kind": "statvar_schema", + "population_type": "FinancialFlow", + "measured_property": "amount", + "stat_type": "measuredValue", + "measurement_qualifier": "Nominal" + }, + "slots": [ + { + "property": "crsCooperationModality", + "binding": { + "state": "unbound" + } + }, + { + "property": "crsPurpose", + "binding": { + "state": "value", + "ref": "CRSPurposeCode12220" + } + }, + { + "property": "crsTypeOfFinance", + "binding": { + "state": "unbound" + } + }, + { + "property": "crsTypeOfFlow", + "binding": { + "state": "value", + "ref": "CRSTypeOfFlow10" + } + }, + { + "property": "financialFlowType", + "binding": { + "state": "value", + "ref": "Disbursement" + } + }, + { + "property": "nettingBasis", + "binding": { + "state": "value", + "ref": "Gross" + } + } + ], + "entities": { + "destination": [ + "country/NGA" + ], + "origin": [ + "country/DEU" + ] + }, + "requested_window": { + "kind": "range", + "start_year": 2020, + "end_year": 2023 + } + } + } + }, + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "artifact": { + "digest": "sha256:c44200969f19d73104d1bc24ebca905e59f749c2939234cdde10ba6f5a131170", + "rendering": { + "format": "parquet", + "bundle": false + } + } + } + ] + }, + "equal": false, + "same_data": false + }, + { + "name": "dataset-and-derived", + "left": { + "format_version": 1, + "members": [ + { + "parts": [], + "sentence": "Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Current US dollars · OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "definition": { + "schema_version": 1, + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "identity": { + "identity_version": "spec1", + "shape": { + "kind": "statvar_schema", + "population_type": "FinancialFlow", + "measured_property": "amount", + "stat_type": "measuredValue", + "measurement_qualifier": "Nominal" + }, + "slots": [ + { + "property": "crsCooperationModality", + "binding": { + "state": "unbound" + } + }, + { + "property": "crsPurpose", + "binding": { + "state": "value", + "ref": "CRSPurposeCode12220" + } + }, + { + "property": "crsTypeOfFinance", + "binding": { + "state": "unbound" + } + }, + { + "property": "crsTypeOfFlow", + "binding": { + "state": "value", + "ref": "CRSTypeOfFlow10" + } + }, + { + "property": "financialFlowType", + "binding": { + "state": "value", + "ref": "Disbursement" + } + }, + { + "property": "nettingBasis", + "binding": { + "state": "value", + "ref": "Gross" + } + } + ], + "entities": { + "destination": [ + "country/NGA" + ], + "origin": [ + "country/DEU" + ] + }, + "requested_window": { + "kind": "range", + "start_year": 2020, + "end_year": 2023 + } + } + } + }, + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "artifact": { + "digest": "sha256:c44200969f19d73104d1bc24ebca905e59f749c2939234cdde10ba6f5a131170", + "rendering": { + "format": "parquet", + "bundle": false, + "without_party_totals": true + } + } + } + ] + }, + "right": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_e1edc6acb97615bdd9e813730378ce19", + "recipe": { + "normal_form_hash": "rnf_4e2298354b4323699033912586859aa7", + "replay_key": "rpk_1185a76368dca1b0301c899170c7dff3" + } + }, + "content_version": "ocv_3187efa2294f182bfe7cab296659b74f", + "output_digest": "out_82b81c6278499b468aa3d6a9c5a6aa6b", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2020" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2023" + } + ], + "artifact": { + "digest": "sha256:4c8fec27543c7bedbe9d2c436f42c5a65f721106c128d0a55d2962c2c67de033", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "equal": false, + "same_data": false + }, + { + "name": "member-count-differs", + "left": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_e1edc6acb97615bdd9e813730378ce19", + "recipe": { + "normal_form_hash": "rnf_4e2298354b4323699033912586859aa7", + "replay_key": "rpk_1185a76368dca1b0301c899170c7dff3" + } + }, + "content_version": "ocv_3187efa2294f182bfe7cab296659b74f", + "output_digest": "out_82b81c6278499b468aa3d6a9c5a6aa6b", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2020" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2023" + } + ], + "artifact": { + "digest": "sha256:4c8fec27543c7bedbe9d2c436f42c5a65f721106c128d0a55d2962c2c67de033", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "right": { + "format_version": 2, + "members": [ + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_e1edc6acb97615bdd9e813730378ce19", + "recipe": { + "normal_form_hash": "rnf_4e2298354b4323699033912586859aa7", + "replay_key": "rpk_1185a76368dca1b0301c899170c7dff3" + } + }, + "content_version": "ocv_3187efa2294f182bfe7cab296659b74f", + "output_digest": "out_82b81c6278499b468aa3d6a9c5a6aa6b", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [ + { + "anchor": "latest", + "offset": -3, + "rule": "half1", + "period": "2020" + }, + { + "anchor": "latest", + "offset": 0, + "rule": "half1", + "period": "2023" + } + ], + "artifact": { + "digest": "sha256:4c8fec27543c7bedbe9d2c436f42c5a65f721106c128d0a55d2962c2c67de033", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + }, + { + "parts": [], + "sentence": "Change in Germany's gross ODA disbursements to Nigeria for basic health care, 2020 to 2023. Computed by DFO from OECD Creditor Reporting System (CRS).", + "spec": { + "spec_id": "ops_a41f297b3f33f2a7c2ac6b3cedd4d3ad", + "recipe": { + "normal_form_hash": "rnf_5f5a142beef739694b06899a7f0426a7", + "replay_key": "rpk_4f697e3235da4b9ae1404f112c2c78b6" + } + }, + "content_version": "ocv_62a72043403b8a2497cc184623df87df", + "output_digest": "out_82b81c6278499b468aa3d6a9c5a6aa6b", + "roots": [ + { + "spec_id": "spec_cb5baa4161de718f18583060674dbca6", + "content_version": "cv_bf242138f8d0a892c97da9373d1626f4", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame", + "without_party_totals": true + } + } + ], + "execution_identity": "0.9.0|1.1.0|0.2.0", + "resolved_periods": [], + "artifact": { + "digest": "sha256:4c8fec27543c7bedbe9d2c436f42c5a65f721106c128d0a55d2962c2c67de033", + "rendering": { + "format": "parquet", + "bundle": false, + "layout": "frame" + } + } + } + ] + }, + "equal": false, + "same_data": false + } + ] +} diff --git a/python/client-contract/src/dfo_client_contract/rules/output-digest.json b/python/client-contract/src/dfo_client_contract/rules/output-digest.json new file mode 100644 index 0000000..fa57304 --- /dev/null +++ b/python/client-contract/src/dfo_client_contract/rules/output-digest.json @@ -0,0 +1,900 @@ +{ + "format_version": 1, + "rule": "output-digest", + "vectors": [ + { + "name": "two-rows", + "frame": { + "columns": [ + "stat_var", + "destination", + "origin", + "time_period", + "value", + "unit", + "status" + ], + "rows": [ + [ + "dcid:Amount_FinancialFlow_ODA_Disbursement", + "country/KEN", + "country/DEU", + "2022", + 120.5, + "USDollar", + null + ], + [ + "dcid:Amount_FinancialFlow_ODA_Disbursement", + "country/NGA", + "country/DEU", + "2022", + 98.25, + "USDollar", + null + ] + ] + }, + "document": "{\"columns\":[\"stat_var\",\"destination\",\"origin\",\"time_period\",\"value\",\"unit\"],\"digest_version\":\"out2\",\"rows\":[[\"dcid:Amount_FinancialFlow_ODA_Disbursement\",\"country/KEN\",\"country/DEU\",\"2022\",120.5,\"USDollar\"],[\"dcid:Amount_FinancialFlow_ODA_Disbursement\",\"country/NGA\",\"country/DEU\",\"2022\",98.25,\"USDollar\"]]}", + "digest": "out_fb35c1b4f4a67b6ca2ecd8f334220dfa" + }, + { + "name": "another-stat-var", + "frame": { + "columns": [ + "stat_var", + "destination", + "origin", + "time_period", + "value", + "unit", + "status" + ], + "rows": [ + [ + "dcid:ops_4b1e0c9a7d2f3e5b6a8c9d0e1f2a3b4c", + "country/KEN", + "country/DEU", + "2022", + 120.5, + "USDollar", + null + ], + [ + "dcid:ops_4b1e0c9a7d2f3e5b6a8c9d0e1f2a3b4c", + "country/NGA", + "country/DEU", + "2022", + 98.25, + "USDollar", + null + ] + ] + }, + "document": "{\"columns\":[\"stat_var\",\"destination\",\"origin\",\"time_period\",\"value\",\"unit\"],\"digest_version\":\"out2\",\"rows\":[[\"dcid:ops_4b1e0c9a7d2f3e5b6a8c9d0e1f2a3b4c\",\"country/KEN\",\"country/DEU\",\"2022\",120.5,\"USDollar\"],[\"dcid:ops_4b1e0c9a7d2f3e5b6a8c9d0e1f2a3b4c\",\"country/NGA\",\"country/DEU\",\"2022\",98.25,\"USDollar\"]]}", + "digest": "out_2753cf795f19e151b39cc9c6d14e698d" + }, + { + "name": "rows-in-another-order", + "frame": { + "columns": [ + "stat_var", + "destination", + "origin", + "time_period", + "value", + "unit", + "status" + ], + "rows": [ + [ + "dcid:Amount_FinancialFlow_ODA_Disbursement", + "country/NGA", + "country/DEU", + "2022", + 98.25, + "USDollar", + null + ], + [ + "dcid:Amount_FinancialFlow_ODA_Disbursement", + "country/KEN", + "country/DEU", + "2022", + 120.5, + "USDollar", + null + ] + ] + }, + "document": "{\"columns\":[\"stat_var\",\"destination\",\"origin\",\"time_period\",\"value\",\"unit\"],\"digest_version\":\"out2\",\"rows\":[[\"dcid:Amount_FinancialFlow_ODA_Disbursement\",\"country/KEN\",\"country/DEU\",\"2022\",120.5,\"USDollar\"],[\"dcid:Amount_FinancialFlow_ODA_Disbursement\",\"country/NGA\",\"country/DEU\",\"2022\",98.25,\"USDollar\"]]}", + "digest": "out_fb35c1b4f4a67b6ca2ecd8f334220dfa" + }, + { + "name": "facets-left-out", + "frame": { + "columns": [ + "stat_var", + "destination", + "origin", + "time_period", + "value", + "unit", + "measurementMethod", + "provenance", + "status" + ], + "rows": [ + [ + "dcid:Amount_FinancialFlow_ODA_Disbursement", + "country/KEN", + "country/DEU", + "2022", + 120.5, + "USDollar", + "dcid:OECDReported", + "dcid:OECD_DAC2a", + null + ], + [ + "dcid:Amount_FinancialFlow_ODA_Disbursement", + "country/NGA", + "country/DEU", + "2022", + 98.25, + "USDollar", + "dcid:OECDReported", + "dcid:OECD_DAC2a", + null + ] + ] + }, + "document": "{\"columns\":[\"stat_var\",\"destination\",\"origin\",\"time_period\",\"value\",\"unit\"],\"digest_version\":\"out2\",\"rows\":[[\"dcid:Amount_FinancialFlow_ODA_Disbursement\",\"country/KEN\",\"country/DEU\",\"2022\",120.5,\"USDollar\"],[\"dcid:Amount_FinancialFlow_ODA_Disbursement\",\"country/NGA\",\"country/DEU\",\"2022\",98.25,\"USDollar\"]]}", + "digest": "out_fb35c1b4f4a67b6ca2ecd8f334220dfa" + }, + { + "name": "another-value", + "frame": { + "columns": [ + "stat_var", + "destination", + "origin", + "time_period", + "value", + "unit", + "status" + ], + "rows": [ + [ + "dcid:Amount_FinancialFlow_ODA_Disbursement", + "country/KEN", + "country/DEU", + "2022", + 120.5, + "USDollar", + null + ], + [ + "dcid:Amount_FinancialFlow_ODA_Disbursement", + "country/NGA", + "country/DEU", + "2022", + 98.5, + "USDollar", + null + ] + ] + }, + "document": "{\"columns\":[\"stat_var\",\"destination\",\"origin\",\"time_period\",\"value\",\"unit\"],\"digest_version\":\"out2\",\"rows\":[[\"dcid:Amount_FinancialFlow_ODA_Disbursement\",\"country/KEN\",\"country/DEU\",\"2022\",120.5,\"USDollar\"],[\"dcid:Amount_FinancialFlow_ODA_Disbursement\",\"country/NGA\",\"country/DEU\",\"2022\",98.5,\"USDollar\"]]}", + "digest": "out_a2e2612bf1d8ec3e5e94ccb9d601e028" + }, + { + "name": "another-unit", + "frame": { + "columns": [ + "stat_var", + "destination", + "origin", + "time_period", + "value", + "unit", + "status" + ], + "rows": [ + [ + "dcid:Amount_FinancialFlow_ODA_Disbursement", + "country/KEN", + "country/DEU", + "2022", + 120.5, + "Euro", + null + ], + [ + "dcid:Amount_FinancialFlow_ODA_Disbursement", + "country/NGA", + "country/DEU", + "2022", + 98.25, + "Euro", + null + ] + ] + }, + "document": "{\"columns\":[\"stat_var\",\"destination\",\"origin\",\"time_period\",\"value\",\"unit\"],\"digest_version\":\"out2\",\"rows\":[[\"dcid:Amount_FinancialFlow_ODA_Disbursement\",\"country/KEN\",\"country/DEU\",\"2022\",120.5,\"Euro\"],[\"dcid:Amount_FinancialFlow_ODA_Disbursement\",\"country/NGA\",\"country/DEU\",\"2022\",98.25,\"Euro\"]]}", + "digest": "out_f79a304d277ad9a080a37d71f4e5af39" + }, + { + "name": "another-period", + "frame": { + "columns": [ + "stat_var", + "destination", + "origin", + "time_period", + "value", + "unit", + "status" + ], + "rows": [ + [ + "dcid:Amount_FinancialFlow_ODA_Disbursement", + "country/KEN", + "country/DEU", + "2023", + 120.5, + "USDollar", + null + ], + [ + "dcid:Amount_FinancialFlow_ODA_Disbursement", + "country/NGA", + "country/DEU", + "2022", + 98.25, + "USDollar", + null + ] + ] + }, + "document": "{\"columns\":[\"stat_var\",\"destination\",\"origin\",\"time_period\",\"value\",\"unit\"],\"digest_version\":\"out2\",\"rows\":[[\"dcid:Amount_FinancialFlow_ODA_Disbursement\",\"country/KEN\",\"country/DEU\",\"2023\",120.5,\"USDollar\"],[\"dcid:Amount_FinancialFlow_ODA_Disbursement\",\"country/NGA\",\"country/DEU\",\"2022\",98.25,\"USDollar\"]]}", + "digest": "out_51e11cc3d1e0228562e557a15a9ed8cf" + }, + { + "name": "another-role-name", + "frame": { + "columns": [ + "stat_var", + "observationAbout", + "time_period", + "value", + "unit", + "status" + ], + "rows": [ + [ + "dcid:Amount_FinancialFlow_ODA_Disbursement", + "country/KEN", + "2022", + 120.5, + "USDollar", + null + ] + ] + }, + "document": "{\"columns\":[\"stat_var\",\"observationAbout\",\"time_period\",\"value\",\"unit\"],\"digest_version\":\"out2\",\"rows\":[[\"dcid:Amount_FinancialFlow_ODA_Disbursement\",\"country/KEN\",\"2022\",120.5,\"USDollar\"]]}", + "digest": "out_1694c016b8cd08bb08e68151d4c2a9bc" + }, + { + "name": "converted-unit", + "frame": { + "columns": [ + "stat_var", + "destination", + "origin", + "time_period", + "value", + "unit", + "status" + ], + "rows": [ + [ + "dcid:Amount_FinancialFlow_ODA_Disbursement", + "country/KEN", + "country/DEU", + "2022", + 120.5, + { + "currency": "EUR", + "prices": "constant", + "base_year": 2021 + }, + null + ] + ] + }, + "document": "{\"columns\":[\"stat_var\",\"destination\",\"origin\",\"time_period\",\"value\",\"unit\"],\"digest_version\":\"out2\",\"rows\":[[\"dcid:Amount_FinancialFlow_ODA_Disbursement\",\"country/KEN\",\"country/DEU\",\"2022\",120.5,{\"base_year\":2021,\"currency\":\"EUR\",\"prices\":\"constant\"}]]}", + "digest": "out_e7f45286159d090b59d8c07efba3311c" + }, + { + "name": "converted-unit-recipe-left-out", + "frame": { + "columns": [ + "stat_var", + "destination", + "origin", + "time_period", + "value", + "unit", + "status" + ], + "rows": [ + [ + "dcid:Amount_FinancialFlow_ODA_Disbursement", + "country/KEN", + "country/DEU", + "2022", + 120.5, + { + "currency": "EUR", + "prices": "constant", + "base_year": 2021, + "recipe": "fx_oecd_2025_10" + }, + null + ] + ] + }, + "document": "{\"columns\":[\"stat_var\",\"destination\",\"origin\",\"time_period\",\"value\",\"unit\"],\"digest_version\":\"out2\",\"rows\":[[\"dcid:Amount_FinancialFlow_ODA_Disbursement\",\"country/KEN\",\"country/DEU\",\"2022\",120.5,{\"base_year\":2021,\"currency\":\"EUR\",\"prices\":\"constant\"}]]}", + "digest": "out_e7f45286159d090b59d8c07efba3311c" + }, + { + "name": "composed-unit-recipe-left-out", + "frame": { + "columns": [ + "stat_var", + "destination", + "time_period", + "value", + "unit", + "status" + ], + "rows": [ + [ + "dcid:ops_4b1e0c9a7d2f3e5b6a8c9d0e1f2a3b4c", + "country/KEN", + "2022", + 2.25, + { + "numerator": { + "currency": "EUR", + "prices": "constant", + "base_year": 2021, + "recipe": "fx_oecd_2025_10" + }, + "denominator": { + "stat_var": "dcid:Count_Person" + } + }, + null + ] + ] + }, + "document": "{\"columns\":[\"stat_var\",\"destination\",\"time_period\",\"value\",\"unit\"],\"digest_version\":\"out2\",\"rows\":[[\"dcid:ops_4b1e0c9a7d2f3e5b6a8c9d0e1f2a3b4c\",\"country/KEN\",\"2022\",2.25,{\"denominator\":{\"stat_var\":\"dcid:Count_Person\"},\"numerator\":{\"base_year\":2021,\"currency\":\"EUR\",\"prices\":\"constant\"}}]]}", + "digest": "out_9dcf831992537dbf2dd2cabe816ee483" + }, + { + "name": "composed-unit", + "frame": { + "columns": [ + "stat_var", + "destination", + "time_period", + "value", + "unit", + "status" + ], + "rows": [ + [ + "dcid:ops_4b1e0c9a7d2f3e5b6a8c9d0e1f2a3b4c", + "country/KEN", + "2022", + 2.25, + { + "numerator": { + "currency": "EUR", + "prices": "constant", + "base_year": 2021 + }, + "denominator": { + "stat_var": "dcid:Count_Person" + } + }, + null + ] + ] + }, + "document": "{\"columns\":[\"stat_var\",\"destination\",\"time_period\",\"value\",\"unit\"],\"digest_version\":\"out2\",\"rows\":[[\"dcid:ops_4b1e0c9a7d2f3e5b6a8c9d0e1f2a3b4c\",\"country/KEN\",\"2022\",2.25,{\"denominator\":{\"stat_var\":\"dcid:Count_Person\"},\"numerator\":{\"base_year\":2021,\"currency\":\"EUR\",\"prices\":\"constant\"}}]]}", + "digest": "out_9dcf831992537dbf2dd2cabe816ee483" + }, + { + "name": "join-of-two-variables", + "frame": { + "columns": [ + "stat_var", + "destination", + "time_period", + "value", + "unit", + "status" + ], + "rows": [ + [ + "dcid:Amount_FinancialFlow_ODA_Disbursement", + "country/KEN", + "2022", + 120.5, + "USDollar", + null + ], + [ + "dcid:Amount_EconomicActivity_GrossNationalIncome", + "country/KEN", + "2022", + 104000.0, + "USDollar", + null + ] + ] + }, + "document": "{\"columns\":[\"stat_var\",\"destination\",\"time_period\",\"value\",\"unit\"],\"digest_version\":\"out2\",\"rows\":[[\"dcid:Amount_EconomicActivity_GrossNationalIncome\",\"country/KEN\",\"2022\",104000,\"USDollar\"],[\"dcid:Amount_FinancialFlow_ODA_Disbursement\",\"country/KEN\",\"2022\",120.5,\"USDollar\"]]}", + "digest": "out_e010a9208379d7d1acec4994d4218d18" + }, + { + "name": "join-sides-swapped", + "frame": { + "columns": [ + "stat_var", + "destination", + "time_period", + "value", + "unit", + "status" + ], + "rows": [ + [ + "dcid:Amount_EconomicActivity_GrossNationalIncome", + "country/KEN", + "2022", + 104000.0, + "USDollar", + null + ], + [ + "dcid:Amount_FinancialFlow_ODA_Disbursement", + "country/KEN", + "2022", + 120.5, + "USDollar", + null + ] + ] + }, + "document": "{\"columns\":[\"stat_var\",\"destination\",\"time_period\",\"value\",\"unit\"],\"digest_version\":\"out2\",\"rows\":[[\"dcid:Amount_EconomicActivity_GrossNationalIncome\",\"country/KEN\",\"2022\",104000,\"USDollar\"],[\"dcid:Amount_FinancialFlow_ODA_Disbursement\",\"country/KEN\",\"2022\",120.5,\"USDollar\"]]}", + "digest": "out_e010a9208379d7d1acec4994d4218d18" + }, + { + "name": "join-values-under-swapped-variables", + "frame": { + "columns": [ + "stat_var", + "destination", + "time_period", + "value", + "unit", + "status" + ], + "rows": [ + [ + "dcid:Amount_EconomicActivity_GrossNationalIncome", + "country/KEN", + "2022", + 120.5, + "USDollar", + null + ], + [ + "dcid:Amount_FinancialFlow_ODA_Disbursement", + "country/KEN", + "2022", + 104000.0, + "USDollar", + null + ] + ] + }, + "document": "{\"columns\":[\"stat_var\",\"destination\",\"time_period\",\"value\",\"unit\"],\"digest_version\":\"out2\",\"rows\":[[\"dcid:Amount_EconomicActivity_GrossNationalIncome\",\"country/KEN\",\"2022\",120.5,\"USDollar\"],[\"dcid:Amount_FinancialFlow_ODA_Disbursement\",\"country/KEN\",\"2022\",104000,\"USDollar\"]]}", + "digest": "out_ef49e17ee52f27ccdccb6709fe12a404" + }, + { + "name": "quotient-of-variable-named", + "frame": { + "columns": [ + "stat_var", + "financingRecipient", + "financingRecipient_party_kind", + "time_period", + "value", + "unit", + "status" + ], + "rows": [ + [ + "dcid:ops_f72f69cb463d4fdbeb6751800ba2544a", + "country/KEN", + "party", + "2021", + 0.2, + { + "numerator": "USDollar", + "denominator": "Person" + }, + null + ], + [ + "dcid:ops_f72f69cb463d4fdbeb6751800ba2544a", + "country/KEN", + "party", + "2022", + 0.2, + { + "numerator": "USDollar", + "denominator": "Person" + }, + null + ], + [ + "dcid:ops_f72f69cb463d4fdbeb6751800ba2544a", + "country/NGA", + "party", + "2021", + 0.1, + { + "numerator": "USDollar", + "denominator": "Person" + }, + null + ], + [ + "dcid:ops_f72f69cb463d4fdbeb6751800ba2544a", + "country/NGA", + "party", + "2022", + 0.14285714285714285, + { + "numerator": "USDollar", + "denominator": "Person" + }, + null + ] + ] + }, + "variables": { + "ops_f72f69cb463d4fdbeb6751800ba2544a": "mv_d355e59c50ff8cf07e86a4da12b64297" + }, + "name_documents": { + "mv_d355e59c50ff8cf07e86a4da12b64297": "{\"denominator_frame_ref\":[\"dcid:Count_Person\"],\"frame_ref\":[\"dcid:Amount_ODA\"],\"verb\":\"quotient_of\"}" + }, + "document": "{\"columns\":[\"stat_var\",\"financingRecipient\",\"financingRecipient_party_kind\",\"time_period\",\"value\",\"unit\"],\"digest_version\":\"out2\",\"rows\":[[\"mv_d355e59c50ff8cf07e86a4da12b64297\",\"country/KEN\",\"party\",\"2021\",0.2,{\"denominator\":\"Person\",\"numerator\":\"USDollar\"}],[\"mv_d355e59c50ff8cf07e86a4da12b64297\",\"country/KEN\",\"party\",\"2022\",0.2,{\"denominator\":\"Person\",\"numerator\":\"USDollar\"}],[\"mv_d355e59c50ff8cf07e86a4da12b64297\",\"country/NGA\",\"party\",\"2021\",0.1,{\"denominator\":\"Person\",\"numerator\":\"USDollar\"}],[\"mv_d355e59c50ff8cf07e86a4da12b64297\",\"country/NGA\",\"party\",\"2022\",0.14285714285714285,{\"denominator\":\"Person\",\"numerator\":\"USDollar\"}]]}", + "digest": "out_e64e2b613cf8ceb4e3b0cd93a54cf5ce" + }, + { + "name": "filter-then-quotient-of", + "frame": { + "columns": [ + "stat_var", + "financingRecipient", + "financingRecipient_party_kind", + "time_period", + "value", + "unit", + "status" + ], + "rows": [ + [ + "dcid:ops_664db264b67e0ba2cec6b95c18ea4635", + "country/KEN", + "party", + "2021", + 0.2, + { + "numerator": "USDollar", + "denominator": "Person" + }, + null + ], + [ + "dcid:ops_664db264b67e0ba2cec6b95c18ea4635", + "country/KEN", + "party", + "2022", + 0.2, + { + "numerator": "USDollar", + "denominator": "Person" + }, + null + ], + [ + "dcid:ops_664db264b67e0ba2cec6b95c18ea4635", + "country/NGA", + "party", + "2021", + 0.1, + { + "numerator": "USDollar", + "denominator": "Person" + }, + null + ], + [ + "dcid:ops_664db264b67e0ba2cec6b95c18ea4635", + "country/NGA", + "party", + "2022", + 0.14285714285714285, + { + "numerator": "USDollar", + "denominator": "Person" + }, + null + ] + ] + }, + "variables": { + "ops_664db264b67e0ba2cec6b95c18ea4635": "mv_d355e59c50ff8cf07e86a4da12b64297" + }, + "name_documents": { + "mv_d355e59c50ff8cf07e86a4da12b64297": "{\"denominator_frame_ref\":[\"dcid:Count_Person\"],\"frame_ref\":[\"dcid:Amount_ODA\"],\"verb\":\"quotient_of\"}" + }, + "document": "{\"columns\":[\"stat_var\",\"financingRecipient\",\"financingRecipient_party_kind\",\"time_period\",\"value\",\"unit\"],\"digest_version\":\"out2\",\"rows\":[[\"mv_d355e59c50ff8cf07e86a4da12b64297\",\"country/KEN\",\"party\",\"2021\",0.2,{\"denominator\":\"Person\",\"numerator\":\"USDollar\"}],[\"mv_d355e59c50ff8cf07e86a4da12b64297\",\"country/KEN\",\"party\",\"2022\",0.2,{\"denominator\":\"Person\",\"numerator\":\"USDollar\"}],[\"mv_d355e59c50ff8cf07e86a4da12b64297\",\"country/NGA\",\"party\",\"2021\",0.1,{\"denominator\":\"Person\",\"numerator\":\"USDollar\"}],[\"mv_d355e59c50ff8cf07e86a4da12b64297\",\"country/NGA\",\"party\",\"2022\",0.14285714285714285,{\"denominator\":\"Person\",\"numerator\":\"USDollar\"}]]}", + "digest": "out_e64e2b613cf8ceb4e3b0cd93a54cf5ce" + }, + { + "name": "quotient-of-sides-swapped", + "frame": { + "columns": [ + "stat_var", + "financingRecipient", + "financingRecipient_party_kind", + "time_period", + "value", + "unit", + "status" + ], + "rows": [ + [ + "dcid:ops_f72f69cb463d4fdbeb6751800ba2544a", + "country/KEN", + "party", + "2021", + 0.2, + { + "numerator": "USDollar", + "denominator": "Person" + }, + null + ], + [ + "dcid:ops_f72f69cb463d4fdbeb6751800ba2544a", + "country/KEN", + "party", + "2022", + 0.2, + { + "numerator": "USDollar", + "denominator": "Person" + }, + null + ], + [ + "dcid:ops_f72f69cb463d4fdbeb6751800ba2544a", + "country/NGA", + "party", + "2021", + 0.1, + { + "numerator": "USDollar", + "denominator": "Person" + }, + null + ], + [ + "dcid:ops_f72f69cb463d4fdbeb6751800ba2544a", + "country/NGA", + "party", + "2022", + 0.14285714285714285, + { + "numerator": "USDollar", + "denominator": "Person" + }, + null + ] + ] + }, + "variables": { + "ops_f72f69cb463d4fdbeb6751800ba2544a": "mv_df0088e6b8552d16ee6000c8b7da3814" + }, + "name_documents": { + "mv_df0088e6b8552d16ee6000c8b7da3814": "{\"denominator_frame_ref\":[\"dcid:Amount_ODA\"],\"frame_ref\":[\"dcid:Count_Person\"],\"verb\":\"quotient_of\"}" + }, + "document": "{\"columns\":[\"stat_var\",\"financingRecipient\",\"financingRecipient_party_kind\",\"time_period\",\"value\",\"unit\"],\"digest_version\":\"out2\",\"rows\":[[\"mv_df0088e6b8552d16ee6000c8b7da3814\",\"country/KEN\",\"party\",\"2021\",0.2,{\"denominator\":\"Person\",\"numerator\":\"USDollar\"}],[\"mv_df0088e6b8552d16ee6000c8b7da3814\",\"country/KEN\",\"party\",\"2022\",0.2,{\"denominator\":\"Person\",\"numerator\":\"USDollar\"}],[\"mv_df0088e6b8552d16ee6000c8b7da3814\",\"country/NGA\",\"party\",\"2021\",0.1,{\"denominator\":\"Person\",\"numerator\":\"USDollar\"}],[\"mv_df0088e6b8552d16ee6000c8b7da3814\",\"country/NGA\",\"party\",\"2022\",0.14285714285714285,{\"denominator\":\"Person\",\"numerator\":\"USDollar\"}]]}", + "digest": "out_7b765553f4377c65a8c7799c070be936" + }, + { + "name": "quotient-of-a-sum-of", + "frame": { + "columns": [ + "stat_var", + "financingRecipient", + "financingRecipient_party_kind", + "time_period", + "value", + "unit", + "status" + ], + "rows": [ + [ + "dcid:ops_a3a85219d81e742e831d19c7966e562f", + "country/KEN", + "party", + "2021", + 0.6, + { + "numerator": "USDollar", + "denominator": "Person" + }, + null + ], + [ + "dcid:ops_a3a85219d81e742e831d19c7966e562f", + "country/KEN", + "party", + "2022", + 0.6, + { + "numerator": "USDollar", + "denominator": "Person" + }, + null + ], + [ + "dcid:ops_a3a85219d81e742e831d19c7966e562f", + "country/NGA", + "party", + "2021", + 0.3, + { + "numerator": "USDollar", + "denominator": "Person" + }, + null + ], + [ + "dcid:ops_a3a85219d81e742e831d19c7966e562f", + "country/NGA", + "party", + "2022", + 0.42857142857142855, + { + "numerator": "USDollar", + "denominator": "Person" + }, + null + ] + ] + }, + "variables": { + "ops_a3a85219d81e742e831d19c7966e562f": "mv_288cc4adaed3cc3195d7e8c6cc4f87aa" + }, + "name_documents": { + "mv_ae90cefe711d4addb439f873f2010dba": "{\"frame_ref\":[\"dcid:Amount_ODA\"],\"right_frame_ref\":[\"dcid:Amount_ODA\"],\"verb\":\"sum_of\"}", + "mv_288cc4adaed3cc3195d7e8c6cc4f87aa": "{\"denominator_frame_ref\":[\"dcid:Count_Person\"],\"frame_ref\":[\"mv_ae90cefe711d4addb439f873f2010dba\"],\"verb\":\"quotient_of\"}" + }, + "document": "{\"columns\":[\"stat_var\",\"financingRecipient\",\"financingRecipient_party_kind\",\"time_period\",\"value\",\"unit\"],\"digest_version\":\"out2\",\"rows\":[[\"mv_288cc4adaed3cc3195d7e8c6cc4f87aa\",\"country/KEN\",\"party\",\"2021\",0.6,{\"denominator\":\"Person\",\"numerator\":\"USDollar\"}],[\"mv_288cc4adaed3cc3195d7e8c6cc4f87aa\",\"country/KEN\",\"party\",\"2022\",0.6,{\"denominator\":\"Person\",\"numerator\":\"USDollar\"}],[\"mv_288cc4adaed3cc3195d7e8c6cc4f87aa\",\"country/NGA\",\"party\",\"2021\",0.3,{\"denominator\":\"Person\",\"numerator\":\"USDollar\"}],[\"mv_288cc4adaed3cc3195d7e8c6cc4f87aa\",\"country/NGA\",\"party\",\"2022\",0.42857142857142855,{\"denominator\":\"Person\",\"numerator\":\"USDollar\"}]]}", + "digest": "out_61999d3b24a82e4c28621d0fe434959a" + }, + { + "name": "relative-growth", + "frame": { + "columns": [ + "stat_var", + "destination", + "time_period", + "value", + "unit", + "status" + ], + "rows": [ + [ + "dcid:Amount_FinancialFlow_ODA_Disbursement", + "country/KEN", + "2018/2022", + 0.25, + null, + null + ], + [ + "dcid:Amount_FinancialFlow_ODA_Disbursement", + "country/NGA", + "2018/2022", + -0.125, + null, + null + ] + ] + }, + "document": "{\"columns\":[\"stat_var\",\"destination\",\"time_period\",\"value\",\"unit\"],\"digest_version\":\"out2\",\"rows\":[[\"dcid:Amount_FinancialFlow_ODA_Disbursement\",\"country/KEN\",\"2018/2022\",0.25,null],[\"dcid:Amount_FinancialFlow_ODA_Disbursement\",\"country/NGA\",\"2018/2022\",-0.125,null]]}", + "digest": "out_f648b4fa7a61f8a10daf34eb6b1e8e01" + }, + { + "name": "number-spellings", + "frame": { + "columns": [ + "stat_var", + "destination", + "time_period", + "value", + "unit", + "status" + ], + "rows": [ + [ + "dcid:Amount_FinancialFlow_ODA_Disbursement", + "country/KEN", + "2020", + 1e+21, + "USDollar", + null + ], + [ + "dcid:Amount_FinancialFlow_ODA_Disbursement", + "country/KEN", + "2021", + 1.5e-07, + "USDollar", + null + ], + [ + "dcid:Amount_FinancialFlow_ODA_Disbursement", + "country/KEN", + "2022", + 3.0, + "USDollar", + null + ] + ] + }, + "document": "{\"columns\":[\"stat_var\",\"destination\",\"time_period\",\"value\",\"unit\"],\"digest_version\":\"out2\",\"rows\":[[\"dcid:Amount_FinancialFlow_ODA_Disbursement\",\"country/KEN\",\"2020\",1e+21,\"USDollar\"],[\"dcid:Amount_FinancialFlow_ODA_Disbursement\",\"country/KEN\",\"2021\",1.5e-7,\"USDollar\"],[\"dcid:Amount_FinancialFlow_ODA_Disbursement\",\"country/KEN\",\"2022\",3,\"USDollar\"]]}", + "digest": "out_c02c263d6ff798a41efd17e1872382d5" + }, + { + "name": "empty-frame", + "frame": { + "columns": [ + "stat_var", + "destination", + "origin", + "time_period", + "value", + "unit", + "status" + ], + "rows": [] + }, + "document": "{\"columns\":[\"stat_var\",\"destination\",\"origin\",\"time_period\",\"value\",\"unit\"],\"digest_version\":\"out2\",\"rows\":[]}", + "digest": "out_a8fab64c203e3de84f95af840aa6257c" + } + ] +} diff --git a/python/client-contract/src/dfo_client_contract/rules/publication.json b/python/client-contract/src/dfo_client_contract/rules/publication.json new file mode 100644 index 0000000..55b36eb --- /dev/null +++ b/python/client-contract/src/dfo_client_contract/rules/publication.json @@ -0,0 +1,444 @@ +{ + "format_version": 1, + "rule": "publication", + "vectors": [ + { + "name": "live-replace-keeps-slots", + "description": "Replacing a live share's cards keeps its slots, so each slot's embed and data URLs draw the new card.", + "steps": [ + { + "op": "share", + "share": "tile", + "latest": true, + "cards": [ + "cmp_a", + "cmp_b" + ] + }, + { + "op": "replace", + "share": "tile", + "cards": [ + "cmp_c", + "cmp_d" + ], + "write_token": "held" + } + ], + "outcomes": [ + { + "share": { + "latest": true, + "slots": [ + { + "slot": 1, + "card": "cmp_a" + }, + { + "slot": 2, + "card": "cmp_b" + } + ] + } + }, + { + "share": { + "latest": true, + "slots": [ + { + "slot": 1, + "card": "cmp_c" + }, + { + "slot": 2, + "card": "cmp_d" + } + ] + } + } + ] + }, + { + "name": "live-replace-with-fewer-cards", + "description": "A replacement with fewer cards leaves the later slots empty, and a later one with more fills them again, under the same URLs.", + "steps": [ + { + "op": "share", + "share": "tile", + "latest": true, + "cards": [ + "cmp_a", + "cmp_b", + "cmp_c" + ] + }, + { + "op": "replace", + "share": "tile", + "cards": [ + "cmp_d" + ], + "write_token": "held" + }, + { + "op": "replace", + "share": "tile", + "cards": [ + "cmp_a", + "cmp_b", + "cmp_e" + ], + "write_token": "held" + } + ], + "outcomes": [ + { + "share": { + "latest": true, + "slots": [ + { + "slot": 1, + "card": "cmp_a" + }, + { + "slot": 2, + "card": "cmp_b" + }, + { + "slot": 3, + "card": "cmp_c" + } + ] + } + }, + { + "share": { + "latest": true, + "slots": [ + { + "slot": 1, + "card": "cmp_d" + }, + { + "slot": 2, + "card": null + }, + { + "slot": 3, + "card": null + } + ] + } + }, + { + "share": { + "latest": true, + "slots": [ + { + "slot": 1, + "card": "cmp_a" + }, + { + "slot": 2, + "card": "cmp_b" + }, + { + "slot": 3, + "card": "cmp_e" + } + ] + } + } + ] + }, + { + "name": "snapshot-replace-refused", + "description": "A snapshot cannot be replaced, even with its write token, so a cited URL draws the same figure for good.", + "steps": [ + { + "op": "share", + "share": "figure", + "latest": false, + "cards": [ + "cmp_a" + ] + }, + { + "op": "replace", + "share": "figure", + "cards": [ + "cmp_b" + ], + "write_token": "held" + }, + { + "op": "read", + "share": "figure" + } + ], + "outcomes": [ + { + "share": { + "latest": false, + "slots": [ + { + "slot": 1, + "card": "cmp_a" + } + ] + } + }, + { + "error": "share_not_replaceable" + }, + { + "share": { + "latest": false, + "slots": [ + { + "slot": 1, + "card": "cmp_a" + } + ] + } + } + ] + }, + { + "name": "replace-without-token-refused", + "description": "Replacing a share needs its write token.", + "steps": [ + { + "op": "share", + "share": "tile", + "latest": true, + "cards": [ + "cmp_a" + ] + }, + { + "op": "replace", + "share": "tile", + "cards": [ + "cmp_b" + ] + }, + { + "op": "read", + "share": "tile" + } + ], + "outcomes": [ + { + "share": { + "latest": true, + "slots": [ + { + "slot": 1, + "card": "cmp_a" + } + ] + } + }, + { + "error": "write_token_required" + }, + { + "share": { + "latest": true, + "slots": [ + { + "slot": 1, + "card": "cmp_a" + } + ] + } + } + ] + }, + { + "name": "replace-with-another-token-refused", + "description": "A write token other than the share's is refused, and the check comes before the share's mode, so a caller without the token learns nothing else.", + "steps": [ + { + "op": "share", + "share": "figure", + "latest": false, + "cards": [ + "cmp_a" + ] + }, + { + "op": "replace", + "share": "figure", + "cards": [ + "cmp_b" + ], + "write_token": "other" + } + ], + "outcomes": [ + { + "share": { + "latest": false, + "slots": [ + { + "slot": 1, + "card": "cmp_a" + } + ] + } + }, + { + "error": "write_token_invalid" + } + ] + }, + { + "name": "tokenless-share-stays-fixed", + "description": "A share created with no write token, as a guest's share migrates, refuses every token, so it stays as it is.", + "steps": [ + { + "op": "share", + "share": "guest", + "latest": true, + "cards": [ + "cmp_a" + ], + "tokenless": true + }, + { + "op": "replace", + "share": "guest", + "cards": [ + "cmp_b" + ], + "write_token": "held" + }, + { + "op": "unshare", + "share": "guest", + "write_token": "held" + }, + { + "op": "read", + "share": "guest" + } + ], + "outcomes": [ + { + "share": { + "latest": true, + "slots": [ + { + "slot": 1, + "card": "cmp_a" + } + ] + } + }, + { + "error": "write_token_invalid" + }, + { + "error": "write_token_invalid" + }, + { + "share": { + "latest": true, + "slots": [ + { + "slot": 1, + "card": "cmp_a" + } + ] + } + } + ] + }, + { + "name": "unshare-snapshot", + "description": "A snapshot can be taken down with its write token.", + "steps": [ + { + "op": "share", + "share": "figure", + "latest": false, + "cards": [ + "cmp_a" + ] + }, + { + "op": "unshare", + "share": "figure", + "write_token": "held" + } + ], + "outcomes": [ + { + "share": { + "latest": false, + "slots": [ + { + "slot": 1, + "card": "cmp_a" + } + ] + } + }, + { + "removed": true + } + ] + }, + { + "name": "unshare-without-token-refused", + "description": "Taking a share down needs its write token.", + "steps": [ + { + "op": "share", + "share": "tile", + "latest": true, + "cards": [ + "cmp_a" + ] + }, + { + "op": "unshare", + "share": "tile" + }, + { + "op": "read", + "share": "tile" + } + ], + "outcomes": [ + { + "share": { + "latest": true, + "slots": [ + { + "slot": 1, + "card": "cmp_a" + } + ] + } + }, + { + "error": "write_token_required" + }, + { + "share": { + "latest": true, + "slots": [ + { + "slot": 1, + "card": "cmp_a" + } + ] + } + } + ] + } + ] +} diff --git a/python/client-contract/src/dfo_client_contract/rules/rolling-composition.json b/python/client-contract/src/dfo_client_contract/rules/rolling-composition.json new file mode 100644 index 0000000..b7f555a --- /dev/null +++ b/python/client-contract/src/dfo_client_contract/rules/rolling-composition.json @@ -0,0 +1,530 @@ +{ + "format_version": 1, + "rule": "rolling-composition", + "vectors": [ + { + "name": "one-rolling-step", + "description": "One rolling step takes the newest period at least half of every population's series hold: Kenya alone holds 2024, so 2023.", + "series": [ + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "coordinates": { + "destination": "country/KEN" + }, + "periods": [ + "2021", + "2022", + "2023", + "2024" + ] + }, + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "coordinates": { + "destination": "country/GHA" + }, + "periods": [ + "2021", + "2022", + "2023" + ] + }, + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "coordinates": { + "destination": "country/NGA" + }, + "periods": [ + "2021", + "2022", + "2023" + ] + }, + { + "spec_id": "spec_1a3c5e7f9b2d4f6a8c0e2b4d6f8a0c3e", + "coordinates": { + "observationAbout": "country/KEN" + }, + "periods": [ + "2021", + "2022", + "2023", + "2024" + ] + }, + { + "spec_id": "spec_1a3c5e7f9b2d4f6a8c0e2b4d6f8a0c3e", + "coordinates": { + "observationAbout": "country/GHA" + }, + "periods": [ + "2021", + "2022", + "2023", + "2024" + ] + } + ], + "steps": [ + { + "step": "growth", + "population": [ + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "bindings": {} + } + ] + } + ], + "period": "2023" + }, + { + "name": "two-steps-one-population", + "description": "Two rolling steps that state the same populations, however they list them, share one latest period.", + "series": [ + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "coordinates": { + "destination": "country/KEN" + }, + "periods": [ + "2021", + "2022", + "2023", + "2024" + ] + }, + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "coordinates": { + "destination": "country/GHA" + }, + "periods": [ + "2021", + "2022", + "2023" + ] + }, + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "coordinates": { + "destination": "country/NGA" + }, + "periods": [ + "2021", + "2022", + "2023" + ] + }, + { + "spec_id": "spec_1a3c5e7f9b2d4f6a8c0e2b4d6f8a0c3e", + "coordinates": { + "observationAbout": "country/KEN" + }, + "periods": [ + "2021", + "2022", + "2023", + "2024" + ] + }, + { + "spec_id": "spec_1a3c5e7f9b2d4f6a8c0e2b4d6f8a0c3e", + "coordinates": { + "observationAbout": "country/GHA" + }, + "periods": [ + "2021", + "2022", + "2023", + "2024" + ] + } + ], + "steps": [ + { + "step": "growth", + "population": [ + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "bindings": { + "destination": [ + "country/KEN", + "country/GHA" + ] + } + } + ] + }, + { + "step": "rank", + "population": [ + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "bindings": { + "destination": [ + "country/GHA", + "country/KEN" + ] + } + } + ] + } + ], + "period": "2024" + }, + { + "name": "one-population-per-root", + "description": "A tree over two roots counts the latest period over both: each population must hold it.", + "series": [ + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "coordinates": { + "destination": "country/KEN" + }, + "periods": [ + "2021", + "2022", + "2023", + "2024" + ] + }, + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "coordinates": { + "destination": "country/GHA" + }, + "periods": [ + "2021", + "2022", + "2023" + ] + }, + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "coordinates": { + "destination": "country/NGA" + }, + "periods": [ + "2021", + "2022", + "2023" + ] + }, + { + "spec_id": "spec_1a3c5e7f9b2d4f6a8c0e2b4d6f8a0c3e", + "coordinates": { + "observationAbout": "country/KEN" + }, + "periods": [ + "2021", + "2022", + "2023", + "2024" + ] + }, + { + "spec_id": "spec_1a3c5e7f9b2d4f6a8c0e2b4d6f8a0c3e", + "coordinates": { + "observationAbout": "country/GHA" + }, + "periods": [ + "2021", + "2022", + "2023", + "2024" + ] + } + ], + "steps": [ + { + "step": "growth", + "population": [ + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "bindings": {} + }, + { + "spec_id": "spec_1a3c5e7f9b2d4f6a8c0e2b4d6f8a0c3e", + "bindings": {} + } + ] + }, + { + "step": "growth", + "population": [ + { + "spec_id": "spec_1a3c5e7f9b2d4f6a8c0e2b4d6f8a0c3e", + "bindings": {} + }, + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "bindings": {} + } + ] + } + ], + "period": "2023" + }, + { + "name": "same-year-other-series-refused", + "description": "Two rolling steps over different series are refused even when each would settle on the same year, 2024 here, so one latest period is never counted twice.", + "series": [ + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "coordinates": { + "destination": "country/KEN" + }, + "periods": [ + "2021", + "2022", + "2023", + "2024" + ] + }, + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "coordinates": { + "destination": "country/GHA" + }, + "periods": [ + "2021", + "2022", + "2023" + ] + }, + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "coordinates": { + "destination": "country/NGA" + }, + "periods": [ + "2021", + "2022", + "2023" + ] + }, + { + "spec_id": "spec_1a3c5e7f9b2d4f6a8c0e2b4d6f8a0c3e", + "coordinates": { + "observationAbout": "country/KEN" + }, + "periods": [ + "2021", + "2022", + "2023", + "2024" + ] + }, + { + "spec_id": "spec_1a3c5e7f9b2d4f6a8c0e2b4d6f8a0c3e", + "coordinates": { + "observationAbout": "country/GHA" + }, + "periods": [ + "2021", + "2022", + "2023", + "2024" + ] + } + ], + "steps": [ + { + "step": "growth", + "population": [ + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "bindings": { + "destination": [ + "country/KEN", + "country/GHA" + ] + } + } + ] + }, + { + "step": "rank", + "population": [ + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "bindings": { + "destination": [ + "country/KEN", + "country/NGA" + ] + } + } + ] + } + ], + "error": "rolling_populations_differ" + }, + { + "name": "other-year-refused", + "description": "Two rolling steps over different populations that would settle on different years are refused.", + "series": [ + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "coordinates": { + "destination": "country/KEN" + }, + "periods": [ + "2021", + "2022", + "2023", + "2024" + ] + }, + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "coordinates": { + "destination": "country/GHA" + }, + "periods": [ + "2021", + "2022", + "2023" + ] + }, + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "coordinates": { + "destination": "country/NGA" + }, + "periods": [ + "2021", + "2022", + "2023" + ] + }, + { + "spec_id": "spec_1a3c5e7f9b2d4f6a8c0e2b4d6f8a0c3e", + "coordinates": { + "observationAbout": "country/KEN" + }, + "periods": [ + "2021", + "2022", + "2023", + "2024" + ] + }, + { + "spec_id": "spec_1a3c5e7f9b2d4f6a8c0e2b4d6f8a0c3e", + "coordinates": { + "observationAbout": "country/GHA" + }, + "periods": [ + "2021", + "2022", + "2023", + "2024" + ] + } + ], + "steps": [ + { + "step": "growth", + "population": [ + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "bindings": {} + } + ] + }, + { + "step": "growth", + "population": [ + { + "spec_id": "spec_1a3c5e7f9b2d4f6a8c0e2b4d6f8a0c3e", + "bindings": {} + } + ] + } + ], + "error": "rolling_populations_differ" + }, + { + "name": "empty-population-unresolved", + "description": "A population no series falls in names no latest period.", + "series": [ + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "coordinates": { + "destination": "country/KEN" + }, + "periods": [ + "2021", + "2022", + "2023", + "2024" + ] + }, + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "coordinates": { + "destination": "country/GHA" + }, + "periods": [ + "2021", + "2022", + "2023" + ] + }, + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "coordinates": { + "destination": "country/NGA" + }, + "periods": [ + "2021", + "2022", + "2023" + ] + }, + { + "spec_id": "spec_1a3c5e7f9b2d4f6a8c0e2b4d6f8a0c3e", + "coordinates": { + "observationAbout": "country/KEN" + }, + "periods": [ + "2021", + "2022", + "2023", + "2024" + ] + }, + { + "spec_id": "spec_1a3c5e7f9b2d4f6a8c0e2b4d6f8a0c3e", + "coordinates": { + "observationAbout": "country/GHA" + }, + "periods": [ + "2021", + "2022", + "2023", + "2024" + ] + } + ], + "steps": [ + { + "step": "growth", + "population": [ + { + "spec_id": "spec_6f0c2a9d4b7e1f3a5c8d0b2e4f6a8c1d", + "bindings": { + "destination": [ + "country/SOM" + ] + } + } + ] + } + ], + "error": "rolling_anchor_unresolved" + } + ] +} diff --git a/python/client-contract/src/dfo_client_contract/rules/rules.schema.json b/python/client-contract/src/dfo_client_contract/rules/rules.schema.json index 8ccf5b4..e8dd62a 100644 --- a/python/client-contract/src/dfo_client_contract/rules/rules.schema.json +++ b/python/client-contract/src/dfo_client_contract/rules/rules.schema.json @@ -2,7 +2,7 @@ "$schema": "https://json-schema.org/draft/2020-12/schema", "$id": "https://raw.githubusercontent.com/DevelopmentFinanceObservatory/spec-kit/main/python/client-contract/src/dfo_client_contract/rules/rules.schema.json", "title": "Client rule vectors", - "description": "One file per rule, each holding the vectors a client's implementation of that rule must reproduce. rules/README.md states each rule.", + "description": "One file per rule, each holding the vectors an implementation of that rule must reproduce: a client's for the client rules, DFO's for the analysis and publication rules. rules/README.md states each rule.", "type": "object", "oneOf": [ { @@ -35,6 +35,24 @@ { "$ref": "#/$defs/role_keys" }, + { + "$ref": "#/$defs/output_digest" + }, + { + "$ref": "#/$defs/output_comparison" + }, + { + "$ref": "#/$defs/operation_outputs" + }, + { + "$ref": "#/$defs/rolling_composition" + }, + { + "$ref": "#/$defs/publication" + }, + { + "$ref": "#/$defs/series_left_out" + }, { "$ref": "#/$defs/file_selector" } @@ -1136,6 +1154,894 @@ } } }, + "output_digest": { + "$ref": "#/$defs/suite", + "additionalProperties": false, + "properties": { + "format_version": true, + "rule": { + "const": "output-digest" + }, + "vectors": { + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "name", + "frame", + "document", + "digest" + ], + "dependentRequired": { + "variables": [ + "name_documents" + ] + }, + "properties": { + "name": { + "$ref": "#/$defs/name" + }, + "frame": { + "type": "object", + "additionalProperties": false, + "required": [ + "columns", + "rows" + ], + "properties": { + "columns": { + "description": "The frame's columns in the frame grammar's order.", + "type": "array", + "minItems": 1, + "uniqueItems": true, + "items": { + "type": "string", + "minLength": 1, + "$ref": "#/$defs/one_line" + } + }, + "rows": { + "type": "array", + "items": { + "type": "array", + "items": { + "type": [ + "string", + "number", + "null", + "object" + ] + } + } + } + }, + "description": "A frame as docs/frame-grammar-v1.md defines it, each row in its column order." + }, + "variables": { + "description": "The name of each variable an operation of the frame's recipe wrote, keyed by the operation's ops_ id, which the frame's stat_var cells spell after dcid:.", + "type": "object", + "propertyNames": { + "pattern": "^ops_[0-9a-f]{32}$" + }, + "additionalProperties": { + "type": "string", + "pattern": "^mv_[0-9a-f]{32}$" + } + }, + "name_documents": { + "description": "The name document of each name in variables and of each mv_ name those documents hold, as the UTF-8 text the name hashes, keyed by the name.", + "type": "object", + "minProperties": 1, + "propertyNames": { + "pattern": "^mv_[0-9a-f]{32}$" + }, + "additionalProperties": { + "type": "string", + "minLength": 1 + } + }, + "document": { + "description": "The output digest document, as the UTF-8 text the digest hashes.", + "type": "string", + "minLength": 1 + }, + "digest": { + "type": "string", + "pattern": "^out_[0-9a-f]{32}$" + } + } + } + } + } + }, + "output_comparison": { + "$ref": "#/$defs/suite", + "additionalProperties": false, + "properties": { + "format_version": true, + "rule": { + "const": "output-comparison" + }, + "vectors": { + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "name", + "left", + "right", + "equal", + "same_data" + ], + "properties": { + "name": { + "$ref": "#/$defs/name" + }, + "left": { + "$ref": "../dataset-ref.schema.json" + }, + "right": { + "$ref": "../dataset-ref.schema.json" + }, + "equal": { + "description": "Whether the refs name the same versions, recipes and renderings: `==` in Python, `equals` in TypeScript.", + "type": "boolean" + }, + "same_data": { + "description": "Whether the refs hold the same output: `same_data` in Python, `sameData` in TypeScript.", + "type": "boolean" + } + } + } + } + } + }, + "operation_outputs": { + "$ref": "#/$defs/suite", + "additionalProperties": false, + "properties": { + "format_version": true, + "rule": { + "const": "operation-outputs" + }, + "vectors": { + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "name", + "description", + "verb", + "arguments", + "data" + ], + "properties": { + "name": { + "$ref": "#/$defs/name" + }, + "description": { + "type": "string", + "minLength": 1 + }, + "verb": { + "enum": [ + "share_of", + "join", + "compare", + "sum_of", + "growth" + ] + }, + "arguments": { + "type": "object", + "additionalProperties": false, + "required": [], + "properties": { + "on": { + "description": "Each role of data, to the role of other it pairs with.", + "type": "object", + "minProperties": 1, + "additionalProperties": { + "type": "string", + "minLength": 1 + } + }, + "by": { + "type": "array", + "minItems": 1, + "uniqueItems": true, + "items": { + "type": "string", + "minLength": 1 + } + }, + "window": { + "type": "array", + "minItems": 2, + "maxItems": 2, + "items": { + "type": "string", + "pattern": "^[0-9]{4}$" + } + }, + "change": { + "enum": [ + "relative", + "absolute" + ] + } + } + }, + "data": { + "type": "array", + "items": { + "type": "object", + "required": [ + "time_period", + "value", + "unit" + ], + "properties": { + "stat_var": { + "type": "string", + "pattern": "^dcid:." + }, + "time_period": { + "type": "string", + "minLength": 1 + }, + "value": { + "type": "number" + }, + "unit": { + "type": [ + "string", + "number", + "null", + "object" + ] + } + }, + "additionalProperties": { + "type": "string", + "minLength": 1 + } + }, + "description": "The rows of the first reading, each its frame row by column name." + }, + "other": { + "type": "array", + "items": { + "type": "object", + "required": [ + "time_period", + "value", + "unit" + ], + "properties": { + "stat_var": { + "type": "string", + "pattern": "^dcid:." + }, + "time_period": { + "type": "string", + "minLength": 1 + }, + "value": { + "type": "number" + }, + "unit": { + "type": [ + "string", + "number", + "null", + "object" + ] + } + }, + "additionalProperties": { + "type": "string", + "minLength": 1 + } + }, + "description": "The rows of the second reading, for an operation that takes one." + }, + "rows": { + "type": "array", + "items": { + "type": "object", + "required": [ + "time_period", + "value", + "unit" + ], + "properties": { + "stat_var": { + "type": "string", + "pattern": "^dcid:." + }, + "time_period": { + "type": "string", + "minLength": 1 + }, + "value": { + "type": "number" + }, + "unit": { + "type": [ + "string", + "number", + "null", + "object" + ] + } + }, + "additionalProperties": { + "type": "string", + "minLength": 1 + } + }, + "description": "The output rows, in any order. A row leaves out stat_var when the operation writes a variable of its own." + }, + "error": { + "description": "The code the operation refuses with in place of rows.", + "enum": [ + "role_ambiguous" + ] + } + }, + "oneOf": [ + { + "required": [ + "rows" + ] + }, + { + "required": [ + "error" + ] + } + ] + } + } + } + }, + "rolling_composition": { + "$ref": "#/$defs/suite", + "additionalProperties": false, + "properties": { + "format_version": true, + "rule": { + "const": "rolling-composition" + }, + "vectors": { + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "name", + "description", + "series", + "steps" + ], + "properties": { + "name": { + "$ref": "#/$defs/name" + }, + "description": { + "type": "string", + "minLength": 1 + }, + "series": { + "description": "Every series of the roots, by its root's spec_id and coordinates, with the periods it holds a row at.", + "type": "array", + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "spec_id", + "coordinates", + "periods" + ], + "properties": { + "spec_id": { + "type": "string", + "pattern": "^spec_[0-9a-f]{32}$" + }, + "coordinates": { + "type": "object", + "additionalProperties": { + "type": "string", + "minLength": 1 + } + }, + "periods": { + "type": "array", + "uniqueItems": true, + "items": { + "type": "string", + "pattern": "^[0-9]{4}$" + } + } + } + } + }, + "steps": { + "description": "The rolling steps of one recipe tree, each with the populations it counts the latest period over.", + "type": "array", + "minItems": 1, + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "step", + "population" + ], + "properties": { + "step": { + "type": "string", + "minLength": 1 + }, + "population": { + "type": "array", + "minItems": 1, + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "spec_id" + ], + "properties": { + "spec_id": { + "type": "string", + "pattern": "^spec_[0-9a-f]{32}$" + }, + "bindings": { + "type": "object", + "additionalProperties": { + "type": "array", + "minItems": 1, + "items": { + "type": "string", + "minLength": 1 + } + } + } + } + } + } + } + } + }, + "period": { + "type": "string", + "pattern": "^[0-9]{4}$" + }, + "error": { + "enum": [ + "rolling_populations_differ", + "rolling_anchor_unresolved" + ] + } + }, + "oneOf": [ + { + "required": [ + "period" + ] + }, + { + "required": [ + "error" + ] + } + ] + } + } + } + }, + "publication": { + "$ref": "#/$defs/suite", + "additionalProperties": false, + "properties": { + "format_version": true, + "rule": { + "const": "publication" + }, + "vectors": { + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "name", + "description", + "steps", + "outcomes" + ], + "properties": { + "name": { + "$ref": "#/$defs/name" + }, + "description": { + "type": "string", + "minLength": 1 + }, + "steps": { + "type": "array", + "minItems": 1, + "items": { + "oneOf": [ + { + "type": "object", + "additionalProperties": false, + "required": [ + "op", + "share", + "latest", + "cards" + ], + "properties": { + "op": { + "const": "share" + }, + "share": { + "type": "string", + "minLength": 1 + }, + "latest": { + "type": "boolean" + }, + "cards": { + "type": "array", + "minItems": 1, + "items": { + "type": "string", + "pattern": "^cmp_[0-9a-z]+$" + } + }, + "tokenless": { + "description": "The share holds no write token, as a guest's share migrates.", + "const": true + } + } + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "op", + "share", + "cards" + ], + "properties": { + "op": { + "const": "replace" + }, + "share": { + "type": "string", + "minLength": 1 + }, + "cards": { + "type": "array", + "minItems": 1, + "items": { + "type": "string", + "pattern": "^cmp_[0-9a-z]+$" + } + }, + "write_token": { + "description": "`held` for the token the share's creation returned, `other` for another; absent sends none.", + "enum": [ + "held", + "other" + ] + } + } + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "op", + "share" + ], + "properties": { + "op": { + "const": "unshare" + }, + "share": { + "type": "string", + "minLength": 1 + }, + "write_token": { + "description": "`held` for the token the share's creation returned, `other` for another; absent sends none.", + "enum": [ + "held", + "other" + ] + } + } + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "op", + "share" + ], + "properties": { + "op": { + "const": "read" + }, + "share": { + "type": "string", + "minLength": 1 + } + } + } + ] + } + }, + "outcomes": { + "description": "One outcome per step.", + "type": "array", + "minItems": 1, + "items": { + "oneOf": [ + { + "type": "object", + "additionalProperties": false, + "required": [ + "share" + ], + "properties": { + "share": { + "type": "object", + "additionalProperties": false, + "required": [ + "latest", + "slots" + ], + "properties": { + "latest": { + "type": "boolean" + }, + "slots": { + "type": "array", + "minItems": 1, + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "slot", + "card" + ], + "properties": { + "slot": { + "type": "integer", + "minimum": 1 + }, + "card": { + "oneOf": [ + { + "type": "string", + "pattern": "^cmp_[0-9a-z]+$" + }, + { + "type": "null" + } + ] + } + } + } + } + } + } + } + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "removed" + ], + "properties": { + "removed": { + "const": true + } + } + }, + { + "type": "object", + "additionalProperties": false, + "required": [ + "error" + ], + "properties": { + "error": { + "enum": [ + "write_token_required", + "write_token_invalid", + "share_not_replaceable" + ] + } + } + } + ] + } + } + } + } + } + } + }, + "series_left_out": { + "$ref": "#/$defs/suite", + "additionalProperties": false, + "properties": { + "format_version": true, + "rule": { + "const": "series-left-out" + }, + "vectors": { + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "name", + "description", + "members", + "strict" + ], + "oneOf": [ + { + "required": [ + "warnings" + ] + }, + { + "required": [ + "raises" + ] + } + ], + "properties": { + "name": { + "$ref": "#/$defs/name" + }, + "description": { + "type": "string", + "minLength": 1, + "$ref": "#/$defs/one_line" + }, + "members": { + "description": "The left-out series of each derived member one fetch returns, in member order, as its derived manifest serves them.", + "type": "array", + "minItems": 1, + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "left_out", + "left_out_complete" + ], + "oneOf": [ + { + "properties": { + "left_out": { + "type": "null" + }, + "left_out_complete": { + "type": "null" + } + } + }, + { + "properties": { + "left_out": { + "type": "array" + }, + "left_out_complete": { + "type": "boolean" + } + } + } + ], + "properties": { + "left_out": { + "description": "Every series a step of the recipe dropped for a data gap; null for a recipe stored without them.", + "type": [ + "array", + "null" + ], + "items": { + "type": "object", + "additionalProperties": false, + "required": [ + "coordinates", + "step", + "side", + "reason", + "rows" + ], + "properties": { + "coordinates": { + "type": "object", + "minProperties": 1, + "additionalProperties": { + "type": "string", + "minLength": 1 + } + }, + "step": { + "type": "object", + "additionalProperties": false, + "required": [ + "verb", + "node" + ], + "properties": { + "verb": { + "type": "string", + "minLength": 1 + }, + "node": { + "type": "integer", + "minimum": 0 + } + } + }, + "side": { + "enum": [ + "data", + "other" + ] + }, + "reason": { + "type": "string", + "minLength": 1 + }, + "rows": { + "type": "integer", + "minimum": 1 + } + } + } + }, + "left_out_complete": { + "description": "False when the steps dropped more series than `left_out` lists; null with a null `left_out`.", + "type": [ + "boolean", + "null" + ] + } + } + } + }, + "strict": { + "type": "boolean" + }, + "warnings": { + "description": "The warnings the fetch issues, each once.", + "type": "array", + "maxItems": 1, + "items": { + "enum": [ + "series_left_out" + ] + } + }, + "raises": { + "description": "The warning a strict fetch raises in place of its data.", + "enum": [ + "series_left_out" + ] + } + } + } + } + } + }, "file_selector": { "$ref": "#/$defs/suite", "additionalProperties": false, diff --git a/python/client-contract/src/dfo_client_contract/rules/series-left-out.json b/python/client-contract/src/dfo_client_contract/rules/series-left-out.json new file mode 100644 index 0000000..a0afe22 --- /dev/null +++ b/python/client-contract/src/dfo_client_contract/rules/series-left-out.json @@ -0,0 +1,199 @@ +{ + "format_version": 1, + "rule": "series-left-out", + "vectors": [ + { + "name": "growth-left-series-out", + "description": "A growth dropped Nigeria, which has no value at the window's end, so the fetch issues series_left_out and returns the frame.", + "members": [ + { + "left_out": [ + { + "coordinates": { + "financingRecipient": "country/NGA" + }, + "step": { + "verb": "growth", + "node": 0 + }, + "side": "data", + "reason": "endpoint_missing", + "rows": 1 + } + ], + "left_out_complete": true + } + ], + "strict": false, + "warnings": [ + "series_left_out" + ] + }, + { + "name": "nothing-left-out", + "description": "A manifest that lists no left-out series and says the list is complete issues nothing.", + "members": [ + { + "left_out": [], + "left_out_complete": true + } + ], + "strict": false, + "warnings": [] + }, + { + "name": "recipe-stored-without-left-out", + "description": "A recipe stored without its left-out series has a null left_out and left_out_complete and issues nothing.", + "members": [ + { + "left_out": null, + "left_out_complete": null + } + ], + "strict": false, + "warnings": [] + }, + { + "name": "list-incomplete", + "description": "A manifest whose list stops short of every left-out series, left_out_complete false, issues series_left_out.", + "members": [ + { + "left_out": [ + { + "coordinates": { + "financingRecipient": "country/GHA" + }, + "step": { + "verb": "quotient_of", + "node": 1 + }, + "side": "data", + "reason": "unmatched_key", + "rows": 2 + } + ], + "left_out_complete": false + } + ], + "strict": false, + "warnings": [ + "series_left_out" + ] + }, + { + "name": "incomplete-with-none-listed", + "description": "left_out_complete false issues series_left_out even when left_out lists no series.", + "members": [ + { + "left_out": [], + "left_out_complete": false + } + ], + "strict": false, + "warnings": [ + "series_left_out" + ] + }, + { + "name": "several-members-warn-once", + "description": "Two members whose manifests each list left-out series issue one series_left_out for the fetch.", + "members": [ + { + "left_out": [ + { + "coordinates": { + "financingRecipient": "country/NGA" + }, + "step": { + "verb": "growth", + "node": 0 + }, + "side": "data", + "reason": "endpoint_missing", + "rows": 1 + } + ], + "left_out_complete": true + }, + { + "left_out": null, + "left_out_complete": null + }, + { + "left_out": [ + { + "coordinates": { + "financingRecipient": "country/GHA" + }, + "step": { + "verb": "quotient_of", + "node": 1 + }, + "side": "data", + "reason": "unmatched_key", + "rows": 2 + }, + { + "coordinates": { + "financingRecipient": "country/NGA" + }, + "step": { + "verb": "quotient_of", + "node": 1 + }, + "side": "data", + "reason": "period_unmatched", + "rows": 1 + } + ], + "left_out_complete": true + } + ], + "strict": false, + "warnings": [ + "series_left_out" + ] + }, + { + "name": "strict-raises", + "description": "Under strict, a fetch whose member's manifest lists left-out series raises series_left_out in place of its data.", + "members": [ + { + "left_out": null, + "left_out_complete": null + }, + { + "left_out": [ + { + "coordinates": { + "financingRecipient": "country/GHA" + }, + "step": { + "verb": "growth", + "node": 0 + }, + "side": "data", + "reason": "zero_or_sign_crossing_base", + "rows": 2 + } + ], + "left_out_complete": true + } + ], + "strict": true, + "raises": "series_left_out" + }, + { + "name": "strict-without-left-out-series", + "description": "Under strict, a fetch whose manifests list no left-out series returns its data with no warning.", + "members": [ + { + "left_out": [], + "left_out_complete": true + } + ], + "strict": true, + "warnings": [] + } + ] +} diff --git a/python/client-contract/src/dfo_client_contract/scenario.schema.json b/python/client-contract/src/dfo_client_contract/scenario.schema.json index c6e5752..d730cd8 100644 --- a/python/client-contract/src/dfo_client_contract/scenario.schema.json +++ b/python/client-contract/src/dfo_client_contract/scenario.schema.json @@ -222,35 +222,44 @@ ] }, "step": { - "oneOf": [ - { - "$ref": "#/$defs/search_step" - }, - { - "$ref": "#/$defs/choose_step" - }, - { - "$ref": "#/$defs/fetch_answer_step" - }, - { - "$ref": "#/$defs/fetch_spec_step" - }, - { - "$ref": "#/$defs/fetch_ref_step" - }, - { - "$ref": "#/$defs/newest_version_step" - }, - { - "$ref": "#/$defs/mint_step" - }, - { - "$ref": "#/$defs/operation_step" - }, - { - "$ref": "#/$defs/result_step" - } - ] + "description": "A step with a `ref` is a fetch of that ref, so a broken ref is reported at the member that breaks it; any other step is one of the other kinds.", + "if": { + "type": "object", + "required": [ + "ref" + ] + }, + "then": { + "$ref": "#/$defs/fetch_ref_step" + }, + "else": { + "oneOf": [ + { + "$ref": "#/$defs/search_step" + }, + { + "$ref": "#/$defs/choose_step" + }, + { + "$ref": "#/$defs/fetch_answer_step" + }, + { + "$ref": "#/$defs/fetch_spec_step" + }, + { + "$ref": "#/$defs/newest_version_step" + }, + { + "$ref": "#/$defs/mint_step" + }, + { + "$ref": "#/$defs/operation_step" + }, + { + "$ref": "#/$defs/result_step" + } + ] + } }, "search_step": { "type": "object", @@ -376,7 +385,19 @@ "const": "fetch" }, "ref": { - "$ref": "dataset-ref.schema.json" + "allOf": [ + { + "$ref": "dataset-ref.schema.json" + }, + { + "properties": { + "format_version": { + "description": "Format 1, a ref of dataset members. No scenario serves a derived member yet, so a format 2 ref fails here.", + "const": 1 + } + } + } + ] } } }, diff --git a/python/client-contract/src/dfo_client_contract/scenarios/search-retry-after-past-cap.json b/python/client-contract/src/dfo_client_contract/scenarios/search-retry-after-past-cap.json new file mode 100644 index 0000000..c772459 --- /dev/null +++ b/python/client-contract/src/dfo_client_contract/scenarios/search-retry-after-past-cap.json @@ -0,0 +1,57 @@ +{ + "format_version": 1, + "name": "search-retry-after-past-cap", + "description": "A retryable refusal whose Retry-After is past the 60-second cap raises at once, with no wait and no retry: ServerError with its code, service_unavailable, and its trace id.", + "origin": { + "kind": "synthetic", + "derived_from": "search-place-ambiguous", + "edit": "A 503 service_unavailable with retryable true and Retry-After 61 answers the search." + }, + "call": [ + { + "op": "search", + "question": "ODA disbursements from Germany to Congo in 2018" + } + ], + "exchanges": [ + { + "request": { + "method": "POST", + "path": "/search/resolve", + "json": { + "schema_version": 1, + "input": { + "kind": "raw_text", + "query": "ODA disbursements from Germany to Congo in 2018" + } + } + }, + "response": { + "status": 503, + "headers": { + "content-type": "application/problem+json", + "retry-after": "61" + }, + "json": { + "type": "/codes/service_unavailable", + "title": "Service Unavailable", + "status": 503, + "detail": "The service is busy.", + "code": "service_unavailable", + "trace_id": "00000000000000000000000000000000", + "capability_version": "synthetic", + "retryable": true + } + } + } + ], + "expect": [ + { + "error": { + "class": "ServerError", + "code": "service_unavailable", + "trace_id": "00000000000000000000000000000000" + } + } + ] +} diff --git a/python/client-contract/src/dfo_client_contract/warning-codes.json b/python/client-contract/src/dfo_client_contract/warning-codes.json index 5d55edf..5b9ee2d 100644 --- a/python/client-contract/src/dfo_client_contract/warning-codes.json +++ b/python/client-contract/src/dfo_client_contract/warning-codes.json @@ -26,6 +26,11 @@ "class": "DfoDeprecationWarning", "condition": "An API response carries a Deprecation or Sunset header, once per process per distinct pair of their values." }, + { + "code": "series_left_out", + "class": "SeriesLeftOutWarning", + "condition": "A derived frame left series out for missing data; the client returns the frame." + }, { "code": "server_warning", "class": "DfoWarning", diff --git a/python/client-contract/src/dfo_client_contract/warning-codes.schema.json b/python/client-contract/src/dfo_client_contract/warning-codes.schema.json index 4427722..d639eea 100644 --- a/python/client-contract/src/dfo_client_contract/warning-codes.schema.json +++ b/python/client-contract/src/dfo_client_contract/warning-codes.schema.json @@ -32,7 +32,8 @@ "class": { "enum": [ "DfoWarning", - "DfoDeprecationWarning" + "DfoDeprecationWarning", + "SeriesLeftOutWarning" ] }, "condition": { diff --git a/python/client-contract/tests/test_analysis_rules.py b/python/client-contract/tests/test_analysis_rules.py new file mode 100644 index 0000000..2faae0c --- /dev/null +++ b/python/client-contract/tests/test_analysis_rules.py @@ -0,0 +1,361 @@ +"""Replay the analysis and publication rule vectors through a reference +implementation of each rule. + +The rules are stated in `dfo_client_contract/rules/README.md`. DFO applies +them on the server; a client reads their results and applies only the +output comparison. Each function here is the shortest faithful reading of one +rule, so a vector that disagrees with its rule fails here before the server +or a client replays it. +""" + +from __future__ import annotations + +import hashlib +import math +from typing import Any + +import pytest +import rfc8785 + +from dfo_client_contract import rule + +# The frame columns that name no role (docs/frame-grammar-v1.md). +_NOT_ROLES = frozenset( + { + "stat_var", + "time_period", + "value", + "unit", + "measurementMethod", + "provenance", + "status", + } +) +_PERCENT = "Percent" + + +class RuleError(Exception): + def __init__(self, code: str) -> None: + super().__init__(code) + self.code = code + + +def _vectors(name: str) -> list[dict[str, Any]]: + return rule(name)["vectors"] + + +def _ids(name: str) -> list[str]: + return [vector["name"] for vector in _vectors(name)] + + +def jcs(value: Any) -> str: + """The RFC 8785 serialisation of a JSON value.""" + return rfc8785.dumps(value).decode("utf-8") + + +def _without_recipe(unit: Any) -> Any: + if not isinstance(unit, dict): + return unit + if "numerator" in unit: + return {side: _without_recipe(part) for side, part in unit.items()} + return {name: item for name, item in unit.items() if name != "recipe"} + + +def output_digest(frame: dict[str, Any], variables: dict[str, str]) -> tuple[str, str]: + """The output digest document of a frame, and its token, with `variables` + naming each variable an operation wrote by its `ops_` id.""" + roles = [column for column in frame["columns"] if column not in _NOT_ROLES] + columns = ["stat_var", *roles, "time_period", "value", "unit"] + rows = [] + for row in frame["rows"]: + cells = dict(zip(frame["columns"], row, strict=True)) + cells["unit"] = _without_recipe(cells["unit"]) + stat_var = cells["stat_var"] + cells["stat_var"] = variables.get(stat_var.removeprefix("dcid:"), stat_var) + rows.append([cells[column] for column in columns]) + rows.sort(key=lambda cells: jcs(cells).encode("utf-8")) + document = jcs({"digest_version": "out2", "columns": columns, "rows": rows}) + digest = hashlib.sha256(document.encode("utf-8")).hexdigest() + return document, f"out_{digest[:32]}" + + +def _identity(member: dict[str, Any]) -> tuple[str, ...]: + spec = member["spec"] + return ( + spec["spec_id"], + member["content_version"], + jcs(member["artifact"]["rendering"]), + jcs(spec.get("recipe")), + ) + + +def _output(member: dict[str, Any]) -> tuple[Any, ...]: + if "output_digest" in member: + return ("derived", member["output_digest"]) + rendering = member["artifact"]["rendering"] + cut = rendering.get("without_party_totals", False) + return ("dataset", member["content_version"], cut) + + +def refs_equal(left: dict[str, Any], right: dict[str, Any]) -> bool: + return [_identity(m) for m in left["members"]] == [ + _identity(m) for m in right["members"] + ] + + +def same_data(left: dict[str, Any], right: dict[str, Any]) -> bool: + return [_output(m) for m in left["members"]] == [ + _output(m) for m in right["members"] + ] + + +def _paired( + data: list[dict[str, Any]], other: list[dict[str, Any]], on: dict[str, str] +) -> list[tuple[dict[str, Any], dict[str, Any]]]: + """Each data row with the other row at its paired keys and period.""" + pairs = [] + for row in data: + found = [ + match + for match in other + if match["time_period"] == row["time_period"] + and all(row[mine] == match[theirs] for mine, theirs in on.items()) + ] + assert len(found) <= 1, row + pairs.extend((row, match) for match in found) + return pairs + + +def _growth(vector: dict[str, Any]) -> list[dict[str, Any]]: + """Each member's change over the window. Two rows of one member in one + period of the window refuse with `role_ambiguous`. A member is left out + without a value at both ends (`endpoint_missing`), for a relative change + with a zero or sign-crossing base (`zero_or_sign_crossing_base`), and + for a change that is not finite (`non_finite`).""" + arguments = vector["arguments"] + start, end = arguments["window"] + by = arguments["by"] + relative = arguments.get("change", "relative") == "relative" + groups: dict[tuple[str, ...], dict[str, dict[str, Any]]] = {} + for row in vector["data"]: + period = row["time_period"] + if not start <= period <= end: + continue + periods = groups.setdefault(tuple(row[r] for r in by), {}) + if period in periods: + raise RuleError("role_ambiguous") + periods[period] = row + rows = [] + for key, periods in groups.items(): + if start not in periods or end not in periods: + continue + first, last = periods[start], periods[end] + base, final = first["value"], last["value"] + if relative and (base == 0 or final == 0 or (base < 0) != (final < 0)): + continue + change = (final - base) / base if relative else final - base + if not math.isfinite(change): + continue + rows.append( + { + "stat_var": first["stat_var"], + **dict(zip(by, key, strict=True)), + "time_period": f"{start}/{end}", + "value": change, + "unit": None if relative else first["unit"], + } + ) + return rows + + +def _share_across(vector: dict[str, Any]) -> list[dict[str, Any]]: + by = vector["arguments"]["by"] + totals: dict[str, list[float]] = {} + sums: dict[tuple[Any, ...], list[float]] = {} + for row in vector["data"]: + totals.setdefault(row["time_period"], []).append(row["value"]) + key = (*(row[r] for r in by), row["time_period"]) + sums.setdefault(key, []).append(row["value"]) + return [ + { + **dict(zip(by, key[:-1], strict=True)), + "time_period": key[-1], + "value": math.fsum(values) / math.fsum(totals[key[-1]]) * 100.0, + "unit": _PERCENT, + } + for key, values in sums.items() + ] + + +def operation_output(vector: dict[str, Any]) -> list[dict[str, Any]]: + verb, arguments = vector["verb"], vector["arguments"] + if verb == "growth": + return _growth(vector) + if verb == "share_of" and "other" not in vector: + return _share_across(vector) + on = arguments["on"] + rows = [] + for row, match in _paired(vector["data"], vector["other"], on): + keys = {mine: row[mine] for mine in on} + period = row["time_period"] + if verb == "join": + rows.extend( + { + "stat_var": side["stat_var"], + **keys, + "time_period": period, + "value": side["value"], + "unit": side["unit"], + } + for side in (row, match) + ) + continue + value, unit = { + "share_of": (row["value"] / match["value"] * 100.0, _PERCENT), + "compare": (row["value"] - match["value"], row["unit"]), + "sum_of": (row["value"] + match["value"], row["unit"]), + }[verb] + rows.append({**keys, "time_period": period, "value": value, "unit": unit}) + return rows + + +def _population(population: dict[str, Any]) -> str: + """A population in its one spelling: each binding's values sorted, once.""" + bindings = { + column: sorted(set(values)) + for column, values in population.get("bindings", {}).items() + } + return jcs({"spec_id": population["spec_id"], "bindings": bindings}) + + +def rolling_anchor(vector: dict[str, Any]) -> str: + stated = { + jcs(sorted({_population(p) for p in step["population"]})) + for step in vector["steps"] + } + if len(stated) > 1: + raise RuleError("rolling_populations_differ") + held = [] + for population in vector["steps"][0]["population"]: + bindings = population.get("bindings", {}) + series = [ + set(item["periods"]) + for item in vector["series"] + if item["spec_id"] == population["spec_id"] + and all(item["coordinates"].get(c) in v for c, v in bindings.items()) + ] + if not series: + raise RuleError("rolling_anchor_unresolved") + held.append(series) + periods = {period for series in held for periods in series for period in periods} + for period in sorted(periods, reverse=True): + if all(2 * sum(period in s for s in series) >= len(series) for series in held): + return period + raise RuleError("rolling_anchor_unresolved") + + +def _slots(cards: list[str], held: list[str | None]) -> list[str | None]: + count = max(len(cards), len(held)) + return [cards[i] if i < len(cards) else None for i in range(count)] + + +def _check_token(share: dict[str, Any], step: dict[str, Any]) -> None: + token = step.get("write_token") + if token is None: + raise RuleError("write_token_required") + if token != "held" or not share["has_token"]: + raise RuleError("write_token_invalid") + + +def _state(share: dict[str, Any]) -> dict[str, Any]: + return { + "latest": share["latest"], + "slots": [ + {"slot": number, "card": card} + for number, card in enumerate(share["slots"], start=1) + ], + } + + +def publish(steps: list[dict[str, Any]]) -> list[dict[str, Any]]: + shares: dict[str, dict[str, Any]] = {} + outcomes: list[dict[str, Any]] = [] + for step in steps: + op = step["op"] + if op == "share": + share = shares[step["share"]] = { + "latest": step["latest"], + "has_token": not step.get("tokenless", False), + "slots": _slots(step["cards"], []), + } + outcomes.append({"share": _state(share)}) + continue + share = shares[step["share"]] + try: + if op in ("replace", "unshare"): + _check_token(share, step) + if op == "replace": + if not share["latest"]: + raise RuleError("share_not_replaceable") + share["slots"] = _slots(step["cards"], share["slots"]) + if op == "unshare": + del shares[step["share"]] + outcomes.append({"removed": True}) + continue + except RuleError as error: + outcomes.append({"error": error.code}) + continue + outcomes.append({"share": _state(share)}) + return outcomes + + +def _rows(rows: list[dict[str, Any]]) -> list[str]: + return sorted(jcs(row) for row in rows) + + +@pytest.mark.parametrize("vector", _vectors("output-digest"), ids=_ids("output-digest")) +def test_output_digest(vector: dict[str, Any]) -> None: + assert output_digest(vector["frame"], vector.get("variables", {})) == ( + vector["document"], + vector["digest"], + ) + + +@pytest.mark.parametrize( + "vector", _vectors("output-comparison"), ids=_ids("output-comparison") +) +def test_output_comparison(vector: dict[str, Any]) -> None: + left, right = vector["left"], vector["right"] + assert refs_equal(left, right) is vector["equal"] + assert same_data(left, right) is vector["same_data"] + assert refs_equal(right, left) is vector["equal"] + assert same_data(right, left) is vector["same_data"] + + +@pytest.mark.parametrize( + "vector", _vectors("operation-outputs"), ids=_ids("operation-outputs") +) +def test_operation_outputs(vector: dict[str, Any]) -> None: + if "error" in vector: + with pytest.raises(RuleError) as excinfo: + operation_output(vector) + assert excinfo.value.code == vector["error"] + return + assert _rows(operation_output(vector)) == _rows(vector["rows"]) + + +@pytest.mark.parametrize( + "vector", _vectors("rolling-composition"), ids=_ids("rolling-composition") +) +def test_rolling_composition(vector: dict[str, Any]) -> None: + if "error" in vector: + with pytest.raises(RuleError) as excinfo: + rolling_anchor(vector) + assert excinfo.value.code == vector["error"] + return + assert rolling_anchor(vector) == vector["period"] + + +@pytest.mark.parametrize("vector", _vectors("publication"), ids=_ids("publication")) +def test_publication(vector: dict[str, Any]) -> None: + assert publish(vector["steps"]) == vector["outcomes"] diff --git a/python/client-contract/tests/test_rules.py b/python/client-contract/tests/test_rules.py index 4dc86d9..72345f2 100644 --- a/python/client-contract/tests/test_rules.py +++ b/python/client-contract/tests/test_rules.py @@ -312,6 +312,15 @@ def posted_entities( return posted +def series_left_out(members: list[dict[str, Any]]) -> bool: + """Whether a fetch of derived members with these derived manifests issues + `series_left_out`: once, when any manifest lists a left-out series or says + `left_out_complete` false. A null `left_out` issues nothing.""" + return any( + member["left_out"] or member["left_out_complete"] is False for member in members + ) + + def selector_query(rendering: dict[str, Any]) -> str: def written(value: Any) -> str: return ("true" if value else "false") if isinstance(value, bool) else value @@ -449,6 +458,17 @@ def test_role_keys(vector: dict[str, Any]) -> None: assert posted == vector["posted"] +@pytest.mark.parametrize( + "vector", _vectors("series-left-out"), ids=_ids("series-left-out") +) +def test_series_left_out(vector: dict[str, Any]) -> None: + issued = ["series_left_out"] if series_left_out(vector["members"]) else [] + if vector["strict"] and issued: + assert vector.get("raises") == issued[0] + else: + assert vector.get("warnings") == issued + + @pytest.mark.parametrize("vector", _vectors("file-selector"), ids=_ids("file-selector")) def test_file_selector(vector: dict[str, Any]) -> None: assert selector_query(vector["rendering"]) == vector["query"] diff --git a/python/client-contract/uv.lock b/python/client-contract/uv.lock index bee0e2a..3bcd032 100644 --- a/python/client-contract/uv.lock +++ b/python/client-contract/uv.lock @@ -140,6 +140,7 @@ dev = [ { name = "pytest" }, { name = "pytest-cov" }, { name = "referencing" }, + { name = "rfc8785" }, { name = "ruff" }, { name = "ty" }, ] @@ -153,6 +154,7 @@ dev = [ { name = "pytest", specifier = ">=9.1.0" }, { name = "pytest-cov", specifier = ">=7.1.0" }, { name = "referencing", specifier = ">=0.36" }, + { name = "rfc8785", specifier = ">=0.1.4" }, { name = "ruff", specifier = ">=0.15.0" }, { name = "ty", specifier = ">=0.0.59" }, ] @@ -383,6 +385,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/2c/58/ca301544e1fa93ed4f80d724bf5b194f6e4b945841c5bfd555878eea9fcb/referencing-0.37.0-py3-none-any.whl", hash = "sha256:381329a9f99628c9069361716891d34ad94af76e461dcb0335825aecc7692231", size = 26766, upload-time = "2025-10-13T15:30:47.625Z" }, ] +[[package]] +name = "rfc8785" +version = "0.1.4" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/ef/2f/fa1d2e740c490191b572d33dbca5daa180cb423c24396b856f5886371d8b/rfc8785-0.1.4.tar.gz", hash = "sha256:e545841329fe0eee4f6a3b44e7034343100c12b4ec566dc06ca9735681deb4da", size = 14321, upload-time = "2024-09-27T16:33:31.206Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/4d/78/119878110660b2ad709888c8a1614fce7e2fab39080ab960656dc8605bf6/rfc8785-0.1.4-py3-none-any.whl", hash = "sha256:520d690b448ecf0703691c76e1a34a24ddcd4fc5bc41d589cb7c58ec651bcd48", size = 9240, upload-time = "2024-09-27T16:33:29.683Z" }, +] + [[package]] name = "rpds-py" version = "2026.9.1" diff --git a/python/descriptions/CHANGELOG.md b/python/descriptions/CHANGELOG.md index 3c6e776..6a59d01 100644 --- a/python/descriptions/CHANGELOG.md +++ b/python/descriptions/CHANGELOG.md @@ -136,5 +136,18 @@ and this package adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 repeated drawn `ref` and a card drawn from a computed version carrying a measure. The card-revision and live-rendering vectors carry the measure, fixed values and drawn values with graph definitions. +- `operations.json` names each analysis operation's `verb`, the function a + client calls, and its `output`, the shape of the rows it writes; an + operation that takes inputs by role carries `arguments`, the roles a + client's `data` and `other` fill and how `on` pairs them, so `sum` and + `compare` pair by role as `quotient`, `share` and `join` do. `compare` gives + `data` minus `other`, with `data` in the `right` role. The `percent` unit + rule names its unit, `Percent`, in `ref`, and a relative growth has no unit. + `tools/validate_descriptions_vectors.py` checks each verb is listed once and + the arguments fill the op's two roles. +- `sum`, `share` and `compare` list `unmatched_key` under `left_out`, the + members of one side with no partner on the other. The `sum` and `compare` + descriptions cover the paired form: members of `data` and `other` pair by + role through `on`, and a call to `compare` returns `data` minus `other`. [unreleased]: https://github.com/DevelopmentFinanceObservatory/spec-kit/commits/main/python diff --git a/python/descriptions/README.md b/python/descriptions/README.md index 201231d..8bd92d1 100644 --- a/python/descriptions/README.md +++ b/python/descriptions/README.md @@ -118,25 +118,27 @@ The size bound is on the stored list and `check_steps` checks it where the list `operations.json` lists, for each operation, its kind, whether it takes one input, inputs by role or none, its one fixed policy, the rule for the unit it produces, the coverage it records, and the data gaps it can leave out with what each counts. The analysis operations carry the policy names and exclusion reasons the `ai` repo's analysis engine already uses. -| `op` | Kind | Inputs | Policy | Unit | Coverage | Known `left_out` reasons | -| ----------------- | --------------- | -------------------------- | ----------------------------------- | ------------------------------------- | ---------- | -------------------------------------------------------------------------------------- | -| `retrieve` | retrieval | one version | `retrieve_minted_version` | as recorded | | | -| `sum` | analysis | `left`, `right` | `sum_of_intersect_periods` | the inputs' shared unit | | `period_unmatched`, `unit_unbound` | -| `quotient` | analysis | `numerator`, `denominator` | `quotient_of_intersect_periods` | composed | per period | `unmatched_key`, `period_unmatched`, `zero_denominator`, `unit_unbound`, `party_total` | -| `share` | analysis | `numerator`, `denominator` | `share_of_intersect_periods` | per cent | per period | `period_unmatched`, `unit_unbound`, `party_total` | -| `growth` | analysis | one | `growth_endpoint_years` | the input's, or a dimensionless ratio | | `endpoint_missing`, `zero_or_sign_crossing_base`, `non_finite`, `unit_unbound` | -| `rank` | analysis | one | `rank_what_is_present` | the input's, or none for an ordinal | per entity | `party_total`, `unit_unbound` | -| `aggregate` | analysis | one | `aggregate_skip_absent_grain_cells` | the input's | per period | `party_total`, `unit_unbound` | -| `filter` | analysis | one | `filter_row_value_predicates` | the input's | | | -| `compare` | analysis | `left`, `right` | `compare_intersect_periods` | the inputs' shared unit | | `period_unmatched`, `unit_unbound` | -| `join` | analysis | `left`, `right` | `join_inner` | each side its own | | `unmatched_key`, `unit_unbound` | -| `source_step` | source pipeline | none | `source_pipeline_as_recorded` | as recorded | | | -| `choose_source` | mint choice | none | `facet_default_by_coverage` | as recorded | | | -| `sum_variables` | mint choice | none | `sum_family_members` | the summed variables' shared unit | | | -| `convert` | mint choice | none | `convert_by_rate_and_deflator` | converted | | `currency_unmapped`, `deflator_missing` | -| `continue_series` | mint choice | none | `continue_with_source_estimates` | as recorded | | | -| `select` | card selection | one version | `card_selection` | the version's | | | -| `change` | card derivation | one | `change_between_basis_periods` | the input's, or per cent | | | +Each analysis operation also names its `verb`, the function a client calls, and its `output`, the shape of the rows it writes with a sentence on their values. `share` writes per cent, 100 times the ratio, under the unit `Percent`, which its unit's `ref` names. `join` writes long format, one row per side at each key and period, told apart by `stat_var`. `growth` writes one row per member with the window as its period, and a relative change as a fraction with no unit. An operation that takes inputs by role carries `arguments`: the role a client's first reading, `data`, fills, the role its second, `other`, fills, and how `on` pairs their roles. `sum` and `compare` pair by `on` as `quotient`, `share` and `join` do, and each side is one series when `on` is left out. `compare` subtracts the left series from the right, so `data` fills `right` and the client's call gives `data` minus `other`. The client contract's `operation-outputs` rule pins these units and shapes with vectors. + +| `op` | Kind | Inputs | Policy | Unit | Coverage | Known `left_out` reasons | +| ----------------- | --------------- | -------------------------- | ----------------------------------- | ----------------------------------- | ---------- | -------------------------------------------------------------------------------------- | +| `retrieve` | retrieval | one version | `retrieve_minted_version` | as recorded | | | +| `sum` | analysis | `left`, `right` | `sum_of_intersect_periods` | the inputs' shared unit | | `unmatched_key`, `period_unmatched`, `unit_unbound` | +| `quotient` | analysis | `numerator`, `denominator` | `quotient_of_intersect_periods` | composed | per period | `unmatched_key`, `period_unmatched`, `zero_denominator`, `unit_unbound`, `party_total` | +| `share` | analysis | `numerator`, `denominator` | `share_of_intersect_periods` | per cent | per period | `unmatched_key`, `period_unmatched`, `unit_unbound`, `party_total` | +| `growth` | analysis | one | `growth_endpoint_years` | the input's, or none for a fraction | | `endpoint_missing`, `zero_or_sign_crossing_base`, `non_finite`, `unit_unbound` | +| `rank` | analysis | one | `rank_what_is_present` | the input's, or none for an ordinal | per entity | `party_total`, `unit_unbound` | +| `aggregate` | analysis | one | `aggregate_skip_absent_grain_cells` | the input's | per period | `party_total`, `unit_unbound` | +| `filter` | analysis | one | `filter_row_value_predicates` | the input's | | | +| `compare` | analysis | `left`, `right` | `compare_intersect_periods` | the inputs' shared unit | | `unmatched_key`, `period_unmatched`, `unit_unbound` | +| `join` | analysis | `left`, `right` | `join_inner` | each side its own | | `unmatched_key`, `unit_unbound` | +| `source_step` | source pipeline | none | `source_pipeline_as_recorded` | as recorded | | | +| `choose_source` | mint choice | none | `facet_default_by_coverage` | as recorded | | | +| `sum_variables` | mint choice | none | `sum_family_members` | the summed variables' shared unit | | | +| `convert` | mint choice | none | `convert_by_rate_and_deflator` | converted | | `currency_unmapped`, `deflator_missing` | +| `continue_series` | mint choice | none | `continue_with_source_estimates` | as recorded | | | +| `select` | card selection | one version | `card_selection` | the version's | | | +| `change` | card derivation | one | `change_between_basis_periods` | the input's, or per cent | | | ## Descriptions diff --git a/python/descriptions/src/dfo_descriptions/schemas/operations.json b/python/descriptions/src/dfo_descriptions/schemas/operations.json index 4e31ff1..06fb175 100644 --- a/python/descriptions/src/dfo_descriptions/schemas/operations.json +++ b/python/descriptions/src/dfo_descriptions/schemas/operations.json @@ -17,66 +17,115 @@ }, { "op": "sum", + "verb": "sum_of", "kind": "analysis", "input": "roles", - "roles": ["left", "right"], + "roles": [ + "left", + "right" + ], + "arguments": { + "data": "left", + "other": "right", + "on": "Pairs each role of `data` with the role of `other` that holds the same entities, a role to a role, or names the same role on both sides as a list. Left out, each side is one series." + }, "policy": "sum_of_intersect_periods", "new_variable": true, "unit": { "rule": "same_as_input", "text": "The unit both sides share." }, + "output": { + "shape": "per_key_and_period", + "text": "One row for each paired key and period both sides hold, its value `data` plus `other`, its keys named as `data` names them." + }, "coverage": null, "left_out": [ + { + "reason": "unmatched_key", + "of": [ + "entities" + ], + "text": "Members of one side with no partner on the other, so their total is unknown." + }, { "reason": "period_unmatched", - "of": ["periods"], + "of": [ + "periods" + ], "text": "Periods one side reports and the other does not, so their total is unknown." }, { "reason": "unit_unbound", - "of": ["entity_periods"], + "of": [ + "entity_periods" + ], "text": "Values that declare no unit." } ], - "description": "Adds two single-entity series period by period, over the periods both report." + "description": "Adds two series period by period, over the periods both report. Given `on`, it pairs the members of `data` and `other` by role and adds each pair, leaving out a member with no partner." }, { "op": "quotient", + "verb": "quotient_of", "kind": "analysis", "input": "roles", - "roles": ["numerator", "denominator"], + "roles": [ + "numerator", + "denominator" + ], + "arguments": { + "data": "numerator", + "other": "denominator", + "on": "Pairs each role of `data` with the role of `other` that holds the same entities, a role to a role, or names the same role on both sides as a list." + }, "policy": "quotient_of_intersect_periods", "new_variable": true, "unit": { "rule": "composed", "text": "The numerator's unit per the denominator's." }, + "output": { + "shape": "per_key_and_period", + "text": "One row for each numerator key and period the denominator also holds, its value `data` divided by `other`." + }, "coverage": "period_coverage", "left_out": [ { "reason": "unmatched_key", - "of": ["entities"], + "of": [ + "entities" + ], "text": "Numerator entities with no denominator." }, { "reason": "period_unmatched", - "of": ["entities", "entity_periods"], + "of": [ + "entities", + "entity_periods" + ], "text": "Entities that share no period with their denominator, or values in periods the other side lacks." }, { "reason": "zero_denominator", - "of": ["entities", "entity_periods"], + "of": [ + "entities", + "entity_periods" + ], "text": "Entities, or values, whose denominator sums to zero." }, { "reason": "unit_unbound", - "of": ["entity_periods"], + "of": [ + "entity_periods" + ], "text": "Values that declare no unit." }, { "reason": "party_total", - "of": ["pooled_entities"], + "of": [ + "pooled_entities" + ], "text": "Flow parties of a pooled role that total other parties of it, left out so nothing counts twice; one entry per pooled role, counting its distinct parties across the step's inputs." } ], @@ -84,30 +133,58 @@ }, { "op": "share", + "verb": "share_of", "kind": "analysis", "input": "roles", - "roles": ["numerator", "denominator"], + "roles": [ + "numerator", + "denominator" + ], + "arguments": { + "data": "numerator", + "other": "denominator", + "on": "Pairs each role of `data` with the role of `other` that holds the same entities, a role to a role, or names the same role on both sides as a list. With no `other`, each member of `by` is divided by the total across every member of `by` in the same period." + }, "policy": "share_of_intersect_periods", "new_variable": true, "unit": { "rule": "percent", - "text": "Per cent." + "ref": "Percent", + "text": "Per cent, under the unit `Percent`." + }, + "output": { + "shape": "per_key_and_period", + "text": "One row for each key and period, its value 100 times `data` divided by `other`." }, "coverage": "period_coverage", "left_out": [ + { + "reason": "unmatched_key", + "of": [ + "entities" + ], + "text": "Members of one side with no partner on the other, so their share is unknown." + }, { "reason": "period_unmatched", - "of": ["periods", "entity_periods"], + "of": [ + "periods", + "entity_periods" + ], "text": "Periods, or values, one side reports and the other does not." }, { "reason": "unit_unbound", - "of": ["entity_periods"], + "of": [ + "entity_periods" + ], "text": "Values that declare no unit." }, { "reason": "party_total", - "of": ["pooled_entities"], + "of": [ + "pooled_entities" + ], "text": "Flow parties of a pooled role that total other parties of it, left out so nothing counts twice; one entry per pooled role, counting its distinct parties across the step's inputs." } ], @@ -115,6 +192,7 @@ }, { "op": "growth", + "verb": "growth", "kind": "analysis", "input": "single", "roles": [], @@ -122,28 +200,40 @@ "new_variable": false, "unit": { "rule": "same_as_input_or_dimensionless", - "text": "The input's unit for an absolute change, a dimensionless ratio for a relative one." + "text": "The input's unit for an absolute change; none for a relative one, whose value is a fraction." + }, + "output": { + "shape": "per_member_over_window", + "text": "One row for each member holding a value at both ends of the window, its period the window as a span such as `2018/2022`, its value the change: a fraction for a relative change, 0.25 for 25 per cent, and an amount in the input's unit for an absolute one." }, "coverage": null, "left_out": [ { "reason": "endpoint_missing", - "of": ["entities"], + "of": [ + "entities" + ], "text": "Entities with no value in the start or the end period." }, { "reason": "zero_or_sign_crossing_base", - "of": ["entities"], + "of": [ + "entities" + ], "text": "Entities whose base is zero or changes sign, so a relative change is undefined." }, { "reason": "non_finite", - "of": ["entities"], + "of": [ + "entities" + ], "text": "Entities whose change overflows to a non-finite value." }, { "reason": "unit_unbound", - "of": ["entity_periods"], + "of": [ + "entity_periods" + ], "text": "Values that declare no unit." } ], @@ -151,6 +241,7 @@ }, { "op": "rank", + "verb": "rank", "kind": "analysis", "input": "single", "roles": [], @@ -160,16 +251,24 @@ "rule": "same_as_input_or_none", "text": "The input's unit; none for a per-period ranking without a limit, whose values are rank positions." }, + "output": { + "shape": "ranked_members", + "text": "One row for each ranked member, or each member and period for a ranking per period, its value the measured value, or the rank position for a ranking per period without a limit." + }, "coverage": "entity_coverage", "left_out": [ { "reason": "party_total", - "of": ["pooled_entities"], + "of": [ + "pooled_entities" + ], "text": "Flow parties of a pooled role that total other parties of it, left out so nothing counts twice; one entry per pooled role, counting its distinct parties across the step's inputs." }, { "reason": "unit_unbound", - "of": ["entity_periods"], + "of": [ + "entity_periods" + ], "text": "Values that declare no unit." } ], @@ -177,6 +276,7 @@ }, { "op": "aggregate", + "verb": "aggregate", "kind": "analysis", "input": "single", "roles": [], @@ -186,16 +286,24 @@ "rule": "same_as_input", "text": "The input's unit." }, + "output": { + "shape": "per_key_and_period", + "text": "One row for each key and period, its value the reduction of the rows it pools." + }, "coverage": "period_coverage", "left_out": [ { "reason": "party_total", - "of": ["pooled_entities"], + "of": [ + "pooled_entities" + ], "text": "Flow parties of a pooled role that total other parties of it, left out so nothing counts twice; one entry per pooled role, counting its distinct parties across the step's inputs." }, { "reason": "unit_unbound", - "of": ["entity_periods"], + "of": [ + "entity_periods" + ], "text": "Values that declare no unit." } ], @@ -203,6 +311,7 @@ }, { "op": "filter", + "verb": "filter", "kind": "analysis", "input": "single", "roles": [], @@ -212,57 +321,102 @@ "rule": "same_as_input", "text": "The input's unit." }, + "output": { + "shape": "input_rows", + "text": "The input's rows that meet the predicates, unchanged." + }, "coverage": null, "left_out": [], "description": "Keeps the values that meet the caller's predicates. What the predicates drop is the caller's selection and stays in the parameters." }, { "op": "compare", + "verb": "compare", "kind": "analysis", "input": "roles", - "roles": ["left", "right"], + "roles": [ + "left", + "right" + ], + "arguments": { + "data": "right", + "other": "left", + "on": "Pairs each role of `data` with the role of `other` that holds the same entities, a role to a role, or names the same role on both sides as a list. Left out, each side is one series." + }, "policy": "compare_intersect_periods", "new_variable": true, "unit": { "rule": "same_as_input", "text": "The unit both sides share." }, + "output": { + "shape": "per_key_and_period", + "text": "One row for each paired key and period both sides hold, its value `data` minus `other`, its keys named as `data` names them. `data` fills the `right` role, since the step subtracts the left series from the right." + }, "coverage": null, "left_out": [ + { + "reason": "unmatched_key", + "of": [ + "entities" + ], + "text": "Members of one side with no partner on the other." + }, { "reason": "period_unmatched", - "of": ["periods"], + "of": [ + "periods" + ], "text": "Periods one side reports and the other does not." }, { "reason": "unit_unbound", - "of": ["entity_periods"], + "of": [ + "entity_periods" + ], "text": "Values that declare no unit." } ], - "description": "Subtracts the left series from the right for each period both report, and writes the difference under a variable of its own. The step records in `sides` the entities each side holds." + "description": "Subtracts the left series from the right for each period both report, so a call returns `data` minus `other`, and writes the difference under a variable of its own. Given `on`, it pairs the members of `data` and `other` by role and compares each pair, leaving out a member with no partner. The step records in `sides` the entities each side holds." }, { "op": "join", + "verb": "join", "kind": "analysis", "input": "roles", - "roles": ["left", "right"], + "roles": [ + "left", + "right" + ], + "arguments": { + "data": "left", + "other": "right", + "on": "Pairs each role of `data` with the role of `other` that holds the same entities, a role to a role, or names the same role on both sides as a list. Left out, the roles both readings hold under one name." + }, "policy": "join_inner", "new_variable": false, "unit": { "rule": "per_side", "text": "Each side keeps its own unit." }, + "output": { + "shape": "long_by_stat_var", + "text": "Two rows for each paired key and period both readings hold, one from each side, told apart by `stat_var`, each keeping its own value and unit, the key named as `data` names it." + }, "coverage": null, "left_out": [ { "reason": "unmatched_key", - "of": ["entity_periods"], + "of": [ + "entity_periods" + ], "text": "Keys and periods one side holds and the other does not." }, { "reason": "unit_unbound", - "of": ["entity_periods"], + "of": [ + "entity_periods" + ], "text": "Values that declare no unit." } ], @@ -328,12 +482,17 @@ "left_out": [ { "reason": "currency_unmapped", - "of": ["entities"], + "of": [ + "entities" + ], "text": "Entities no single national currency names, in a conversion into each entity's own currency." }, { "reason": "deflator_missing", - "of": ["entities", "entity_periods"], + "of": [ + "entities", + "entity_periods" + ], "text": "Entities with no deflator in the base year, or values in years the deflator table lacks." } ], diff --git a/python/descriptions/src/dfo_descriptions/schemas/operations.schema.json b/python/descriptions/src/dfo_descriptions/schemas/operations.schema.json index 8dbeb80..e6ec469 100644 --- a/python/descriptions/src/dfo_descriptions/schemas/operations.schema.json +++ b/python/descriptions/src/dfo_descriptions/schemas/operations.schema.json @@ -5,7 +5,9 @@ "description": "The operations a step record can name: for each, its kind, how it takes its inputs, its one fixed policy, whether it writes a new variable, the unit it produces, the coverage it records, and the data gaps it can leave out with what each counts.", "type": "object", "additionalProperties": false, - "required": ["operations"], + "required": [ + "operations" + ], "properties": { "operations": { "type": "array", @@ -35,13 +37,38 @@ "op": { "$ref": "steps.schema.json#/$defs/name" }, + "verb": { + "description": "The analysis function a client calls for this operation, such as `share_of` for `share`.", + "enum": [ + "aggregate", + "compare", + "filter", + "growth", + "join", + "quotient_of", + "rank", + "share_of", + "sum_of" + ] + }, "kind": { "description": "`retrieval` reads a version; `analysis` is one of the analysis engine's operations; `source_pipeline` is a step the source's pipeline applied before the DFO read the data; `mint_choice` is a choice the DFO made while minting a version; `card_selection` is what a card draws from its version; `card_derivation` is a value a card derives from what it draws.", - "enum": ["retrieval", "analysis", "source_pipeline", "mint_choice", "card_selection", "card_derivation"] + "enum": [ + "retrieval", + "analysis", + "source_pipeline", + "mint_choice", + "card_selection", + "card_derivation" + ] }, "input": { "description": "`single`: one `input`. `roles`: `inputs` keyed by the names in `roles`. `none`: the step takes no input, since it records a choice inside the version being made.", - "enum": ["single", "roles", "none"] + "enum": [ + "single", + "roles", + "none" + ] }, "roles": { "type": "array", @@ -50,6 +77,28 @@ "$ref": "steps.schema.json#/$defs/name" } }, + "arguments": { + "description": "How a client's call fills the roles of an analysis operation that takes inputs by role: `data`, its first reading, and `other`, its second, each name the role it fills, and `on` states how the call pairs the two readings' roles.", + "type": "object", + "additionalProperties": false, + "required": [ + "data", + "other", + "on" + ], + "properties": { + "data": { + "$ref": "steps.schema.json#/$defs/name" + }, + "other": { + "$ref": "steps.schema.json#/$defs/name" + }, + "on": { + "type": "string", + "minLength": 1 + } + } + }, "policy": { "$ref": "steps.schema.json#/$defs/name" }, @@ -60,10 +109,13 @@ "unit": { "type": "object", "additionalProperties": false, - "required": ["rule", "text"], + "required": [ + "rule", + "text" + ], "properties": { "rule": { - "description": "`as_recorded`: the unit the version holds. `same_as_input`: the input's unit, which every input must share. `composed`: the numerator's unit per the denominator's. `percent`: per cent. `same_as_input_or_dimensionless`: the input's unit for an absolute change, a dimensionless ratio for a relative one. `per_side`: each side keeps its own unit. `converted`: the unit converted into. `same_as_input_or_percent`: the input's unit for a difference, per cent for a percent change. `same_as_input_or_none`: the input's unit for a ranking of values, none for an ordinal ranking, whose values are rank positions.", + "description": "`as_recorded`: the unit the version holds. `same_as_input`: the input's unit, which every input must share. `composed`: the numerator's unit per the denominator's. `percent`: per cent, 100 times the ratio, under the unit `ref` names. `same_as_input_or_dimensionless`: the input's unit for an absolute change, none for a relative one, whose value is a fraction. `per_side`: each side keeps its own unit. `converted`: the unit converted into. `same_as_input_or_percent`: the input's unit for a difference, per cent for a percent change. `same_as_input_or_none`: the input's unit for a ranking of values, none for an ordinal ranking, whose values are rank positions.", "enum": [ "as_recorded", "same_as_input", @@ -76,6 +128,35 @@ "same_as_input_or_none" ] }, + "ref": { + "description": "The unit's graph reference, for a rule that names one unit: `Percent` for `percent`.", + "type": "string", + "minLength": 1 + }, + "text": { + "type": "string", + "minLength": 1 + } + } + }, + "output": { + "description": "The rows an analysis operation writes. `per_key_and_period`: one row for each key and period. `long_by_stat_var`: one row for each side at each key and period, told apart by `stat_var`. `per_member_over_window`: one row for each member, its period the window as a span. `ranked_members`: one row for each ranked member, or each member and period. `input_rows`: the input's rows that pass.", + "type": "object", + "additionalProperties": false, + "required": [ + "shape", + "text" + ], + "properties": { + "shape": { + "enum": [ + "per_key_and_period", + "long_by_stat_var", + "per_member_over_window", + "ranked_members", + "input_rows" + ] + }, "text": { "type": "string", "minLength": 1 @@ -84,7 +165,11 @@ }, "coverage": { "description": "`period_coverage`: how many entities each period's total covers. `entity_coverage`: how many periods each ranked entity holds. `null`: none.", - "enum": ["period_coverage", "entity_coverage", null] + "enum": [ + "period_coverage", + "entity_coverage", + null + ] }, "left_out": { "description": "The data gaps the operation can leave out. A step may name a reason this list lacks.", @@ -92,7 +177,11 @@ "items": { "type": "object", "additionalProperties": false, - "required": ["reason", "of", "text"], + "required": [ + "reason", + "of", + "text" + ], "properties": { "reason": { "$ref": "steps.schema.json#/$defs/name" @@ -103,7 +192,12 @@ "minItems": 1, "uniqueItems": true, "items": { - "enum": ["entities", "periods", "entity_periods", "pooled_entities"] + "enum": [ + "entities", + "periods", + "entity_periods", + "pooled_entities" + ] } }, "text": { @@ -117,7 +211,102 @@ "type": "string", "minLength": 1 } - } + }, + "allOf": [ + { + "if": { + "properties": { + "kind": { + "const": "analysis" + } + } + }, + "then": { + "required": [ + "verb", + "output" + ] + }, + "else": { + "not": { + "anyOf": [ + { + "required": [ + "verb" + ] + }, + { + "required": [ + "output" + ] + }, + { + "required": [ + "arguments" + ] + } + ] + } + } + }, + { + "if": { + "properties": { + "kind": { + "const": "analysis" + }, + "input": { + "const": "roles" + } + } + }, + "then": { + "required": [ + "arguments" + ] + }, + "else": { + "not": { + "required": [ + "arguments" + ] + } + } + }, + { + "if": { + "properties": { + "unit": { + "properties": { + "rule": { + "const": "percent" + } + } + } + } + }, + "then": { + "properties": { + "unit": { + "required": [ + "ref" + ] + } + } + }, + "else": { + "properties": { + "unit": { + "not": { + "required": [ + "ref" + ] + } + } + } + } + } + ] } } } diff --git a/python/spec-kit/CHANGELOG.md b/python/spec-kit/CHANGELOG.md index d2b6879..70e3735 100644 --- a/python/spec-kit/CHANGELOG.md +++ b/python/spec-kit/CHANGELOG.md @@ -64,6 +64,32 @@ and this package adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 differ from the published figure. Its params name the total (`total`, `total_label`) and the party classes it covers (`whole_of`: `official`, `private` or `residual`, comma-joined). +- Code `allowance_exhausted` in `registries/codes.json` and its bundled + mirror: a 429 refusal on `datasets.pin`, `presentation.shares` and + `presentation.write` once the caller has spent its daily allowance of + pins, shares or cards. Its problem document says `retryable: false`, and + its `Retry-After` header names the reset. +- Endpoint `datasets.analysis` in the registry schema, with its refusals in + `registries/codes.json` and its bundled mirror: 36 new codes at 422, + `rolling_populations_differ`, `several_members` and the 34 the analysis + engine serves, among them `unit_heterogeneous`, `cross_frame_unit_mismatch`, + `join_output_collision`, `rolling_anchor_unresolved`, `role_ambiguous`, + `compare_no_key_matched` and `sum_of_no_key_matched`; `timeout` at 504; and + 27 existing codes at the status each has on `datasets.mint`, + `datasets.snapshot` or the platform rows, among them `request_invalid` + (400), `snapshot_revoked` (410), `over_ceiling` (422), `upstream_error` + (502) and `service_unavailable` (503). +- Endpoint `presentation.shares` in the registry schema, with its refusals in + `registries/codes.json` and its bundled mirror: `write_token_required` (403) + for a replace or delete without a write token; `write_token_invalid` (403) + for one whose token is not the share's, or that names a share created + without a token; `share_not_replaceable` (409); `payload_too_large` (413) + for a body larger than the share routes accept; and `binding_invalid`, + `schema_invalid`, `constraint_violation`, `resource_exhausted` and + `request_invalid` (400), `not_found` and `frame_unavailable` (404), + `edit_conflict` (409), `internal_error` (500), `release_mismatch` and + `service_unavailable` (503), and `timeout` (504), at the statuses the + components routes give them. ### Changed @@ -95,6 +121,9 @@ and this package adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Code `upstream_error` maps to `graph.members` at 502, the status the members route answers when the read that checks whether the graph holds the node fails upstream, as `graph.node` answers for the same read. +- `pin_not_permitted` is retired in `registries/codes.json` and its bundled + mirror: the pin route admits every caller under its allowance, so nothing + raises it. Its `datasets.pin` mapping stays. - `starved_axis_binding_restored` is retired in `registries/codes.json` and its bundled mirror: Search no longer restores a starved constraint axis, so nothing raises it. diff --git a/python/spec-kit/src/dfo_spec_kit/registry/codes.json b/python/spec-kit/src/dfo_spec_kit/registry/codes.json index faf341b..6ec8b03 100644 --- a/python/spec-kit/src/dfo_spec_kit/registry/codes.json +++ b/python/spec-kit/src/dfo_spec_kit/registry/codes.json @@ -1,6 +1,32 @@ { "format_version": 1, "entries": [ + { + "code": "allowance_exhausted", + "classification": "refusal", + "definition": "The caller has spent its daily allowance for this kind of write, shares, cards or pins, so the write is refused until the allowance resets, which the response's Retry-After header names. The problem document says retryable false, so a client raises at once and waits for no reset.", + "owning_contract": "platform", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.pin", + "channel": "problem_details", + "http_status": 429 + }, + { + "endpoint": "presentation.write", + "channel": "problem_details", + "http_status": 429 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 429 + } + ], + "lifecycle": "active" + }, { "code": "alternates_unevaluated", "classification": "warning", @@ -47,6 +73,11 @@ "endpoint": "datasets.files", "channel": "problem_details", "http_status": 410 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 410 } ], "lifecycle": "active" @@ -186,6 +217,11 @@ "endpoint": "presentation.render", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 400 } ], "lifecycle": "active" @@ -288,6 +324,11 @@ "endpoint": "datasets.files", "channel": "problem_details", "http_status": 501 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 501 } ], "lifecycle": "active" @@ -429,6 +470,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -475,6 +521,54 @@ ], "lifecycle": "active" }, + { + "code": "compare_no_key_matched", + "classification": "refusal", + "definition": "No member of a compare step's data side pairs with a member of its other side through the step's on mapping, so there is nothing to compare.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "comparison_periods_disjoint", + "classification": "refusal", + "definition": "The two sides of a compare, quotient or sum_of step have no period in common.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "composed_unit_sum_unsupported", + "classification": "refusal", + "definition": "An analysis step adds values whose unit is a quotient, and quotients added together are not a quotient in that unit.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "conflicting_requested_options", "classification": "refusal", @@ -587,6 +681,11 @@ "endpoint": "presentation.render", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 400 } ], "lifecycle": "active" @@ -618,6 +717,70 @@ ], "lifecycle": "active" }, + { + "code": "coverage_insufficient", + "classification": "refusal", + "definition": "The grid an aggregate step groups over is too sparse: the share of expected cells with a value is below the floor the step requires.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "cross_frame_facet_mismatch", + "classification": "refusal", + "definition": "The two sides of an analysis step hold one StatVar read under different values of a facet, whose values do not combine.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "cross_frame_unit_mismatch", + "classification": "refusal", + "definition": "The two sides of an analysis step resolve to different units, and combining their values would be wrong.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "cross_frame_unit_unbound", + "classification": "refusal", + "definition": "One side of an analysis step declares no unit while the other declares one, so the sides cannot be combined.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "currency_unmapped_omitted", "classification": "warning", @@ -691,6 +854,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -742,6 +910,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -867,6 +1040,38 @@ ], "lifecycle": "active" }, + { + "code": "derived_input_ordinal", + "classification": "refusal", + "definition": "An analysis step reads a derived result whose values are rank positions, and a ratio or growth rate over a rank position has no meaning.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "derived_input_unusable", + "classification": "refusal", + "definition": "An analysis step reads a derived result whose rows hold spans of periods, which a growth step cannot window over.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "direction_discarded", "classification": "warning", @@ -943,6 +1148,11 @@ "endpoint": "presentation.render", "channel": "problem_details", "http_status": 409 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 409 } ], "lifecycle": "active" @@ -1285,6 +1495,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -1340,6 +1555,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -1371,6 +1591,22 @@ ], "lifecycle": "active" }, + { + "code": "facet_heterogeneous", + "classification": "refusal", + "definition": "An analysis step reads an input that carries more than one value of a facet, such as the measurement method, and the response lists the candidates.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "facet_not_available", "classification": "refusal", @@ -1427,6 +1663,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -1467,6 +1708,38 @@ ], "lifecycle": "active" }, + { + "code": "filter_predicate_type_mismatch", + "classification": "refusal", + "definition": "A filter step names a predicate of a kind, or with a value of a type, that does not fit the column it applies to.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "filter_unit_conversion_unsupported", + "classification": "refusal", + "definition": "A filter step puts a unit predicate on an input whose units are converted, which the step cannot match.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "frame_not_refreshable", "classification": "refusal", @@ -1483,6 +1756,22 @@ ], "lifecycle": "active" }, + { + "code": "frame_too_large", + "classification": "refusal", + "definition": "The result of an analysis step is too large to store, so the step is refused and nothing is stored.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "frame_unavailable", "classification": "refusal", @@ -1500,6 +1789,27 @@ "endpoint": "presentation.render", "channel": "problem_details", "http_status": 409 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 404 + } + ], + "lifecycle": "active" + }, + { + "code": "growth_gap_year", + "classification": "refusal", + "definition": "No entity has a value in both endpoint years of a growth step, so the step has no growth to compute.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -1541,6 +1851,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 500 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 500 } ], "lifecycle": "active" @@ -1607,6 +1922,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 500 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 500 } ], "lifecycle": "active" @@ -1688,6 +2008,16 @@ "endpoint": "datasets.files", "channel": "problem_details", "http_status": 500 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 500 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 500 } ], "lifecycle": "active" @@ -1734,6 +2064,38 @@ ], "lifecycle": "active" }, + { + "code": "join_no_common_key_columns", + "classification": "refusal", + "definition": "A join step names no key columns and the two sides share no role column to join on.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "join_output_collision", + "classification": "refusal", + "definition": "The two sides of a join step have the same StatVar, unit and facets, so the rows a matched key emits cannot be told apart.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "member_not_resolvable", "classification": "refusal", @@ -1766,6 +2128,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -1834,6 +2201,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 500 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 500 } ], "lifecycle": "active" @@ -1893,6 +2265,22 @@ ], "lifecycle": "active" }, + { + "code": "mixed_time_grain", + "classification": "refusal", + "definition": "An input frame holds periods of more than one grain, or a mix of points and spans, so no analysis step can read it, and reading it again gives the same periods.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "multi_recipient_truncated", "classification": "warning", @@ -2057,6 +2445,22 @@ ], "lifecycle": "active" }, + { + "code": "non_finite_result", + "classification": "refusal", + "definition": "An analysis step overflows to a value that is not finite, so repeating the step with the same input reaches the same result.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "not_found", "classification": "refusal", @@ -2079,6 +2483,11 @@ "endpoint": "presentation.render", "channel": "problem_details", "http_status": 404 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 404 } ], "lifecycle": "active" @@ -2436,6 +2845,22 @@ ], "lifecycle": "active" }, + { + "code": "op_spec_invalid", + "classification": "refusal", + "definition": "The recipe holds an analysis step whose op-spec is not valid, such as an unknown field, a missing argument or a value out of range.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "open_role_unavailable", "classification": "refusal", @@ -2487,6 +2912,22 @@ ], "lifecycle": "active" }, + { + "code": "operation_uncovered", + "classification": "refusal", + "definition": "An aggregate step names an aggregation the analysis does not support.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "option_coverage_probe_failed", "classification": "warning", @@ -2612,6 +3053,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -2665,6 +3111,22 @@ ], "lifecycle": "active" }, + { + "code": "payload_too_large", + "classification": "refusal", + "definition": "A share's request body is larger than the share routes accept.", + "owning_contract": "presentation", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 413 + } + ], + "lifecycle": "active" + }, { "code": "pin_not_permitted", "classification": "refusal", @@ -2679,7 +3141,7 @@ "http_status": 403 } ], - "lifecycle": "active" + "lifecycle": "retired" }, { "code": "pivot_not_homogeneous", @@ -2784,6 +3246,22 @@ ], "lifecycle": "active" }, + { + "code": "place_levels_mixed", + "classification": "refusal", + "definition": "A sum step pools parties whose totals contain one another, such as a region and its countries, which would count members twice.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "place_scope_defaulted", "classification": "warning", @@ -2987,6 +3465,54 @@ ], "lifecycle": "active" }, + { + "code": "quotient_of_no_key_matched", + "classification": "refusal", + "definition": "No key of a quotient step's numerator matches a key of its denominator, so no quotient can be computed.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "quotient_of_side_unit_unsupported", + "classification": "refusal", + "definition": "A side of a quotient step cannot be a part of the result's unit: it is a share or growth rate, already a composed unit, or a sum_of result with no unit.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "quotient_of_zero_denominator", + "classification": "refusal", + "definition": "The denominator of a quotient step sums to zero in every matched cell, so the step has no quotient to return.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "rate_coverage_gap", "classification": "refusal", @@ -3030,6 +3556,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -3085,6 +3616,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -3101,6 +3637,11 @@ "endpoint": "datasets.recipe", "channel": "problem_details", "http_status": 404 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 404 } ], "lifecycle": "active" @@ -3162,6 +3703,11 @@ "endpoint": "presentation.render", "channel": "problem_details", "http_status": 409 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 503 } ], "lifecycle": "active" @@ -3191,7 +3737,12 @@ "introduced_in": "v1", "mappings": [ { - "endpoint": "datasets.refresh", + "endpoint": "datasets.refresh", + "channel": "problem_details", + "http_status": 422 + }, + { + "endpoint": "datasets.analysis", "channel": "problem_details", "http_status": 422 } @@ -3328,6 +3879,16 @@ "endpoint": "datasets.snapshot", "channel": "problem_details", "http_status": 400 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 400 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 400 } ], "lifecycle": "active" @@ -3402,6 +3963,11 @@ "endpoint": "presentation.render", "channel": "problem_details", "http_status": 409 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 400 } ], "lifecycle": "active" @@ -3452,6 +4018,22 @@ ], "lifecycle": "active" }, + { + "code": "role_ambiguous", + "classification": "refusal", + "definition": "A column an analysis step names, to group, rank, join or filter on, is not one its input has a grain for.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "role_declaration_unverified", "classification": "warning", @@ -3618,6 +4200,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -3640,6 +4227,54 @@ ], "lifecycle": "active" }, + { + "code": "rolling_anchor_unresolved", + "classification": "refusal", + "definition": "The latest period a rolling step counts from is not established, so the step cannot resolve its periods.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "rolling_grain_unsupported", + "classification": "refusal", + "definition": "A rolling step counts the latest period over data whose periods are not calendar years, which it does not support.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "rolling_populations_differ", + "classification": "refusal", + "definition": "The rolling steps of one analysis recipe count the latest period over different populations of series, so the recipe is refused before it runs, whether or not the periods they would settle on agree.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "rollup_not_summable", "classification": "refusal", @@ -3688,6 +4323,11 @@ "endpoint": "presentation.render", "channel": "problem_details", "http_status": 400 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 400 } ], "lifecycle": "active" @@ -3865,6 +4505,32 @@ "endpoint": "graph.documentation", "channel": "problem_details", "http_status": 503 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 503 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 503 + } + ], + "lifecycle": "active" + }, + { + "code": "several_members", + "classification": "refusal", + "definition": "An analysis step takes one reading for each input, and an input names a dataset that holds several.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -3888,6 +4554,86 @@ ], "lifecycle": "active" }, + { + "code": "share_not_replaceable", + "classification": "refusal", + "definition": "A replace names a snapshot share, whose cards are fixed for good. Only a live share takes new cards.", + "owning_contract": "presentation", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 409 + } + ], + "lifecycle": "active" + }, + { + "code": "share_of_grain_mismatch", + "classification": "refusal", + "definition": "No key of a share step has a period its numerator and denominator both report, so there is no period to divide.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "share_of_key_unknown", + "classification": "refusal", + "definition": "The denominator mapping of a share step covers only some of the numerator's key columns, which leaves the pairing of the others undefined.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "share_of_key_unmatched", + "classification": "refusal", + "definition": "No key of a share step's numerator has a denominator group, so no share can be computed.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "share_of_zero_denominator", + "classification": "refusal", + "definition": "The denominator of a share step sums to zero in a period, so the share is undefined.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "skill_attachment_kind_unconfirmed", "classification": "warning", @@ -4309,6 +5055,11 @@ "endpoint": "datasets.description", "channel": "problem_details", "http_status": 404 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 404 } ], "lifecycle": "active" @@ -4370,6 +5121,11 @@ "endpoint": "datasets.description", "channel": "problem_details", "http_status": 410 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 410 } ], "lifecycle": "active" @@ -4537,6 +5293,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 409 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 409 } ], "lifecycle": "active" @@ -4598,6 +5359,11 @@ "endpoint": "datasets.description", "channel": "problem_details", "http_status": 404 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 404 } ], "lifecycle": "active" @@ -4641,6 +5407,22 @@ ], "lifecycle": "retired" }, + { + "code": "stat_var_unresolved", + "classification": "refusal", + "definition": "An analysis step needs one StatVar and its input holds several, or none the step can use. A composite input read with one series per StatVar is the common case.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "stat_vars_summed", "classification": "warning", @@ -4668,6 +5450,22 @@ ], "lifecycle": "active" }, + { + "code": "sum_of_no_key_matched", + "classification": "refusal", + "definition": "No member of a sum_of step's data side pairs with a member of its other side through the step's on mapping, so there is nothing to add.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "time_text_not_parsed", "classification": "warning", @@ -4785,6 +5583,16 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 504 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 504 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 504 } ], "lifecycle": "active" @@ -4835,6 +5643,22 @@ ], "lifecycle": "active" }, + { + "code": "unit_heterogeneous", + "classification": "refusal", + "definition": "An analysis step reads an input that carries more than one unit, or a pinned unit the input does not declare, and the response lists the candidates to pin.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "unit_not_resolved", "classification": "refusal", @@ -5040,6 +5864,11 @@ "endpoint": "graph.members", "channel": "problem_details", "http_status": 502 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 502 } ], "lifecycle": "active" @@ -5081,6 +5910,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 503 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 503 } ], "lifecycle": "active" @@ -5208,6 +6042,38 @@ } ], "lifecycle": "active" + }, + { + "code": "write_token_invalid", + "classification": "refusal", + "definition": "A replace or delete of a share carries a write token other than the share's, or names a share created without one, which no token can change.", + "owning_contract": "presentation", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 403 + } + ], + "lifecycle": "active" + }, + { + "code": "write_token_required", + "classification": "refusal", + "definition": "A replace or delete of a share carries no write token.", + "owning_contract": "presentation", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 403 + } + ], + "lifecycle": "active" } ] } diff --git a/registries/codes.json b/registries/codes.json index faf341b..6ec8b03 100644 --- a/registries/codes.json +++ b/registries/codes.json @@ -1,6 +1,32 @@ { "format_version": 1, "entries": [ + { + "code": "allowance_exhausted", + "classification": "refusal", + "definition": "The caller has spent its daily allowance for this kind of write, shares, cards or pins, so the write is refused until the allowance resets, which the response's Retry-After header names. The problem document says retryable false, so a client raises at once and waits for no reset.", + "owning_contract": "platform", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.pin", + "channel": "problem_details", + "http_status": 429 + }, + { + "endpoint": "presentation.write", + "channel": "problem_details", + "http_status": 429 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 429 + } + ], + "lifecycle": "active" + }, { "code": "alternates_unevaluated", "classification": "warning", @@ -47,6 +73,11 @@ "endpoint": "datasets.files", "channel": "problem_details", "http_status": 410 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 410 } ], "lifecycle": "active" @@ -186,6 +217,11 @@ "endpoint": "presentation.render", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 400 } ], "lifecycle": "active" @@ -288,6 +324,11 @@ "endpoint": "datasets.files", "channel": "problem_details", "http_status": 501 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 501 } ], "lifecycle": "active" @@ -429,6 +470,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -475,6 +521,54 @@ ], "lifecycle": "active" }, + { + "code": "compare_no_key_matched", + "classification": "refusal", + "definition": "No member of a compare step's data side pairs with a member of its other side through the step's on mapping, so there is nothing to compare.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "comparison_periods_disjoint", + "classification": "refusal", + "definition": "The two sides of a compare, quotient or sum_of step have no period in common.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "composed_unit_sum_unsupported", + "classification": "refusal", + "definition": "An analysis step adds values whose unit is a quotient, and quotients added together are not a quotient in that unit.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "conflicting_requested_options", "classification": "refusal", @@ -587,6 +681,11 @@ "endpoint": "presentation.render", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 400 } ], "lifecycle": "active" @@ -618,6 +717,70 @@ ], "lifecycle": "active" }, + { + "code": "coverage_insufficient", + "classification": "refusal", + "definition": "The grid an aggregate step groups over is too sparse: the share of expected cells with a value is below the floor the step requires.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "cross_frame_facet_mismatch", + "classification": "refusal", + "definition": "The two sides of an analysis step hold one StatVar read under different values of a facet, whose values do not combine.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "cross_frame_unit_mismatch", + "classification": "refusal", + "definition": "The two sides of an analysis step resolve to different units, and combining their values would be wrong.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "cross_frame_unit_unbound", + "classification": "refusal", + "definition": "One side of an analysis step declares no unit while the other declares one, so the sides cannot be combined.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "currency_unmapped_omitted", "classification": "warning", @@ -691,6 +854,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -742,6 +910,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -867,6 +1040,38 @@ ], "lifecycle": "active" }, + { + "code": "derived_input_ordinal", + "classification": "refusal", + "definition": "An analysis step reads a derived result whose values are rank positions, and a ratio or growth rate over a rank position has no meaning.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "derived_input_unusable", + "classification": "refusal", + "definition": "An analysis step reads a derived result whose rows hold spans of periods, which a growth step cannot window over.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "direction_discarded", "classification": "warning", @@ -943,6 +1148,11 @@ "endpoint": "presentation.render", "channel": "problem_details", "http_status": 409 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 409 } ], "lifecycle": "active" @@ -1285,6 +1495,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -1340,6 +1555,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -1371,6 +1591,22 @@ ], "lifecycle": "active" }, + { + "code": "facet_heterogeneous", + "classification": "refusal", + "definition": "An analysis step reads an input that carries more than one value of a facet, such as the measurement method, and the response lists the candidates.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "facet_not_available", "classification": "refusal", @@ -1427,6 +1663,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -1467,6 +1708,38 @@ ], "lifecycle": "active" }, + { + "code": "filter_predicate_type_mismatch", + "classification": "refusal", + "definition": "A filter step names a predicate of a kind, or with a value of a type, that does not fit the column it applies to.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "filter_unit_conversion_unsupported", + "classification": "refusal", + "definition": "A filter step puts a unit predicate on an input whose units are converted, which the step cannot match.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "frame_not_refreshable", "classification": "refusal", @@ -1483,6 +1756,22 @@ ], "lifecycle": "active" }, + { + "code": "frame_too_large", + "classification": "refusal", + "definition": "The result of an analysis step is too large to store, so the step is refused and nothing is stored.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "frame_unavailable", "classification": "refusal", @@ -1500,6 +1789,27 @@ "endpoint": "presentation.render", "channel": "problem_details", "http_status": 409 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 404 + } + ], + "lifecycle": "active" + }, + { + "code": "growth_gap_year", + "classification": "refusal", + "definition": "No entity has a value in both endpoint years of a growth step, so the step has no growth to compute.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -1541,6 +1851,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 500 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 500 } ], "lifecycle": "active" @@ -1607,6 +1922,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 500 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 500 } ], "lifecycle": "active" @@ -1688,6 +2008,16 @@ "endpoint": "datasets.files", "channel": "problem_details", "http_status": 500 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 500 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 500 } ], "lifecycle": "active" @@ -1734,6 +2064,38 @@ ], "lifecycle": "active" }, + { + "code": "join_no_common_key_columns", + "classification": "refusal", + "definition": "A join step names no key columns and the two sides share no role column to join on.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "join_output_collision", + "classification": "refusal", + "definition": "The two sides of a join step have the same StatVar, unit and facets, so the rows a matched key emits cannot be told apart.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "member_not_resolvable", "classification": "refusal", @@ -1766,6 +2128,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -1834,6 +2201,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 500 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 500 } ], "lifecycle": "active" @@ -1893,6 +2265,22 @@ ], "lifecycle": "active" }, + { + "code": "mixed_time_grain", + "classification": "refusal", + "definition": "An input frame holds periods of more than one grain, or a mix of points and spans, so no analysis step can read it, and reading it again gives the same periods.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "multi_recipient_truncated", "classification": "warning", @@ -2057,6 +2445,22 @@ ], "lifecycle": "active" }, + { + "code": "non_finite_result", + "classification": "refusal", + "definition": "An analysis step overflows to a value that is not finite, so repeating the step with the same input reaches the same result.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "not_found", "classification": "refusal", @@ -2079,6 +2483,11 @@ "endpoint": "presentation.render", "channel": "problem_details", "http_status": 404 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 404 } ], "lifecycle": "active" @@ -2436,6 +2845,22 @@ ], "lifecycle": "active" }, + { + "code": "op_spec_invalid", + "classification": "refusal", + "definition": "The recipe holds an analysis step whose op-spec is not valid, such as an unknown field, a missing argument or a value out of range.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "open_role_unavailable", "classification": "refusal", @@ -2487,6 +2912,22 @@ ], "lifecycle": "active" }, + { + "code": "operation_uncovered", + "classification": "refusal", + "definition": "An aggregate step names an aggregation the analysis does not support.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "option_coverage_probe_failed", "classification": "warning", @@ -2612,6 +3053,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -2665,6 +3111,22 @@ ], "lifecycle": "active" }, + { + "code": "payload_too_large", + "classification": "refusal", + "definition": "A share's request body is larger than the share routes accept.", + "owning_contract": "presentation", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 413 + } + ], + "lifecycle": "active" + }, { "code": "pin_not_permitted", "classification": "refusal", @@ -2679,7 +3141,7 @@ "http_status": 403 } ], - "lifecycle": "active" + "lifecycle": "retired" }, { "code": "pivot_not_homogeneous", @@ -2784,6 +3246,22 @@ ], "lifecycle": "active" }, + { + "code": "place_levels_mixed", + "classification": "refusal", + "definition": "A sum step pools parties whose totals contain one another, such as a region and its countries, which would count members twice.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "place_scope_defaulted", "classification": "warning", @@ -2987,6 +3465,54 @@ ], "lifecycle": "active" }, + { + "code": "quotient_of_no_key_matched", + "classification": "refusal", + "definition": "No key of a quotient step's numerator matches a key of its denominator, so no quotient can be computed.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "quotient_of_side_unit_unsupported", + "classification": "refusal", + "definition": "A side of a quotient step cannot be a part of the result's unit: it is a share or growth rate, already a composed unit, or a sum_of result with no unit.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "quotient_of_zero_denominator", + "classification": "refusal", + "definition": "The denominator of a quotient step sums to zero in every matched cell, so the step has no quotient to return.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "rate_coverage_gap", "classification": "refusal", @@ -3030,6 +3556,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -3085,6 +3616,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -3101,6 +3637,11 @@ "endpoint": "datasets.recipe", "channel": "problem_details", "http_status": 404 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 404 } ], "lifecycle": "active" @@ -3162,6 +3703,11 @@ "endpoint": "presentation.render", "channel": "problem_details", "http_status": 409 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 503 } ], "lifecycle": "active" @@ -3191,7 +3737,12 @@ "introduced_in": "v1", "mappings": [ { - "endpoint": "datasets.refresh", + "endpoint": "datasets.refresh", + "channel": "problem_details", + "http_status": 422 + }, + { + "endpoint": "datasets.analysis", "channel": "problem_details", "http_status": 422 } @@ -3328,6 +3879,16 @@ "endpoint": "datasets.snapshot", "channel": "problem_details", "http_status": 400 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 400 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 400 } ], "lifecycle": "active" @@ -3402,6 +3963,11 @@ "endpoint": "presentation.render", "channel": "problem_details", "http_status": 409 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 400 } ], "lifecycle": "active" @@ -3452,6 +4018,22 @@ ], "lifecycle": "active" }, + { + "code": "role_ambiguous", + "classification": "refusal", + "definition": "A column an analysis step names, to group, rank, join or filter on, is not one its input has a grain for.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "role_declaration_unverified", "classification": "warning", @@ -3618,6 +4200,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 422 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -3640,6 +4227,54 @@ ], "lifecycle": "active" }, + { + "code": "rolling_anchor_unresolved", + "classification": "refusal", + "definition": "The latest period a rolling step counts from is not established, so the step cannot resolve its periods.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "rolling_grain_unsupported", + "classification": "refusal", + "definition": "A rolling step counts the latest period over data whose periods are not calendar years, which it does not support.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "rolling_populations_differ", + "classification": "refusal", + "definition": "The rolling steps of one analysis recipe count the latest period over different populations of series, so the recipe is refused before it runs, whether or not the periods they would settle on agree.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "rollup_not_summable", "classification": "refusal", @@ -3688,6 +4323,11 @@ "endpoint": "presentation.render", "channel": "problem_details", "http_status": 400 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 400 } ], "lifecycle": "active" @@ -3865,6 +4505,32 @@ "endpoint": "graph.documentation", "channel": "problem_details", "http_status": 503 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 503 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 503 + } + ], + "lifecycle": "active" + }, + { + "code": "several_members", + "classification": "refusal", + "definition": "An analysis step takes one reading for each input, and an input names a dataset that holds several.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 } ], "lifecycle": "active" @@ -3888,6 +4554,86 @@ ], "lifecycle": "active" }, + { + "code": "share_not_replaceable", + "classification": "refusal", + "definition": "A replace names a snapshot share, whose cards are fixed for good. Only a live share takes new cards.", + "owning_contract": "presentation", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 409 + } + ], + "lifecycle": "active" + }, + { + "code": "share_of_grain_mismatch", + "classification": "refusal", + "definition": "No key of a share step has a period its numerator and denominator both report, so there is no period to divide.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "share_of_key_unknown", + "classification": "refusal", + "definition": "The denominator mapping of a share step covers only some of the numerator's key columns, which leaves the pairing of the others undefined.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "share_of_key_unmatched", + "classification": "refusal", + "definition": "No key of a share step's numerator has a denominator group, so no share can be computed.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, + { + "code": "share_of_zero_denominator", + "classification": "refusal", + "definition": "The denominator of a share step sums to zero in a period, so the share is undefined.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "skill_attachment_kind_unconfirmed", "classification": "warning", @@ -4309,6 +5055,11 @@ "endpoint": "datasets.description", "channel": "problem_details", "http_status": 404 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 404 } ], "lifecycle": "active" @@ -4370,6 +5121,11 @@ "endpoint": "datasets.description", "channel": "problem_details", "http_status": 410 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 410 } ], "lifecycle": "active" @@ -4537,6 +5293,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 409 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 409 } ], "lifecycle": "active" @@ -4598,6 +5359,11 @@ "endpoint": "datasets.description", "channel": "problem_details", "http_status": 404 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 404 } ], "lifecycle": "active" @@ -4641,6 +5407,22 @@ ], "lifecycle": "retired" }, + { + "code": "stat_var_unresolved", + "classification": "refusal", + "definition": "An analysis step needs one StatVar and its input holds several, or none the step can use. A composite input read with one series per StatVar is the common case.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "stat_vars_summed", "classification": "warning", @@ -4668,6 +5450,22 @@ ], "lifecycle": "active" }, + { + "code": "sum_of_no_key_matched", + "classification": "refusal", + "definition": "No member of a sum_of step's data side pairs with a member of its other side through the step's on mapping, so there is nothing to add.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "time_text_not_parsed", "classification": "warning", @@ -4785,6 +5583,16 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 504 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 504 + }, + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 504 } ], "lifecycle": "active" @@ -4835,6 +5643,22 @@ ], "lifecycle": "active" }, + { + "code": "unit_heterogeneous", + "classification": "refusal", + "definition": "An analysis step reads an input that carries more than one unit, or a pinned unit the input does not declare, and the response lists the candidates to pin.", + "owning_contract": "datasets", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 422 + } + ], + "lifecycle": "active" + }, { "code": "unit_not_resolved", "classification": "refusal", @@ -5040,6 +5864,11 @@ "endpoint": "graph.members", "channel": "problem_details", "http_status": 502 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 502 } ], "lifecycle": "active" @@ -5081,6 +5910,11 @@ "endpoint": "datasets.refresh", "channel": "problem_details", "http_status": 503 + }, + { + "endpoint": "datasets.analysis", + "channel": "problem_details", + "http_status": 503 } ], "lifecycle": "active" @@ -5208,6 +6042,38 @@ } ], "lifecycle": "active" + }, + { + "code": "write_token_invalid", + "classification": "refusal", + "definition": "A replace or delete of a share carries a write token other than the share's, or names a share created without one, which no token can change.", + "owning_contract": "presentation", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 403 + } + ], + "lifecycle": "active" + }, + { + "code": "write_token_required", + "classification": "refusal", + "definition": "A replace or delete of a share carries no write token.", + "owning_contract": "presentation", + "owner": "platform", + "introduced_in": "v1", + "mappings": [ + { + "endpoint": "presentation.shares", + "channel": "problem_details", + "http_status": 403 + } + ], + "lifecycle": "active" } ] } diff --git a/schemas/registry/README.md b/schemas/registry/README.md index 64825ab..df4da75 100644 --- a/schemas/registry/README.md +++ b/schemas/registry/README.md @@ -59,9 +59,9 @@ Retiring a door is a schema change, reviewed as one: dropping a value from `code - **`endpoint`** lists the Search endpoints `search.resolve` (`POST /api/search/resolve`) and `search.options` (`POST /api/search/options`). - The Datasets endpoints are `datasets.register` (`POST /api/datasets/register`), `datasets.mint` (`POST /api/datasets`), `datasets.definition` (`GET /api/datasets/{id}`), `datasets.options` (`GET /api/datasets/{id}/options`, the write-free preview door), `datasets.members` (`POST /api/datasets/{id}/members`, the write-free member read), `datasets.snapshot` (`GET /api/datasets/{id}/{content_version}`, and `GET /api/datasets/{id}/versions`, which pages a dataset's servable versions), `datasets.files` (`GET /api/datasets/{id}/{content_version}/files/{digest}`, which answers a stored artifact with a 302 redirect to its download link, and `GET /api/datasets/{id}/{content_version}/file`, which selects a stored version's file by format, bundle and layout, renders it when it is missing, and answers the same redirect), `datasets.description` (`GET /api/datasets/{id}/{content_version}/description`, a stored version's description, for a `spec_` id with a `cv_` version or an `ops_` id with an `ocv_` version, and `GET .../description/values`, which pages one dimension's values), `datasets.observations` (`GET /api/datasets/{id}/{content_version}/observations`, a stored version's observations), `datasets.pin` (`POST /api/datasets/{id}/{content_version}/pin`), `datasets.publish` (`POST /api/datasets/frames/publish`, the door an analysis service uses to publish an already-computed frame under its own identity pair), `datasets.recipe` (`PUT` and `GET /api/datasets/frames/{spec_id}/{content_version}/recipe`, the recorded computation behind a published frame, and `GET /api/datasets/recipes/replays/{replay_key}`, the frame stored under a replay key), `datasets.refresh` (`POST /api/datasets/frames/refresh`, which rebuilds a dataset frame's rendering on current data), and `datasets.operations` (`GET /api/datasets/operations/{operation_id}`). + The Datasets endpoints are `datasets.register` (`POST /api/datasets/register`), `datasets.mint` (`POST /api/datasets`), `datasets.definition` (`GET /api/datasets/{id}`), `datasets.options` (`GET /api/datasets/{id}/options`, the write-free preview door), `datasets.members` (`POST /api/datasets/{id}/members`, the write-free member read), `datasets.snapshot` (`GET /api/datasets/{id}/{content_version}`, and `GET /api/datasets/{id}/versions`, which pages a dataset's servable versions), `datasets.files` (`GET /api/datasets/{id}/{content_version}/files/{digest}`, which answers a stored artifact with a 302 redirect to its download link, and `GET /api/datasets/{id}/{content_version}/file`, which selects a stored version's file by format, bundle and layout, renders it when it is missing, and answers the same redirect), `datasets.description` (`GET /api/datasets/{id}/{content_version}/description`, a stored version's description, for a `spec_` id with a `cv_` version or an `ops_` id with an `ocv_` version, and `GET .../description/values`, which pages one dimension's values), `datasets.observations` (`GET /api/datasets/{id}/{content_version}/observations`, a stored version's observations), `datasets.pin` (`POST /api/datasets/{id}/{content_version}/pin`), `datasets.publish` (`POST /api/datasets/frames/publish`, the door an analysis service uses to publish an already-computed frame under its own identity pair), `datasets.recipe` (`PUT` and `GET /api/datasets/frames/{spec_id}/{content_version}/recipe`, the recorded computation behind a published frame, and `GET /api/datasets/recipes/replays/{replay_key}`, the frame stored under a replay key), `datasets.refresh` (`POST /api/datasets/frames/refresh`, which rebuilds a dataset frame's rendering on current data), `datasets.operations` (`GET /api/datasets/operations/{operation_id}`), and `datasets.analysis` (`POST /api/datasets/analysis`, which runs an analysis recipe tree and answers its derived frame, and `POST /api/datasets/analysis/anchor`, which resolves a recipe's latest period and stores nothing). - The Presentation endpoints are `presentation.write` (`POST /api/presentation/components`, `PATCH /api/presentation/components/{component_id}`) and `presentation.render` (`POST /api/presentation/components/{component_id}/revisions/{revision}/render` and `GET .../render/data`, which draw a stored revision on a named target frame and mint no revision). Search's and Datasets' `schema`, `capabilities`, and `healthz` routes and Presentation's read routes carry no permanent codes. + The Presentation endpoints are `presentation.write` (`POST /api/presentation/components`, `PATCH /api/presentation/components/{component_id}`) `presentation.render` (`POST /api/presentation/components/{component_id}/revisions/{revision}/render` and `GET .../render/data`, which draw a stored revision on a named target frame and mint no revision), and `presentation.shares` (the routes under `/api/presentation/shares` that create, replace and delete a share). Search's and Datasets' `schema`, `capabilities`, and `healthz` routes and Presentation's read routes carry no permanent codes. The Graph endpoints are `graph.node` (`GET /api/graph/node/{dcid}`), `graph.nodes` (`POST /api/graph/nodes`, a batch of node reads), `graph.members` (`GET /api/graph/node/{dcid}/members`, one page of a node's list), `graph.roots` (`GET /api/graph/roots`), `graph.documentation` (`GET /api/graph/documentation/{dcid}`, a source's document), `graph.sources` (`GET /api/graph/sources`), and `graph.resolve` (`POST /api/graph/resolve`). `graph.node` and `graph.documentation` carry `node_not_held`, `request_invalid`, `service_unavailable`, and `upstream_error`; `graph.nodes` and `graph.resolve` carry `request_invalid`, `unsupported_schema_version`, `service_unavailable`, and `upstream_error`; `graph.members` carries `node_not_held`, `request_invalid`, `service_unavailable`, and `upstream_error`, the last when the read that checks whether the graph holds the node fails upstream; and `graph.roots` carries `service_unavailable`. A node the graph does not hold answers inside a `graph.nodes` batch as a `not_held` entry, so that endpoint carries no `node_not_held`. Graph's `sources` and `capabilities` routes carry no permanent codes. diff --git a/schemas/registry/code-registry.schema.json b/schemas/registry/code-registry.schema.json index 5c46e24..23765d4 100644 --- a/schemas/registry/code-registry.schema.json +++ b/schemas/registry/code-registry.schema.json @@ -101,6 +101,7 @@ "search.options", "presentation.write", "presentation.render", + "presentation.shares", "datasets.register", "datasets.mint", "datasets.definition", @@ -115,6 +116,7 @@ "datasets.recipe", "datasets.refresh", "datasets.operations", + "datasets.analysis", "graph.node", "graph.nodes", "graph.members", @@ -243,6 +245,7 @@ "enum": [ "presentation.write", "presentation.render", + "presentation.shares", "datasets.register", "datasets.mint", "datasets.definition", @@ -257,6 +260,7 @@ "datasets.recipe", "datasets.refresh", "datasets.operations", + "datasets.analysis", "graph.node", "graph.nodes", "graph.members", diff --git a/tools/capture_client_contract.py b/tools/capture_client_contract.py index d2f108d..c069777 100644 --- a/tools/capture_client_contract.py +++ b/tools/capture_client_contract.py @@ -80,6 +80,7 @@ POLL_BACKOFF_CAP, POLL_BACKOFF_FIRST, POLL_PATH, + RETRY_AFTER_MAX_S, SCENARIOS_BUDGET, SIGNED_FETCH_STATUS, STORAGE_ORIGIN, @@ -749,7 +750,7 @@ def _retried( `may_retry` allows, waiting `retry_wait` before each retry.""" response = send() answers = [_read(response)] - while may_retry(request, answers): + while may_retry(request, answers, self.actions): wait = retry_wait(response.headers, len(answers) - 1) self.retry_waits.append(wait) self.sleep(wait) @@ -2184,6 +2185,18 @@ def _retry_after_503(parent: dict[str, Any], ctx: Context) -> Derived: return _refused_first(parent, 0, refused) +def _retry_after_past_cap(parent: dict[str, Any], ctx: Context) -> Derived: + """A retryable 503 whose Retry-After is one second past RETRY_AFTER_MAX_S + answers the search, which raises it unretried.""" + first = parent["exchanges"][0] + headers = {"retry-after": str(int(RETRY_AFTER_MAX_S) + 1)} + refused = _refusal(first["request"], 503, _BUSY, headers) + error = refusal_error( + 503, _BUSY, ctx.actions, refused["response"]["headers"] + ).outcome(ctx.client_codes) + return Derived(copy.deepcopy(parent["call"]), [refused], [error]) + + def _proxy_429(parent: dict[str, Any], ctx: Context) -> Derived: first = parent["exchanges"][0] refused = _refusal( @@ -3726,6 +3739,14 @@ def _empty_narrowing(parent: dict[str, Any], ctx: Context) -> Derived: "search then reaches the same answer; the retry spends one of the two default retries.", _retry_after_503, ), + Synthetic( + "search-retry-after-past-cap", + "search-place-ambiguous", + "A 503 service_unavailable with retryable true and Retry-After 61 answers the search.", + "A retryable refusal whose Retry-After is past the 60-second cap raises at once, with " + "no wait and no retry: ServerError with its code, service_unavailable, and its trace id.", + _retry_after_past_cap, + ), Synthetic( "search-proxy-rate-limited", "search-place-ambiguous", diff --git a/tools/tests/test_validate_client_contract.py b/tools/tests/test_validate_client_contract.py index 3982480..700a78e 100644 --- a/tools/tests/test_validate_client_contract.py +++ b/tools/tests/test_validate_client_contract.py @@ -96,6 +96,17 @@ def _edit(path: Path, change: Callable[[Any], None]) -> None: path.write_text(json.dumps(document), encoding="utf-8") +def _comparison_member(document: Any, name: str, side: str, index: int) -> Any: + """A member of the output-comparison vector `name` on `side`.""" + vector = next(vector for vector in document["vectors"] if vector["name"] == name) + return vector[side]["members"][index] + + +def _digest_vector(document: Any, name: str) -> Any: + """The output-digest vector `name`.""" + return next(vector for vector in document["vectors"] if vector["name"] == name) + + def test_the_real_contract_is_valid(package_dir: Path) -> None: assert validator.validate() == [] assert validator.main() == 0 @@ -190,6 +201,21 @@ def _vector(name: str, change: Callable[[Any], None]) -> Callable[[Any], None]: return lambda doc: change(next(v for v in doc["vectors"] if v["name"] == name)) +def _allowance_exhausted(doc: Any) -> None: + """The first search's refusal becomes a 429 `allowance_exhausted`, which + a client raises unretried whatever its document's `retryable` says.""" + response = doc["exchanges"][0]["response"] + response["status"] = 429 + response["json"].update( + code="allowance_exhausted", status=429, type="/codes/allowance_exhausted" + ) + + +def _rename_series_left_out(doc: Any) -> None: + vectors = doc["$defs"]["series_left_out"]["properties"]["vectors"] + vectors["items"]["properties"]["warnings"]["items"]["enum"] = ["series_omitted"] + + def _served_order_after_reading_columns(holder: Any) -> None: """Moves the varying co-operation modality pair back to its served place, just before `crsPurpose`, an order that ignores whether a column @@ -962,6 +988,107 @@ def change(doc: Any) -> None: ), "refs/ref-without-party-totals.json: member 0 records rendering", ), + ( + "refs/ref-derived-member.json", + lambda doc: doc["ref"]["members"][0]["roots"][0].update( + content_version="cv_" + "0" * 32 + ), + "member 0 root 0 is no member fetch-spec-known-sentence serves", + ), + ( + "refs/ref-derived-member.json", + lambda doc: doc["ref"]["members"][0]["roots"][0]["rendering"].pop( + "without_party_totals" + ), + "refs/ref-derived-member.json: member 0 root 0 records rendering", + ), + ( + "refs/ref-derived-member.json", + lambda doc: doc["ref"]["members"][0]["roots"][0]["rendering"].pop("layout"), + "refs/ref-derived-member.json: member 0 root 0 records rendering", + ), + ( + "refs/ref-derived-member.json", + lambda doc: doc["ref"]["members"][0]["resolved_periods"][0].update( + period="2019" + ), + "share replay_key but differ in resolved_periods", + ), + ( + "rules/output-comparison.json", + lambda doc: _comparison_member( + doc, "member-count-differs", "right", 1 + ).update( + content_version=_comparison_member( + doc, "member-count-differs", "right", 0 + )["content_version"] + ), + "share content_version but differ in spec_id", + ), + ( + "rules/output-comparison.json", + lambda doc: _comparison_member( + doc, "derived-file-format-differs", "right", 0 + )["artifact"].update( + digest=_comparison_member( + doc, "derived-file-format-differs", "left", 0 + )["artifact"]["digest"] + ), + "share artifact_digest but differ in artifact_format", + ), + ( + "rules/output-comparison.json", + lambda doc: _comparison_member(doc, "same-derived-ref", "right", 0)[ + "roots" + ][0]["rendering"].update( + groupby={"by": ["time_period"], "op": "sum", "levels": {}} + ), + "share content_version but differ in roots", + ), + ( + "rules/output-digest.json", + lambda doc: doc["vectors"][0]["frame"]["rows"][0].__setitem__( + 0, "dcid:Count_Person" + ), + "vector 'two-rows' document is not its frame's", + ), + ( + "rules/output-digest.json", + lambda doc: doc["vectors"][0].update(digest=doc["vectors"][1]["digest"]), + "vector 'two-rows' digest is not its document's", + ), + ( + "rules/output-digest.json", + lambda doc: next( + vector + for vector in doc["vectors"] + if vector["name"] == "quotient-of-sides-swapped" + ).update( + variables=next( + vector + for vector in doc["vectors"] + if vector["name"] == "quotient-of-variable-named" + )["variables"] + ), + "vector 'quotient-of-sides-swapped' document is not its frame's", + ), + ( + "rules/output-digest.json", + lambda doc: _digest_vector(doc, "quotient-of-a-sum-of")[ + "name_documents" + ].pop("mv_ae90cefe711d4addb439f873f2010dba"), + "mv_ae90cefe711d4addb439f873f2010dba has no name document", + ), + ( + "rules/output-digest.json", + lambda doc: _digest_vector(doc, "quotient-of-variable-named")[ + "name_documents" + ].update( + mv_d355e59c50ff8cf07e86a4da12b64297='{"frame_ref":["dcid:Amount_ODA"],' + '"verb":"quotient_of"}' + ), + "mv_d355e59c50ff8cf07e86a4da12b64297 is not its document's", + ), ( "rules/key-format.json", lambda doc: doc["vectors"].append(doc["vectors"][0]), @@ -1307,6 +1434,11 @@ def change(doc: Any) -> None: _drop_action("route_deprecated"), "no row for 'route_deprecated'", ), + ( + "warning-codes.json", + _drop_action("series_left_out"), + "no row for 'series_left_out'", + ), ( "warning-codes.json", _add_action( @@ -1817,6 +1949,41 @@ def change(doc: Any) -> None: "reading_columns" ), ), + ( + "rules/operation-outputs.json", + _vector( + "growth-as-a-fraction", + lambda v: v["arguments"].update(window=["2018", "2018"]), + ), + "vector 'growth-as-a-fraction' has a window that does not start before", + ), + ( + "rules/rules.schema.json", + _rename_series_left_out, + "warning-codes.json: no row for 'series_omitted'", + ), + ( + "refs/ref-derived-member.json", + lambda doc: doc["ref"]["members"][0].update(parts=["ODA to Kenya"]), + "refs/ref-derived-member.json: ref fails the schema at members/0/parts", + ), + ( + "scenarios/fetch-ref.json", + lambda doc: doc["call"][0]["ref"].update(format_version=2), + "scenarios/fetch-ref.json: schema: call/0/ref/format_version: 1 was", + ), + ( + "scenarios/search-retry-after-past-cap.json", + lambda doc: doc["exchanges"][0]["response"]["headers"].update( + {"retry-after": "60"} + ), + "the record holds no retry of POST /search/resolve", + ), + ( + "scenarios/search-retry-after-503.json", + _allowance_exhausted, + "scenarios/search-retry-after-503.json: the steps never make exchanges", + ), ( "rules/reading-order.json", _vector( @@ -1965,6 +2132,18 @@ def change(doc: Any) -> None: "ref-without-scenario", "ref-parts-not-served", "ref-rendering-not-posted", + "ref-derived-root-not-served", + "ref-derived-root-rendering-not-posted", + "ref-derived-root-not-in-frame-layout", + "ref-derived-replay-with-other-periods", + "derived-version-with-other-op-spec", + "derived-file-in-two-formats", + "derived-version-with-other-root-rendering", + "output-digest-document-not-frames", + "output-digest-not-documents", + "output-digest-variable-not-named", + "output-digest-inner-name-without-document", + "output-digest-name-not-its-documents", "rule-vector-repeated", "client-code-without-condition", "client-codes-unsorted", @@ -2030,6 +2209,7 @@ def change(doc: Any) -> None: "proxy-code-is-a-registry-code", "warning-code-ref-digest-changed-missing", "warning-code-missing", + "warning-code-series-left-out-missing", "warning-code-no-rule-issues", "warning-code-is-a-client-code", "location-localhost-alias", @@ -2141,6 +2321,12 @@ def change(doc: Any) -> None: "reading-order-series-interleaved-by-year", "reading-order-null-sorted-first", "reading-order-without-reading-columns", + "growth-window-that-does-not-advance", + "series-left-out-warning-renamed-in-its-rule", + "derived-member-with-parts", + "fetch-of-a-format-2-ref", + "retry-after-at-the-cap-waited", + "allowance-exhausted-not-retried", "reading-order-vector-row-ragged", "reading-order-vector-list-items-mix-types", "reading-order-vector-struct-field-mixes-types", diff --git a/tools/tests/test_validate_descriptions_vectors.py b/tools/tests/test_validate_descriptions_vectors.py index d2e5696..f41ef47 100644 --- a/tools/tests/test_validate_descriptions_vectors.py +++ b/tools/tests/test_validate_descriptions_vectors.py @@ -124,7 +124,25 @@ def test_an_operation_listed_twice_is_reported(package_dir: Path) -> None: document["operations"].append(document["operations"][1]) path.write_text(json.dumps(document)) - assert validator.validate() == ["operations.json: op 'sum' is listed twice"] + assert validator.validate() == [ + "operations.json: op 'sum' is listed twice", + "operations.json: verb 'sum_of' is listed twice", + ] + + +def test_arguments_outside_the_roles_are_reported(package_dir: Path) -> None: + path = package_dir / "schemas" / "operations.json" + document = json.loads(path.read_text()) + compare = next(e for e in document["operations"] if e["op"] == "compare") + compare["arguments"]["data"] = "numerator" + path.write_text(json.dumps(document)) + + assert validator.validate() == [ + ( + "operations.json: compare: arguments' data and other must fill its " + "two roles ['left', 'right']" + ) + ] def test_a_missing_analysis_operation_is_reported(package_dir: Path) -> None: diff --git a/tools/validate_client_contract.py b/tools/validate_client_contract.py index d9c1937..9f1502f 100644 --- a/tools/validate_client_contract.py +++ b/tools/validate_client_contract.py @@ -36,8 +36,8 @@ - `proxy-codes.json` and `warning-codes.json` list codes in order, each once, none of them a code another table or `registries/codes.json` holds, and the warning codes are those the client rules issue: `ref_digest_changed`, - `route_deprecated`, `server_warning`, and one `key_` per key-format - reason. + `route_deprecated`, `server_warning`, one `key_` per key-format + reason, and each warning the series-left-out rule's vectors name. - Each scenario's file name matches its `name`, names are unique, `expect` holds one outcome per step, a step that acts on an earlier result follows one, a synthetic scenario derives from a recorded one, and every body file @@ -86,19 +86,28 @@ SCENARIOS_BUDGET bytes. - Each ref example's file name matches its `name` and gets the verdict it states: from `dataset-ref.schema.json`, then from the rules that each - member's definition carries its own `spec_id` and that members hold + dataset member's definition carries its own `spec_id` and that members hold distinct `spec_id`s. A ref example whose ref holds members names the recorded `scenario` that serves them, and each member's `spec_id`, `parts`, `content_version` and `sentence` are a dataset member's of that scenario, and its artifact digest is the digest of a file the scenario downloads. - Each member of an example the schema accepts records the rendering the - scenario's build of its spec posted. + Each root of a derived member is a dataset member's `spec_id` and + `content_version` of that scenario. Each member of an example the schema + accepts records the rendering the scenario's build of its spec posted, and + each root of a derived one that rendering in the frame layout. +- The derived members of the ref examples the schema accepts and of the + output-comparison vectors agree wherever DERIVED_IDENTITY_RULES says one + identity fixes another, so one `ocv_` names one op-spec and one output. - Each rule file under `rules/` validates against `rules.schema.json`, names - the rule its file name names, and names each vector once, and each + the rule its file name names, and names each vector once, each reading-order vector's `columns` and `rows` are what `presented_rows` - gives, and each file-selector vector's `query` is what `selector_query` - gives. A missing or unreadable `rules/reading-order.json` or - `rules/file-selector.json` is a problem. + gives, each file-selector vector's `query` is what `selector_query` + gives, each output-digest vector's document holds its frame's rows, each + operation's variable read as the name its `variables` give it, and + its digest hashes that document, each minted name hashes its name + document and each name it or a name document holds has one, and each + growth vector's window starts before it ends. A missing or unreadable + `rules/reading-order.json` or `rules/file-selector.json` is a problem. It does not import `dfo_client_contract`, whose tests replay the rule vectors. `tools/capture_client_contract.py` imports the outcome @@ -503,18 +512,20 @@ def _check_other_codes( def _check_warning_codes( document: dict[str, Any], taken: Mapping[str, set[str]], - key_reasons: Sequence[str], + rules_schema: Mapping[str, Any], ) -> list[str]: """As `_check_other_codes`, and the codes are those a client's rules - issue: `ref_digest_changed`, each header warning, and one `key_` - warning per key-format reason.""" + issue: `ref_digest_changed`, each header warning, one `key_` + warning per key-format reason, and each warning the series-left-out + rule issues.""" codes = [row["code"] for row in document["codes"]] problems = _check_other_codes("warning-codes.json", codes, taken) wanted = { REF_DIGEST_CHANGED, ROUTE_DEPRECATED, SERVER_WARNING, - *(f"key_{r}" for r in key_reasons), + *(f"key_{r}" for r in key_format_reasons(rules_schema)), + *series_left_out_warnings(rules_schema), } problems.extend( f"warning-codes.json: no row for {code!r}, a warning a client issues" @@ -533,6 +544,13 @@ def key_format_reasons(rules_schema: Mapping[str, Any]) -> list[str]: return vectors["items"]["properties"]["reason"]["enum"] +def series_left_out_warnings(rules_schema: Mapping[str, Any]) -> set[str]: + """The warnings the series-left-out rule issues, or raises under strict.""" + vectors = rules_schema["$defs"]["series_left_out"]["properties"]["vectors"] + members = vectors["items"]["properties"] + return {*members["warnings"]["items"]["enum"], *members["raises"]["enum"]} + + # The class of every error the key-lookup rule raises. KEY_LOOKUP_CLASS = "AuthenticationError" @@ -773,6 +791,7 @@ def _check_proxy_problem(label: str, response: Mapping[str, Any]) -> list[str]: "take_newest_version": "VersionUnavailableError", "log_in": "AuthenticationError", "request_scope": "PermissionDeniedError", + "stop": "PermissionDeniedError", "wait": "RateLimitError", "wait_for_reset": "QuotaExceededError", "report": "ServerError", @@ -794,22 +813,33 @@ def _check_proxy_problem(label: str, response: Mapping[str, Any]) -> list[str]: RETRYABLE_STATUSES = frozenset({408, 429, 500, 502, 503, 504}) # The retries a client makes by default after a retryable refusal. MAX_RETRIES = 2 +# The longest `Retry-After`, in seconds, a client waits before it tries again. +# A refusal that asks for longer raises at once. +RETRY_AFTER_MAX_S = 60.0 +# The action of the refusals a client never retries, whatever their document +# and status say: a spent allowance clears only when it resets. +NOT_RETRIED_ACTION = "wait_for_reset" # The status that cuts a synchronous build off at the server's limit. BUILD_CUT_OFF = 504 def may_retry( - request: Mapping[str, Any], responses: Sequence[Mapping[str, Any]] + request: Mapping[str, Any], + responses: Sequence[Mapping[str, Any]], + actions: Mapping[str, Mapping[str, Any]], ) -> bool: """Whether a client sends `request` again after `responses`, the answers - to it so far in order: the last is a retryable refusal, the retries made - stay under MAX_RETRIES, and for a build, no earlier answer was a 504 - cut-off, after which the build is posted once more only. So a build - refused 504 then 503 raises the 503, and one refused 503 then 504 is - posted once more after the 504.""" + to it so far in order: the last is a refusal `tries_again` allows, the + retries made stay under MAX_RETRIES, and for a build, no earlier answer + was a 504 cut-off, after which the build is posted once more only. So a + build refused 504 then 503 raises the 503, and one refused 503 then 504 + is posted once more after the 504.""" *earlier, response = responses status = response["status"] - if status < 400 or not retryable(status, problem_body(response)): + headers = response.get("headers", {}) + if status < 400 or not tries_again( + status, problem_body(response), headers, actions + ): return False if len(earlier) >= MAX_RETRIES: return False @@ -854,14 +884,32 @@ def error_class( return STATUS_CLASSES.get(status, fallback) -def retryable(status: int, body: Any) -> bool: - """Whether a client retries a refusal: as its problem document's - `retryable` says, and otherwise by its status.""" +def retryable(status: int, body: Any, actions: Mapping[str, Mapping[str, Any]]) -> bool: + """Whether a client retries a refusal: never when its code's action is + NOT_RETRIED_ACTION, else as its problem document's `retryable` says, and + otherwise by its status.""" + row = actions.get(refusal_code(body) or "") + if row is not None and row["action"] == NOT_RETRIED_ACTION: + return False if isinstance(body, dict) and isinstance(body.get("retryable"), bool): return body["retryable"] return status in RETRYABLE_STATUSES +def tries_again( + status: int, + body: Any, + headers: Mapping[str, str], + actions: Mapping[str, Mapping[str, Any]], +) -> bool: + """Whether a client sends a refused request again, a poll included: the + refusal is `retryable` and its `Retry-After`, when it carries one, is at + most RETRY_AFTER_MAX_S.""" + wait = retry_after(headers) + over_cap = wait is not None and wait > RETRY_AFTER_MAX_S + return retryable(status, body, actions) and not over_cap + + def retry_after(headers: Mapping[str, str]) -> float | None: """The seconds a `Retry-After` header asks for, or None.""" try: @@ -2397,11 +2445,11 @@ def failed_operation( error=error_class(500, OPERATION_FAILED, actions), trace_id=trace_id(None, headers), ) - return error, retryable(500, None) + return error, retryable(500, None, actions) status = problem.get("status") status = status if isinstance(status, int) else 500 error = refusal_error(status, problem, actions, headers) - return error, retryable(status, problem) + return error, retryable(status, problem, actions) class LongBuild: @@ -2467,9 +2515,9 @@ def polled(self, status: int, headers: Mapping[str, str], body: Any) -> bool: error, again = failed_operation(body, self.actions, headers) else: error = refusal_error(status, body, self.actions, headers) - again = retryable(status, body) + again = retryable(status, body, self.actions) if error.code != OPERATION_FAILED: - if not again: + if not tries_again(status, body, headers, self.actions): raise error if status != 429: if self.retries >= MAX_RETRIES: @@ -2571,7 +2619,7 @@ def call( retries and self.cursor < len(self.exchanges) and self.exchanges[self.cursor]["request"] == request - and may_retry(request, answers) + and may_retry(request, answers, self.actions) ): headers = exchange["response"].get("headers", {}) self.retry_waits.append(retry_wait(headers, len(answers) - 1)) @@ -2579,7 +2627,7 @@ def call( self.cursor += 1 self.took(exchange) answers.append(exchange["response"]) - if retries and may_retry(request, answers): + if retries and may_retry(request, answers, self.actions): raise _UnfitError( f"exchange {self.cursor - 1} is a retryable refusal with retries " f"left, and the record holds no retry of {method} {target}" @@ -3596,7 +3644,7 @@ def _ref_rule(ref: dict[str, Any]) -> str | None: seen: set[str] = set() for member in ref["members"]: spec = member["spec"] - if spec["definition"]["spec_id"] != spec["spec_id"]: + if "definition" in spec and spec["definition"]["spec_id"] != spec["spec_id"]: return "definition_spec_id_mismatch" if spec["spec_id"] in seen: return "repeated_spec_id" @@ -3604,12 +3652,19 @@ def _ref_rule(ref: dict[str, Any]) -> str | None: return None +def _derived(member: Any) -> bool: + """Whether a ref member is a derived member, which names its roots.""" + return isinstance(member, dict) and "roots" in member + + def _served_by(label: str, record: dict[str, Any]) -> list[str]: """Where a ref example's members differ from the members its `scenario` serves: a `spec_id` no dataset member of it has, other `parts`, another `content_version` or `sentence`, or an artifact digest that is not the - digest of a file the scenario downloads. A ref with no members names no - scenario.""" + digest of a file the scenario downloads. A derived member's roots are + checked instead, each a dataset member's `spec_id` and `content_version`, + since no scenario serves the derived member itself. A ref with no members + names no scenario.""" ref = record.get("ref") members = ref.get("members") if isinstance(ref, dict) else None if not isinstance(members, list) or not members: @@ -3636,6 +3691,14 @@ def _served_by(label: str, record: dict[str, Any]) -> list[str]: digests.add(digest_of(path.read_bytes())) problems = [] for index, member in enumerate(members): + if _derived(member): + problems.extend( + f"{label}: member {index} root {position} is no member {name} serves" + for position, root in enumerate(read_objects(member, "roots")) + if (found := served.get(root.get("spec_id"))) is None + or found.get("content_version") != root.get("content_version") + ) + continue spec = member.get("spec") if isinstance(member, dict) else None found = served.get(spec.get("spec_id") if isinstance(spec, dict) else None) if found is None: @@ -3655,13 +3718,10 @@ def _served_by(label: str, record: dict[str, Any]) -> list[str]: return problems -def posted_renderings(scenario: Mapping[str, Any]) -> dict[str, list[Any]]: - """The renderings the builds in `scenario` posted as their `options`, by - the digest of the artifact each build's manifest lists for them: a - manifest's own, or for a 202 the manifest the poll of its operation - answered done. Builds that answered the same file list each rendering in - build order.""" - renderings: dict[str, list[Any]] = {} +def _answered_builds(scenario: Mapping[str, Any]) -> Iterator[tuple[Any, Any]]: + """Each build in `scenario` that answered a manifest, as the manifest and + the `options` the build posted: a manifest's own, or for a 202 the + manifest the poll of its operation answered done, in build order.""" by_operation: dict[str, Any] = {} for exchange in read_objects(scenario, "exchanges"): request = read_object(exchange, "request") or {} @@ -3676,12 +3736,32 @@ def posted_renderings(scenario: Mapping[str, Any]) -> dict[str, list[Any]]: options = by_operation[operation_id] else: continue + yield manifest, options + + +def posted_renderings(scenario: Mapping[str, Any]) -> dict[str, list[Any]]: + """The renderings the builds in `scenario` posted as their `options`, by + the digest of the artifact each build's manifest lists for them. Builds + that answered the same file list each rendering in build order.""" + renderings: dict[str, list[Any]] = {} + for manifest, options in _answered_builds(scenario): digest = _artifact_digest(manifest, options) if digest is not None: renderings.setdefault(digest, []).append(options) return renderings +def _spec_renderings(scenario: Mapping[str, Any]) -> dict[str, list[Any]]: + """The renderings the builds in `scenario` posted as their `options`, by + the spec_id each build's manifest answered, in build order.""" + renderings: dict[str, list[Any]] = {} + for manifest, options in _answered_builds(scenario): + spec_id = read_string(read_object(manifest, "definition"), "spec_id") + if spec_id is not None: + renderings.setdefault(spec_id, []).append(options) + return renderings + + def _artifact_digest(manifest: Any, options: Any) -> str | None: """The digest of the artifact `manifest` lists for the posted `options`, or None when either does not read.""" @@ -3692,19 +3772,54 @@ def _artifact_digest(manifest: Any, options: Any) -> str | None: return read_string(artifact, "artifact_digest") +def _root_rendering(posted: Any, rendering: Any) -> Any: + """The rendering a derived member's root records for a build that posted + `posted`: DFO reads an analysis root in the frame layout, and a root read + summed also records the `groupby` DFO rendered it with, which no recorded + build posts.""" + if not isinstance(posted, dict): + return posted + wanted = {**posted, "layout": "frame"} + if isinstance(rendering, dict) and "groupby" in rendering: + wanted.setdefault("groupby", rendering["groupby"]) + return wanted + + def _posted_by(label: str, record: dict[str, Any]) -> list[str]: """Where a ref example's member records a rendering other than the one the build that answered its file in its `scenario` posted, since a ref - records the rendering its fetch posted.""" + records the rendering its fetch posted, or a derived member's root + records one no build of its spec there posted, read as `_root_rendering` + gives. A root's `groupby` is the schema's alone to check.""" ref = record.get("ref") members = ref.get("members") if isinstance(ref, dict) else None name = record.get("scenario") path = PACKAGE_DIR / "scenarios" / f"{name}.json" if not isinstance(members, list) or not isinstance(name, str) or not path.is_file(): return [] # `_served_by` reports a ref that names no scenario - posted = posted_renderings(_load(path)) + scenario = _load(path) + posted = posted_renderings(scenario) + by_spec = _spec_renderings(scenario) problems = [] for index, member in enumerate(members): + if _derived(member): + for position, root in enumerate(read_objects(member, "roots")): + where = f"member {index} root {position}" + spec_id = read_string(root, "spec_id") + rendering = read_object(root, "rendering") + if spec_id not in by_spec: + problems.append( + f"{label}: {where}'s spec is built by no {name} build" + ) + continue + wanted = [_root_rendering(o, rendering) for o in by_spec[spec_id]] + if rendering not in wanted: + problems.append( + f"{label}: {where} records rendering {json.dumps(rendering)}, " + f"but {name}'s builds give " + f"{' and '.join(map(json.dumps, wanted))}" + ) + continue artifact = read_object(member, "artifact") digest = read_string(artifact, "digest") rendering = read_object(artifact, "rendering") @@ -3759,9 +3874,238 @@ def _check_refs(validator: jsonschema.Draft202012Validator) -> list[str]: return problems +# --- derived identities ----------------------------------------------------- + +# What fixes what among a derived member's identities. Two members that agree +# on every field of a key agree on every field it determines: `ocv_` hashes +# the `ops_` id, the root versions and the execution identity, and names one +# output; `rpk_` hashes the normal form, the root versions and the execution +# identities, so one replay computes one op-spec, version and output; the +# normal form, the root versions and the execution identity give one `rpk_`; +# and one file holds one output in one format. `roots` holds each root's +# version with the rendering it was read with, since a root read summed +# under a groupby is another input. +DERIVED_IDENTITY_RULES: tuple[tuple[tuple[str, ...], tuple[str, ...]], ...] = ( + ( + ("content_version",), + ("spec_id", "roots", "execution_identity", "output_digest"), + ), + ( + ("replay_key",), + ( + "normal_form_hash", + "roots", + "execution_identity", + "spec_id", + "content_version", + "output_digest", + "resolved_periods", + ), + ), + (("normal_form_hash", "roots", "execution_identity"), ("replay_key",)), + (("artifact_digest",), ("artifact_format", "output_digest")), +) + + +def derived_identities(member: Mapping[str, Any]) -> dict[str, str | None]: + """The identities of a derived member that `DERIVED_IDENTITY_RULES` read, + each as text, None where the member lacks it.""" + spec = read_object(member, "spec") + recipe = read_object(spec, "recipe") + artifact = read_object(member, "artifact") + roots = [ + [ + read_string(root, "spec_id"), + read_string(root, "content_version"), + read_object(root, "rendering"), + ] + for root in read_objects(member, "roots") + ] + periods = member.get("resolved_periods") + return { + "spec_id": read_string(spec, "spec_id"), + "normal_form_hash": read_string(recipe, "normal_form_hash"), + "replay_key": read_string(recipe, "replay_key"), + "content_version": read_string(member, "content_version"), + "output_digest": read_string(member, "output_digest"), + "roots": json.dumps(roots, sort_keys=True), + "execution_identity": read_string(member, "execution_identity"), + "resolved_periods": None + if periods is None + else json.dumps(periods, sort_keys=True), + "artifact_digest": read_string(artifact, "digest"), + "artifact_format": read_string(read_object(artifact, "rendering"), "format"), + } + + +def derived_identity_problems( + members: Sequence[tuple[str, Mapping[str, Any]]], +) -> list[str]: + """Where two labelled derived members agree on a key of + `DERIVED_IDENTITY_RULES` but differ in a field it determines.""" + identities = [(label, derived_identities(member)) for label, member in members] + problems = [] + for key, determined in DERIVED_IDENTITY_RULES: + first: dict[tuple[str | None, ...], tuple[str, dict[str, str | None]]] = {} + for label, facts in identities: + values = tuple(facts[name] for name in key) + if None in values: + continue + if values not in first: + first[values] = (label, facts) + continue + other_label, other = first[values] + problems.extend( + f"{label} and {other_label} share {' and '.join(key)} " + f"but differ in {name}" + for name in determined + if facts[name] != other[name] + ) + return problems + + +def _check_derived_identities() -> list[str]: + """Every derived member of a ref example the schema accepts and of the + output-comparison vectors, checked together, since the examples name one + computation in several places.""" + members: list[tuple[str, Mapping[str, Any]]] = [] + for path in sorted((PACKAGE_DIR / "refs").glob("*.json")): + record = _load(path) + if not isinstance(record, dict) or read_object(record, "error") == { + "by": "schema" + }: + continue + members.extend( + (f"refs/{path.name} member {index}", member) + for index, member in enumerate( + read_objects(read_object(record, "ref"), "members") + ) + if _derived(member) + ) + comparison = _load(PACKAGE_DIR / "rules" / "output-comparison.json") + for vector in read_objects(comparison, "vectors"): + for side in ("left", "right"): + members.extend( + ( + f"rules/output-comparison.json {vector.get('name')} {side} member {index}", + member, + ) + for index, member in enumerate( + read_objects(read_object(vector, side), "members") + ) + if _derived(member) + ) + return derived_identity_problems(members) + + # --- rules ------------------------------------------------------------------ +# The frame columns that name no role (docs/frame-grammar-v1.md). +FRAME_FIXED_COLUMNS = frozenset( + { + "stat_var", + "time_period", + "value", + "unit", + "measurementMethod", + "provenance", + "status", + } +) + + +def _without_recipe(unit: Any) -> Any: + """A unit cell with `recipe` dropped from every unit-request object.""" + if not isinstance(unit, dict): + return unit + if "numerator" in unit or "denominator" in unit: + return {side: _without_recipe(part) for side, part in unit.items()} + return {name: item for name, item in unit.items() if name != "recipe"} + + +def output_digest_problems(label: str, vector: Mapping[str, Any]) -> list[str]: + """Where an output-digest vector's `document` is not its frame's, read as + JSON, or its `digest` is not the document's: the columns `stat_var`, the + role columns, `time_period`, `value` and `unit`, each frame row once in + those columns with no unit `recipe` and a `stat_var` of `dcid:` and an + `ops_` id its `variables` names read as that name, digest version `out2`, + and `out_` and 32 hex digits of the document's SHA-256. The JCS bytes + themselves are the reference implementation's to check.""" + frame = vector["frame"] + variables = vector.get("variables", {}) + roles = [name for name in frame["columns"] if name not in FRAME_FIXED_COLUMNS] + columns = ["stat_var", *roles, "time_period", "value", "unit"] + rows = [] + for row in frame["rows"]: + cells = dict(zip(frame["columns"], row, strict=True)) + cells["unit"] = _without_recipe(cells.get("unit")) + stat_var = cells["stat_var"] + cells["stat_var"] = variables.get(stat_var.removeprefix("dcid:"), stat_var) + rows.append([cells.get(name) for name in columns]) + document = json.loads(vector["document"]) + problems = [] + wanted = {"digest_version": "out2", "columns": columns} + unmatched = list(document.get("rows", [])) + for row in rows: + if row in unmatched: + unmatched.remove(row) + else: + unmatched.append(None) + if {key: document.get(key) for key in wanted} != wanted or unmatched: + problems.append( + f"{label}: vector {vector['name']!r} document is not its frame's" + ) + hashed = "out_" + hashlib.sha256(vector["document"].encode()).hexdigest()[:32] + if vector["digest"] != hashed: + problems.append( + f"{label}: vector {vector['name']!r} digest is not its document's, {hashed}" + ) + return problems + name_document_problems(label, vector) + + +def name_document_problems(label: str, vector: Mapping[str, Any]) -> list[str]: + """Where an output-digest vector's `name_documents` misname a minted + variable: a name that is not `mv_` and 32 hex digits of its document's + SHA-256, or a name in `variables` or in a document's slots with no + document of its own. So a nested step's document holds its input's + `mv_` name, which has its own document.""" + documents = vector.get("name_documents", {}) + problems = [ + f"{label}: vector {vector['name']!r} name {name} is not its document's" + for name, text in documents.items() + if name != "mv_" + hashlib.sha256(text.encode()).hexdigest()[:32] + ] + named = set(vector.get("variables", {}).values()) + for name, text in documents.items(): + try: + document = json.loads(text) + except ValueError: + document = None + slots = ( + [items for slot, items in document.items() if slot != "verb"] + if isinstance(document, dict) and isinstance(document.get("verb"), str) + else None + ) + if slots is None or not all(isinstance(items, list) for items in slots): + problems.append( + f"{label}: vector {vector['name']!r} name {name} document is not " + f"a verb and its slots" + ) + continue + named.update( + held + for items in slots + for held in items + if isinstance(held, str) and held.startswith("mv_") + ) + problems.extend( + f"{label}: vector {vector['name']!r} name {name} has no name document" + for name in sorted(named - set(documents)) + ) + return problems + + def _json_type(value: Any) -> str: if isinstance(value, bool): return "boolean" @@ -3818,6 +4162,17 @@ def _check_rules(validator: jsonschema.Draft202012Validator) -> list[str]: f"{label}: vector {name!r} repeats" for name in sorted({n for n in names if names.count(n) > 1}) ) + problems.extend( + f"{label}: vector {vector['name']!r} has a window that does not " + f"start before it ends" + for vector in document["vectors"] + if document["rule"] == "operation-outputs" + and (window := vector["arguments"].get("window")) is not None + and int(window[0]) >= int(window[1]) + ) + if document["rule"] == "output-digest": + for vector in document["vectors"]: + problems.extend(output_digest_problems(label, vector)) if document["rule"] == "reading-order": for vector in document["vectors"]: unfit = _unorderable(vector["members"]) @@ -3916,7 +4271,7 @@ def validate() -> list[str]: or _check_warning_codes( warning_codes, {**taken, "proxy-codes.json": {row["code"] for row in proxy_rows}}, - key_format_reasons(rules_schema), + rules_schema, ) ) @@ -3954,6 +4309,7 @@ def validate() -> list[str]: ) problems.extend(_check_refs(validators["dataset-ref.schema.json"])) problems.extend(_check_rules(validators["rules/rules.schema.json"])) + problems.extend(_check_derived_identities()) return problems diff --git a/tools/validate_descriptions_vectors.py b/tools/validate_descriptions_vectors.py index f2532a4..4200b7f 100644 --- a/tools/validate_descriptions_vectors.py +++ b/tools/validate_descriptions_vectors.py @@ -12,8 +12,9 @@ Checks that the three schemas are valid draft 2020-12, that `operations.json` validates against `operations.schema.json` and names each op once, the nine -analysis operations exactly, each reason once per op, and roles exactly when -an op takes them. Checks every vector against `descriptions-vector.schema.json` +analysis operations exactly, each reason once per op, roles exactly when an +op takes them, each verb once, and a client call's `data` and `other` filling +the two roles of the op they name. Checks every vector against `descriptions-vector.schema.json` and its input against the schema it names: an accepted vector and a vector the structure rules reject must validate, and a vector the schema rejects must not. Then what the record schema cannot express: each file name matches its `id`, @@ -122,6 +123,19 @@ def _check_operations(validator: jsonschema.Draft202012Validator) -> list[str]: f"operations.json: {entry['op']}: roles must be listed exactly when " "input is 'roles'" ) + arguments = entry.get("arguments") + if arguments and sorted((arguments["data"], arguments["other"])) != sorted( + entry["roles"] + ): + problems.append( + f"operations.json: {entry['op']}: arguments' data and other must " + f"fill its two roles {entry['roles']}" + ) + verbs = [entry["verb"] for entry in entries if "verb" in entry] + problems.extend( + f"operations.json: verb {verb!r} is listed twice" + for verb in sorted({v for v in verbs if verbs.count(v) > 1}) + ) return problems