Skip to content

Commit 7efb9db

Browse files
committed
test(conformance): enroll value and query fixtures
1 parent f3638e4 commit 7efb9db

13 files changed

Lines changed: 446 additions & 21 deletions

File tree

‎conformance/stack-encrypt/README.md‎

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,13 +21,31 @@ native Rust against the ciphertexts Go has just produced. It never regenerates
2121
expectations. CI also runs the Go cases on its other supported host platforms.
2222
The same task runs the shared declaration acceptance/rejection corpus in both
2323
languages, with native observations of normalization. It also runs the shared
24-
runtime and read cases described below.
24+
runtime and read cases, and enrolls the existing value-kind and query-operator
25+
fixtures described below. Set no fixture-update variables when checking conformance.
2526

2627
The Rust runner is `packages/stack-encrypt/tests/conformance.rs`; the Go runner
2728
is `languages/golang/encrypt/conformance_test.go`. Go calls the record interface
2829
used by generated code. Testing public SDK declarations is a separate layer.
2930
Python and TypeScript runners are pending; their absence is not a passing result.
3031

32+
## Existing value-kind and query-operator fixtures
33+
34+
The [fixture contract](fixture-contract.md) defines `record-value-kinds-v1` and
35+
`record-query-operators-v1`. Their [enrollment index](v1/fixture-profiles.json)
36+
maps stable requirement IDs to the existing Rust and Go tests. The fixtures
37+
remain under `packages/stack-encrypt/tests/fixtures`; no vectors are copied.
38+
39+
These profiles cover the represented narrow/128-bit integers, dates,
40+
timestamps, decimals, equality/order terms, query operand metadata, match
41+
verification and fetch limits. They preserve typed Rust and generated Go API
42+
checks. The task runs the existing tests directly, and a missing guest fails
43+
when conformance is required. The ordinary CI suites already execute them.
44+
45+
This is representative fixture coverage, not an exhaustive type/operator
46+
matrix. Wider fresh artifact exchange, shared query refusals and EQL/JSON
47+
operations remain separate work.
48+
3149
## `record-reads-v1` contract
3250

3351
The [read contract](read-contract.md) defines cases in `v1/reads.json` for

‎conformance/stack-encrypt/coverage.md‎

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,11 @@ Paths are relative to the repository root. New profile cases run from `v1/`.
66

77
| Requirement | Existing evidence | Remaining work |
88
|---|---|---|
9-
| Scalar value widths and lossless transport | `packages/stack-encrypt/tests/value_kinds.rs`; `languages/golang/encrypt/value_kinds_test.go`; initial shared scalar cases | Expand shared cases to decimals, dates, timestamps, floats, bytes and all widths; Python/TS mappings |
9+
| Scalar value widths and lossless transport | Initial scalar cases and [value-kind requirements](fixture-contract.md#value-kind-requirements), executed by the existing Rust and generated Go runners | Complete per-kind boundary matrices, bytes and broader floats; Python/TS mappings |
1010
| Owned opaque ciphertext and storage envelope | `packages/stack-encrypt/tests/scalar.rs`, `frozen_bytes.rs`; initial cross-opening cases | Blob profile; encoded targets; supported version matrix |
1111
| Record declaration and lowering | [Declaration grammar](declaration-grammar.md); `v1/declarations.json` shared by native Rust and Go; native normalization observations; existing record lowering suites | EQL-resolver profile; SDK-to-declaration mappings; runtime source/read shapes |
1212
| Context binding and composition | `packages/stack-encrypt/tests/target.rs`, `descriptor.rs`, `record_lowering.rs`; `languages/golang/encrypt/context_field_test.go`; initial wrong-context cases | Shared composition vectors; expected-context and required-input matrix |
13-
| Query terms and diagnostics | `packages/stack-encrypt/tests/query_operators.rs`; `languages/golang/encrypt/query_test.go`; initial equality/query-error cases | Common rejection corpus for every operator, capability and input shape |
13+
| Query terms and diagnostics | [Query-operator requirements](fixture-contract.md#query-operator-requirements), existing Rust/Go runners and initial equality/query-error cases | Common rejection corpus for every operator, capability and input shape; wider kind/index combinations |
1414
| Missing, SQL NULL and sealed null | `packages/stack-encrypt/src/dynamic/record.rs` null tests; `languages/golang/encrypt/null_test.go` | Explicit language mappings and common missing/null fixtures |
1515
| Fresh cross-language storage | `.github/workflows/tests-golang.yml` exports fresh Go EQL rows for Rust; initial scalar artifact exchange | Every producer/consumer/backend pair and supported released versions |
1616
| JSON queries, extraction and tampering | `packages/stack-encrypt/tests/json_tree_vectors.rs`; `packages/eql/tests/encryption/tests/json_tree.rs`; `languages/golang/encrypt/json_tree_test.go`, `json_postgres_test.go` | Formal operation profile; JSON vectors consumed by all runners |
@@ -59,17 +59,25 @@ raw-byte profile must cover that boundary explicitly.
5959

6060
## Consolidation policy
6161

62+
The value-kind/query-operator enrollment adds named requirements for the
63+
existing fixtures through [the enrollment index](v1/fixture-profiles.json).
64+
The conformance command selects their original runners; CI guards the mapping
65+
and selection. Typed Rust comparisons, generated Go methods, verification and
66+
fetch-limit checks stay in those runners. No fixture is copied and no existing
67+
assertion is removed. Shared negative cases and fresh exchange of the wider
68+
value kinds remain open.
69+
6270
Shared semantic expectations should be maintained once and executed by each
6371
language runner. Conformance can replace duplicate case tables and fixture
6472
maintenance. It does not eliminate the need to execute the Rust and Go paths.
65-
No existing test has been deleted in these first four slices.
73+
No existing test has been deleted by conformance enrollment.
6674

6775
| Existing evidence | Relationship to conformance | Consolidation action |
6876
|---|---|---|
6977
| Rust `dynamic/record.rs` tests `a_malformed_plan`, `a_null_is_refused_as_its_key`, `a_missing_or_null_index_payload_is_the_default` | Many inputs and diagnostic outcomes are now shared declaration cases | Move the remaining cases and retain any unique display/low-level assertions before removing duplicate rows |
7078
| Rust kind/index admission tests | The corpus has the complete 21-kind/five-index acceptance matrix | Share case data; retain direct tests that compare declaration admission with actual term execution |
71-
| Rust `value_kinds.rs`, Go `value_kinds_test.go` | Already share `value_kinds.json`; cover more types and generated SDK code than the initial scalar profile | Enroll that existing fixture under named requirements instead of copying it; keep generated-code assertions |
72-
| Rust `query_operators.rs`, Go `query_test.go` | Already share operator vectors, including verification and target limits | Reuse their fixture and cases when adding an operator profile; preserve host/generated query mapping tests |
79+
| Rust `value_kinds.rs`, Go `value_kinds_test.go` | Enrolled as `record-value-kinds-v1`, retaining the shared fixture and generated SDK calls | Extend the existing fixture/requirements for missing boundaries; retain typed and generated-code assertions |
80+
| Rust `query_operators.rs`, Go `query_test.go` | Enrolled as `record-query-operators-v1`, including verification and target limits | Add shared refusal cases separately; retain host/generated query mapping tests |
7381
| Go checker and host validation tests in `guest_test.go` | Exercise host rejection and diagnostic forwarding through public checker methods | Keep adapter-specific assertions; representable declaration cases also use `Checker.Check`, while raw frames observe the guest boundary |
7482
| Native scalar/record round trips and cross-opening | Some behavior overlaps, but existing tests cover more types and API paths | Replace only exact duplicates after matching requirements and execution paths |
7583

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
# Value-kind and query-operator profiles
2+
3+
`record-value-kinds-v1` and `record-query-operators-v1` enroll the existing
4+
Rust/Go fixtures in the conformance contract. They are experimental profiles
5+
of the cases those fixtures contain, not exhaustive type/operator certification.
6+
They use the plain record resolver (`NoTargets`), with public deterministic
7+
keys and no database or live credentials.
8+
9+
The fixtures remain at their existing paths:
10+
11+
- [`value_kinds.json`](../../packages/stack-encrypt/tests/fixtures/value_kinds.json)
12+
- [`query_operators.json`](../../packages/stack-encrypt/tests/fixtures/query_operators.json)
13+
14+
Their [existing schema documentation](../../packages/stack-encrypt/tests/fixtures/README.md)
15+
continues to own the formats. No expected frames, ciphertexts, terms, verified
16+
pages or fetch limits are copied into this directory.
17+
18+
[`v1/fixture-profiles.json`](v1/fixture-profiles.json) maps stable requirement IDs
19+
to the existing Rust test functions and Go tests that provide their evidence.
20+
It is an enrollment index, not another input corpus or a runtime capability
21+
negotiation format. The CI wiring test checks that its fixtures/tests exist,
22+
that each requirement is documented here, and that the conformance command
23+
selects every listed test. Both language implementations are mandatory for
24+
these profiles; Python and TypeScript remain pending.
25+
26+
## Value-kind requirements
27+
28+
The fixture represents int8, int16, uint8, uint16, int128, uint128, date,
29+
timestamp and decimal values. It includes the signed 128-bit minimum and
30+
unsigned 128-bit maximum. These are representative cases, not every boundary
31+
of every type. Values use the fixture's `type` and `text` with a frozen `frame`;
32+
128-bit integers must never pass through a floating-point JSON number.
33+
34+
| Requirement ID | Required observation | Fixture evidence |
35+
|---|---|---|
36+
| `values.typed-frames` | The actual host value encodes to the exact frozen typed frame, preserving integer width, decimal scale and timestamp nanoseconds. | Every `values` entry |
37+
| `values.frozen-open` | The committed Rust record opens to those values and types, retaining decimal scale and timestamp precision. | `record` and `values` |
38+
| `values.stored-terms` | Fresh records round-trip and store the frozen equality/order terms. Queries for their values derive the corresponding stored terms. Randomized ciphertext bytes are not compared. | Every non-`c` output in `record`, under `plan` |
39+
| `values.normalized-queries` | Decimal queries differing only in trailing zeros and timestamp queries truncated to microseconds derive the stored equality and order terms, while the opened values retain their original precision. | Every `queries` entry and its field's order output |
40+
41+
Rust additionally compares dynamic execution with typed term derivation and
42+
retains its wrong-width write refusal test. Go runs the real generated
43+
`testusers.ValueKinds` encrypt/decrypt and field-query methods, including its
44+
field-context substitution refusal. These assertions stay in their owning
45+
tests. The separate Go invalid-date/decimal/timestamp tests remain in the full
46+
Go suite; they are not claimed as shared fixture requirements here.
47+
48+
## Query-operator requirements
49+
50+
The fixture covers equality on text and uint32, scalar/range ordering on uint32
51+
and float64, and probabilistic text matching. Its operators are `eq`, `lt`,
52+
`lte`, `gt`, `gte`, `between` and `matches`, in the combinations declared by
53+
the fixture. It does not claim every operator for every kind or index.
54+
55+
| Requirement ID | Required observation | Fixture evidence |
56+
|---|---|---|
57+
| `queries.term-contract` | Derive the frozen operand bytes and metadata: operator, fidelity, index, domain, and scalar versus ordered low/high range shape. | Every `queries` entry |
58+
| `queries.match-verification` | Verify the complete page in input order, including matches, nonmatches and null; results agree with the frozen boolean vector. | `verify` |
59+
| `queries.fetch-limits` | Given the frozen match term, requested count and statistics, return the recorded bounded row count or explicit unbounded result. | Every `target_limit` entry |
60+
61+
Rust compares the dynamic path with typed query chains, `MatchQuery::verify`
62+
and typed fetch sizing, and checks that the fixture contains both verification
63+
outcomes and both bounded/unbounded limits. Go exercises the record interface
64+
and generated query methods. Generated string-page verification omits null,
65+
which the record-interface test covers. Its existing SQL parameter conversion,
66+
empty-page, zero-query, inverted-range and floating-point ordering assertions
67+
remain in those selected tests. Wider operator refusal and secret-diagnostic
68+
tests remain in the full suites; this slice does not turn them into shared
69+
negative cases.
70+
71+
## Running and maintaining the profiles
72+
73+
`mise run test:conformance` selects the existing `value_kinds` and
74+
`query_operators` Rust binaries and the Go tests listed in the enrollment index,
75+
alongside the earlier profiles. Each existing test runs once per invocation;
76+
no wrapper test invokes another test a second time. The ordinary Rust and Go
77+
CI suites already run these tests, so enrollment adds no duplicate CI suite.
78+
79+
When `STACK_CONFORMANCE_REQUIRED=1`, a missing deterministic guest fails the
80+
enrolled Go tests instead of skipping them. The aggregate command rejects both
81+
`STACK_CONFORMANCE_UPDATE` and `STACK_ENCRYPT_UPDATE_FIXTURES`, even when set to
82+
an empty value. The enrolled Rust fixture writers independently reject updates
83+
in required mode. Deliberate fixture maintenance remains available outside
84+
required mode through the existing documented regeneration commands.
85+
86+
Keep requirement IDs stable. Add a new profile version for incompatible
87+
expectations, and review fixture regeneration explicitly. The enrollment
88+
index's `version` is its metadata format, not a production storage version.
89+
90+
This slice consolidates the coverage inventory and execution selection, not
91+
the underlying test assertions. No test or frozen artifact is removed. Fresh
92+
cross-language artifact exchange remains the scalar profile's guarantee;
93+
these wider fixtures prove frozen Rust-to-Go opening, local round trips and
94+
term agreement, not fresh Go-to-Rust opening of every represented kind.
95+
96+
Python and TypeScript should consume these same fixture paths and implement
97+
the listed observations through their own host values and public APIs. Their
98+
integer/decimal/time mappings and actual backends must be specified by those
99+
implementations. EQL domains, JSON operations, all-type boundary matrices,
100+
shared query refusals and supported-release matrices remain separate work.
Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
{
2+
"version": 1,
3+
"profiles": [
4+
{
5+
"id": "record-value-kinds-v1",
6+
"fixture": "packages/stack-encrypt/tests/fixtures/value_kinds.json",
7+
"rust_binary": "value_kinds",
8+
"go_source": "languages/golang/encrypt/value_kinds_test.go",
9+
"requirements": [
10+
{
11+
"id": "values.typed-frames",
12+
"rust_tests": [
13+
"the_committed_fixture_opens_and_its_terms_are_todays"
14+
],
15+
"go_tests": ["TestGeneratedCodeOpensTheRustValueKindsFixture"]
16+
},
17+
{
18+
"id": "values.frozen-open",
19+
"rust_tests": [
20+
"the_committed_fixture_opens_and_its_terms_are_todays"
21+
],
22+
"go_tests": ["TestGeneratedCodeOpensTheRustValueKindsFixture"]
23+
},
24+
{
25+
"id": "values.stored-terms",
26+
"rust_tests": [
27+
"every_new_kind_seals_opens_and_queries_as_the_typed_path",
28+
"the_committed_fixture_opens_and_its_terms_are_todays"
29+
],
30+
"go_tests": ["TestGeneratedCodeOpensTheRustValueKindsFixture"]
31+
},
32+
{
33+
"id": "values.normalized-queries",
34+
"rust_tests": [
35+
"every_new_kind_seals_opens_and_queries_as_the_typed_path",
36+
"the_committed_fixture_opens_and_its_terms_are_todays"
37+
],
38+
"go_tests": ["TestGeneratedCodeOpensTheRustValueKindsFixture"]
39+
}
40+
]
41+
},
42+
{
43+
"id": "record-query-operators-v1",
44+
"fixture": "packages/stack-encrypt/tests/fixtures/query_operators.json",
45+
"rust_binary": "query_operators",
46+
"go_source": "languages/golang/encrypt/query_test.go",
47+
"requirements": [
48+
{
49+
"id": "queries.term-contract",
50+
"rust_tests": [
51+
"the_committed_query_fixture_is_what_the_engine_derives"
52+
],
53+
"go_tests": [
54+
"TestQueryTermsAreTheRustFixtures",
55+
"TestGeneratedQueriesAreTheRustFixtures"
56+
]
57+
},
58+
{
59+
"id": "queries.match-verification",
60+
"rust_tests": [
61+
"the_committed_query_fixture_is_what_the_engine_derives",
62+
"the_fixture_pins_both_outcomes"
63+
],
64+
"go_tests": [
65+
"TestVerifyIsTheRustFixtures",
66+
"TestAMatchesQueryVerifiesAndSizesAsTheRustFixtures"
67+
]
68+
},
69+
{
70+
"id": "queries.fetch-limits",
71+
"rust_tests": [
72+
"the_committed_query_fixture_is_what_the_engine_derives",
73+
"the_fixture_pins_both_outcomes"
74+
],
75+
"go_tests": [
76+
"TestTargetLimitIsTheRustFixtures",
77+
"TestAMatchesQueryVerifiesAndSizesAsTheRustFixtures"
78+
]
79+
}
80+
]
81+
}
82+
]
83+
}

‎docs/plans/2026-10-31-stack-encrypt-conformance.md‎

Lines changed: 29 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Stack Encrypt cross-language conformance
22

3-
Date: 2026-10-31. Status: scalar, declaration, initial runtime and read slices implemented; the complete contract is not frozen.
3+
Date: 2026-10-31. Status: scalar, declaration, initial runtime and read slices implemented; existing value-kind and query-operator fixtures enrolled. The complete contract is not frozen.
44

55
This work is stacked on [PR #1404](https://github.com/cipherstash/stack/pull/1404),
66
at `b684a217e2da629c762efa70fac958ec7e21646a`. It follows the accepted
@@ -145,11 +145,37 @@ Local validation passes for all four profiles, fresh scalar
145145
artifact exchange, Rust clippy/formatting, Go lint, Biome and the CI wiring checks.
146146
The deterministic guest is reused from the unchanged parent engine.
147147

148-
Next, enroll the existing wider value-kind and operator fixtures under named
149-
requirements, then extend context composition and EQL/JSON operation profiles.
148+
The fixture checkpoint below enrolls the existing wider value-kind and operator
149+
fixtures. Next, extend context composition and EQL/JSON operation profiles.
150150
Repeated-key and malformed-leaf cases need an explicit raw-byte transport path.
151151
Python and TypeScript consumers remain pending.
152152

153+
## Value-kind and query-operator checkpoint
154+
155+
The next slice is on `feat/stack-encrypt-value-query-conformance`, stacked on
156+
[PR #1462](https://github.com/cipherstash/stack/pull/1462). The
157+
[fixture contract](../../conformance/stack-encrypt/fixture-contract.md) and
158+
[enrollment index](../../conformance/stack-encrypt/v1/fixture-profiles.json)
159+
assign stable requirement IDs to the existing `value_kinds.json` and
160+
`query_operators.json` evidence. The conformance task directly runs their
161+
existing Rust and Go tests. Typed Rust comparisons, generated Go methods,
162+
match verification and fetch sizing remain intact.
163+
164+
The fixture files stay in place with no duplicated expectations. CI guards
165+
the requirement-to-test mapping and test selection. Required Go runs fail if
166+
the guest is missing; required checks cannot regenerate frozen fixtures.
167+
This is consolidation of inventory and execution, not removal of assertions.
168+
169+
Local validation passes for the expanded conformance command, fresh scalar
170+
artifact exchange, missing-guest regressions, Rust clippy/formatting, Go lint,
171+
Biome and CI wiring checks. Both enrolled Rust writers refuse regeneration in
172+
required mode, and the frozen fixture files remain unchanged.
173+
174+
The coverage is representative, not a complete kind/operator boundary matrix.
175+
Shared operator refusals, wider fresh artifact exchange, Python/TypeScript
176+
consumers and version matrices remain open. The next implementation slice is
177+
shared context-composition vectors, followed by EQL/JSON operation profiles.
178+
153179
## Follow-up slices
154180

155181
- Extend the declaration grammar to EQL resolver capabilities and the operation

0 commit comments

Comments
 (0)