Skip to content

Commit 5e0ef66

Browse files
committed
fix(conformance): align runtime observations
Compare stored lengths after SQL-null omission, preserve structured source diagnostics at the Go boundary, and share output and refusal assertions. Refs #1461
1 parent 04eeef6 commit 5e0ef66

11 files changed

Lines changed: 127 additions & 139 deletions

File tree

‎conformance/stack-encrypt/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ Python and TypeScript runners are pending; their absence is not a passing result
2929

3030
## `record-runtime-v1` contract
3131

32-
The [runtime contract](runtime-contract.md) defines 37 cases in `v1/runtime.json`:
32+
The [runtime contract](runtime-contract.md) defines cases in `v1/runtime.json`:
3333
missing and explicit null values, SQL-null omission, sealed-null storage and
3434
equality, typed batch round trips, write validation and lossless query conversions.
3535
The runners are `packages/stack-encrypt/tests/runtime_conformance.rs` and

‎conformance/stack-encrypt/coverage.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ absence is a delivery gap, not a test skip that counts toward certification.
2525

2626
## Runtime checkpoint
2727

28-
`v1/runtime.json` adds 37 shared cases for typed writes and reads, null storage,
28+
`v1/runtime.json` adds shared cases for typed writes and reads, null storage,
2929
batches, equality queries and lossless numeric conversion. See the
3030
[runtime contract](runtime-contract.md) for exact observations and limits.
3131
Native data-key counters cover zero-call preflight failures and single-call

‎conformance/stack-encrypt/runtime-contract.md‎

Lines changed: 12 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,9 @@ Each case uses one field, `value`, with ciphertext (`c`) and equality (`eq`)
88
outputs, under context `["conformance", "value"]`. `kind` declares its type;
99
`null` is empty for a required value, `sql`, or `sealed`. Values use the scalar
1010
profile's `{type, text}` representation, extended with `float64` decimal text.
11-
Runners construct typed values before calling the engine.
11+
Runners construct typed values before calling the engine. This profile admits
12+
`int32`, `int64`, `uint64`, `float64`, `bool`, and `string` as declared kinds;
13+
both runners reject unknown kinds and null-policy spellings.
1214

1315
## Write and read
1416

@@ -55,11 +57,14 @@ A null equality query succeeds only for sealed-null fields.
5557
Native Rust must reject before producing a pending operation. Go must expose
5658
`ErrEncoding` and the same diagnostic when the request reaches the guest.
5759

58-
Missing and unknown fields are rejected earlier by Go's record adapter.
59-
Those cases additionally specify `go_message`, an exact adapter regression
60-
observation, and require no engine diagnostic. This prose is not a portable
61-
engine error contract. A future SDK should report equivalent semantics at its
62-
earliest supported stage, without adopting Go's wording.
60+
Missing and unknown fields can be rejected earlier by Go's record adapter.
61+
The adapter preserves the same structured source diagnostic (code, reason and
62+
field); the corpus does not require a particular English message or rejection
63+
at the host boundary. Forwarding to the guest later remains conformant.
64+
65+
Each refused source row contains exactly one independent fault. The corpus does
66+
not specify precedence between multiple faults or source-object traversal order.
67+
Batch cases may place that single invalid row after valid rows to test preflight.
6368

6469
The native runner instruments data-key generation and retrieval calls. A
6570
successful batch makes one of each when it stores ciphertext, and zero when
@@ -71,7 +76,7 @@ guest's lack of network requests is not proof of logical key-call counts.
7176

7277
## Scope and remaining work
7378

74-
The 37 cases cover runtime writes, corresponding reads, and equality queries.
79+
The cases cover runtime writes, corresponding reads, and equality queries.
7580
They do not cover malformed stored records, authenticated type mismatches on
7681
read, context composition/preflight, provider failures, or other operators.
7782
Fresh cross-language artifact exchange remains the scalar profile's coverage.

‎conformance/stack-encrypt/v1/runtime.json‎

Lines changed: 7 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -17,8 +17,7 @@
1717
"expected_error": {
1818
"code": "stack_encrypt::dynamic_source",
1919
"reason": "field_missing",
20-
"field": "value",
21-
"go_message": "cipherstash: malformed input: row 0: no value for field \"value\""
20+
"field": "value"
2221
}
2322
},
2423
{
@@ -200,8 +199,7 @@
200199
"expected_error": {
201200
"code": "stack_encrypt::dynamic_source",
202201
"reason": "field_missing",
203-
"field": "value",
204-
"go_message": "cipherstash: malformed input: row 0: no value for field \"value\""
202+
"field": "value"
205203
}
206204
},
207205
{
@@ -457,8 +455,7 @@
457455
"expected_error": {
458456
"code": "stack_encrypt::dynamic_source",
459457
"reason": "field_missing",
460-
"field": "value",
461-
"go_message": "cipherstash: malformed input: row 1: no value for field \"value\""
458+
"field": "value"
462459
}
463460
},
464461
{
@@ -481,8 +478,7 @@
481478
"expected_error": {
482479
"code": "stack_encrypt::dynamic_source",
483480
"reason": "unknown_field",
484-
"field": "extra",
485-
"go_message": "cipherstash: malformed input: row 0: the plan has no field \"extra\""
481+
"field": "extra"
486482
}
487483
},
488484
{
@@ -811,6 +807,8 @@
811807
"type": "null",
812808
"text": ""
813809
}
814-
}
810+
},
811+
{"id": "sql-mixed-equal-lengths", "operation": "seal", "kind": "int32", "null": "sql", "rows": [{}, {"value": {"type": "int32", "text": "1"}}, {"value": {"type": "int32", "text": "2"}}], "expected_rows": [{"value": {"type": "null", "text": ""}}, {"value": {"type": "int32", "text": "1"}}, {"value": {"type": "int32", "text": "2"}}], "expected_stored": [false, true, true], "equal_lengths": true},
812+
{"id": "sql-empty-equal-lengths", "operation": "seal", "kind": "int32", "null": "sql", "rows": [], "expected_rows": [], "expected_stored": [], "equal_lengths": true}
815813
]
816814
}

‎docs/plans/2026-10-31-stack-encrypt-conformance.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -110,15 +110,15 @@ remain. No existing tests have been removed.
110110
## Runtime checkpoint
111111

112112
The initial runtime slice is on `feat/stack-encrypt-runtime-conformance`, stacked
113-
on draft [PR #1460](https://github.com/cipherstash/stack/pull/1460). Its 37 shared
113+
on draft [PR #1460](https://github.com/cipherstash/stack/pull/1460). Its shared
114114
cases cover missing/null storage, typed batch round trips, source refusals, and
115115
lossless equality query conversions. Native Rust observes data-key call counts.
116116
Go explicitly records host validation where it precedes guest diagnostics.
117117
The [runtime contract](../../conformance/stack-encrypt/runtime-contract.md)
118118
describes the corpus and limits. No production API or existing test is removed.
119119

120-
Local validation passes for all three profiles (249 shared cases), fresh scalar
121-
ciphertext exchange, Rust clippy/formatting, Go lint, Biome and 34 CI wiring
120+
Local validation passes for all three profiles, fresh scalar
121+
ciphertext exchange, Rust clippy/formatting, Go lint, Biome and the CI wiring
122122
checks. The deterministic guest is reused from the unchanged parent engine.
123123

124124
The next runtime extension should cover malformed stored records, authenticated

‎languages/golang/encrypt/conformance_test.go‎

Lines changed: 12 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -177,13 +177,7 @@ func TestStackConformance(t *testing.T) {
177177
input := conformanceValue(t, c.Input)
178178
query, err := cipher.QueryTerm(ctx, plan, corpus.Field, "eq", input)
179179
if c.ExpectedError != nil {
180-
var diagnostic *encrypt.Diagnostic
181-
if !errors.Is(err, encrypt.ErrEncoding) || !errors.As(err, &diagnostic) {
182-
t.Fatalf("expected encoding diagnostic, got %v", err)
183-
}
184-
if diagnostic.Code != c.ExpectedError.Code || diagnostic.Reason() != c.ExpectedError.Reason || diagnostic.Field() != corpus.Field {
185-
t.Fatalf("unexpected diagnostic: %s reason=%s field=%s", diagnostic.Code, diagnostic.Reason(), diagnostic.Field())
186-
}
180+
assertConformanceDiagnostic(t, err, c.ExpectedError.Code, c.ExpectedError.Reason, corpus.Field)
187181
return
188182
}
189183
if c.Kind != c.Input.Type {
@@ -247,3 +241,14 @@ func TestStackConformance(t *testing.T) {
247241
}
248242
}
249243
}
244+
245+
func assertConformanceDiagnostic(t *testing.T, err error, code, reason, field string) {
246+
t.Helper()
247+
var diagnostic *encrypt.Diagnostic
248+
if !errors.Is(err, encrypt.ErrEncoding) || !errors.As(err, &diagnostic) {
249+
t.Fatalf("expected encoding diagnostic, got %v", err)
250+
}
251+
if diagnostic.Code != code || diagnostic.Reason() != reason || diagnostic.Field() != field {
252+
t.Fatalf("unexpected diagnostic: %s reason=%s field=%s", diagnostic.Code, diagnostic.Reason(), diagnostic.Field())
253+
}
254+
}

‎languages/golang/encrypt/records.go‎

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ func (cph *Cipher) Seal(ctx context.Context, plan *record.Plan, rows []record.So
3939
for i, row := range rows {
4040
source[i], err = sourceRow(p, row)
4141
if err != nil {
42-
return nil, fmt.Errorf("%w: row %d: %v", ErrEncoding, i, err)
42+
return nil, fmt.Errorf("%w: row %d: %w", ErrEncoding, i, err)
4343
}
4444
if cph.context != "" {
4545
// The cipher names the context every row is under: a value
@@ -235,7 +235,7 @@ func sourceRow(p *record.Plan, row record.Source) (vcvalue.Object, error) {
235235
if len(row) != expected {
236236
for name := range row {
237237
if p.Field(name) == nil && name != p.ContextField {
238-
return nil, fmt.Errorf("the plan has no field %q", name)
238+
return nil, sourceDiagnostic("unknown_field", name, fmt.Sprintf("the plan has no field %q", name))
239239
}
240240
}
241241
}
@@ -257,7 +257,7 @@ func sourceRow(p *record.Plan, row record.Source) (vcvalue.Object, error) {
257257
continue
258258
}
259259
if !ok {
260-
return nil, fmt.Errorf("no value for field %q", f.Name)
260+
return nil, sourceDiagnostic("field_missing", f.Name, fmt.Sprintf("no value for field %q", f.Name))
261261
}
262262
out = append(out, vcvalue.Field{Key: f.Name, Value: v})
263263
}
@@ -507,3 +507,10 @@ func indexWord(o record.Output) string {
507507
}
508508
return string(o)
509509
}
510+
511+
// The adapter reports the same source semantics as the engine, even when it
512+
// can refuse the row before encoding it. Seal preserves this diagnostic chain.
513+
func sourceDiagnostic(reason, field, message string) error {
514+
return &Diagnostic{Code: "stack_encrypt::dynamic_source", Message: message,
515+
Fields: map[string]any{"reason": reason, "field": field}}
516+
}

‎languages/golang/encrypt/runtime_conformance_test.go‎

Lines changed: 11 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -25,10 +25,9 @@ type runtimeCase struct {
2525
Input *conformanceInput `json:"input"`
2626
ExpectedValue *conformanceInput `json:"expected_value"`
2727
ExpectedError *struct {
28-
GoMessage string `json:"go_message"`
29-
Code string `json:"code"`
30-
Reason string `json:"reason"`
31-
Field string `json:"field"`
28+
Code string `json:"code"`
29+
Reason string `json:"reason"`
30+
Field string `json:"field"`
3231
} `json:"expected_error"`
3332
}
3433

@@ -44,24 +43,6 @@ func runtimeRows(t *testing.T, input []map[string]conformanceInput) []record.Sou
4443
return rows
4544
}
4645

47-
func runtimeRefusal(t *testing.T, err error, c runtimeCase) {
48-
t.Helper()
49-
var diagnostic *encrypt.Diagnostic
50-
if c.ExpectedError.GoMessage != "" {
51-
if !errors.Is(err, encrypt.ErrEncoding) || errors.As(err, &diagnostic) || err.Error() != c.ExpectedError.GoMessage {
52-
t.Fatalf("unexpected host refusal: %v", err)
53-
}
54-
return
55-
}
56-
if !errors.Is(err, encrypt.ErrEncoding) || !errors.As(err, &diagnostic) {
57-
t.Fatalf("expected encoding diagnostic, got %v", err)
58-
}
59-
want := c.ExpectedError
60-
if diagnostic.Code != want.Code || diagnostic.Reason() != want.Reason || diagnostic.Field() != want.Field {
61-
t.Fatalf("unexpected diagnostic: %s reason=%s field=%s", diagnostic.Code, diagnostic.Reason(), diagnostic.Field())
62-
}
63-
}
64-
6546
func TestRuntimeConformance(t *testing.T) {
6647
var corpus struct {
6748
Version int `json:"version"`
@@ -93,6 +74,12 @@ func TestRuntimeConformance(t *testing.T) {
9374
}
9475
seen[c.ID] = true
9576
t.Run(c.ID, func(t *testing.T) {
77+
if c.Null != "" && c.Null != "sql" && c.Null != "sealed" {
78+
t.Fatalf("unknown null policy %q", c.Null)
79+
}
80+
if c.Kind != "int32" && c.Kind != "int64" && c.Kind != "uint64" && c.Kind != "float64" && c.Kind != "bool" && c.Kind != "string" {
81+
t.Fatalf("unsupported runtime kind %q", c.Kind)
82+
}
9683
plan := &record.Plan{Context: []string{"conformance"}, Fields: []record.Field{{Name: "value", Kind: record.Kind(c.Kind), Null: record.Null(c.Null), Outputs: []record.Output{record.Ciphertext, record.Equality}}}}
9784
switch c.Operation {
9885
case "seal":
@@ -104,7 +91,7 @@ func TestRuntimeConformance(t *testing.T) {
10491
if c.ExpectedRows != nil || c.ExpectedStored != nil || c.EqualLengths {
10592
t.Fatal("ambiguous seal outcome")
10693
}
107-
runtimeRefusal(t, err, c)
94+
assertConformanceDiagnostic(t, err, c.ExpectedError.Code, c.ExpectedError.Reason, c.ExpectedError.Field)
10895
return
10996
}
11097
if err != nil {
@@ -165,7 +152,7 @@ func TestRuntimeConformance(t *testing.T) {
165152
if c.ExpectedValue != nil {
166153
t.Fatal("ambiguous query outcome")
167154
}
168-
runtimeRefusal(t, err, c)
155+
assertConformanceDiagnostic(t, err, c.ExpectedError.Code, c.ExpectedError.Reason, c.ExpectedError.Field)
169156
return
170157
}
171158
if err != nil {

‎packages/stack-encrypt/tests/common/conformance.rs‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,3 +27,39 @@ pub fn value(input: &Input) -> ValueFrame {
2727
other => panic!("unsupported input kind {other}"),
2828
}
2929
}
30+
31+
/// Inspect actual stored output, shared by scalar and runtime profiles.
32+
pub fn stored_outputs(
33+
tree: &stack_encrypt::StackCipherText,
34+
field: &str,
35+
) -> Option<(Vec<u8>, usize)> {
36+
use stack_encrypt::CipherText;
37+
let CipherText::Map(fields) = tree else {
38+
panic!("expected record")
39+
};
40+
assert!(fields.len() <= 1 && fields.iter().all(|(name, _)| name == field));
41+
let (_, node) = fields.iter().find(|(name, _)| name == field)?;
42+
let CipherText::Map(outputs) = node else {
43+
panic!("expected outputs")
44+
};
45+
assert_eq!(outputs.len(), 2);
46+
let CipherText::Single(leaf) = &outputs.iter().find(|(name, _)| name == "c").unwrap().1 else {
47+
panic!("expected ciphertext")
48+
};
49+
let CipherText::Passthrough(term) = &outputs.iter().find(|(name, _)| name == "eq").unwrap().1
50+
else {
51+
panic!("expected equality bytes")
52+
};
53+
Some((
54+
vitaminc_aead_frame::serde::from_slice::<&[u8]>(&term.0)
55+
.unwrap()
56+
.to_vec(),
57+
leaf.to_bytes().len(),
58+
))
59+
}
60+
pub fn refusal(error: &stack_encrypt::dynamic::Error, code: &str, reason: &str, field: &str) {
61+
use miette::Diagnostic;
62+
assert_eq!(error.code().unwrap().to_string(), code);
63+
assert_eq!(error.reason().unwrap().as_str(), reason);
64+
assert_eq!(error.field(), Some(field));
65+
}

‎packages/stack-encrypt/tests/conformance.rs‎

Lines changed: 3 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,9 @@ use std::collections::{BTreeMap, BTreeSet};
77
use std::path::PathBuf;
88
use std::sync::atomic::Ordering;
99

10-
use common::conformance::{value, Input};
10+
use common::conformance::{refusal, stored_outputs, value, Input};
1111
use common::frames::{frame, object};
1212
use common::{hex, unhex};
13-
use miette::Diagnostic;
1413
use serde::{Deserialize, Serialize};
1514
use serde_json::json;
1615
use stack_encrypt::dynamic::{
@@ -72,7 +71,6 @@ fn root() -> PathBuf {
7271
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../conformance/stack-encrypt/v1")
7372
}
7473

75-
7674
fn plan(corpus: &Corpus, case: &Case) -> record::Plan {
7775
let mut context = corpus.context.clone();
7876
context.push(corpus.field.clone());
@@ -98,23 +96,6 @@ fn stored(corpus: &Corpus, artifact: &Artifact) -> StackCipherText {
9896
)])
9997
}
10098

101-
fn stored_equality(tree: &StackCipherText, field: &str) -> Vec<u8> {
102-
let CipherText::Map(fields) = tree else {
103-
panic!("expected record")
104-
};
105-
let (_, CipherText::Map(outputs)) = fields.iter().find(|(name, _)| name == field).unwrap()
106-
else {
107-
panic!("expected outputs")
108-
};
109-
let (_, CipherText::Passthrough(term)) = outputs.iter().find(|(name, _)| name == "eq").unwrap()
110-
else {
111-
panic!("expected equality bytes")
112-
};
113-
vitaminc_aead_frame::serde::from_slice::<&[u8]>(&term.0)
114-
.unwrap()
115-
.to_vec()
116-
}
117-
11899
// Exercise the owned scalar API independently of dynamic record lowering.
119100
async fn scalar(
120101
keyset: &KeysetCipher<'_, common::DeterministicSource>,
@@ -243,19 +224,7 @@ async fn shared_record_scalar_profile() {
243224
Err(error) => error,
244225
Ok(_) => panic!("{}: expected preflight rejection", case.id),
245226
};
246-
assert_eq!(
247-
error.code().unwrap().to_string(),
248-
expected.code,
249-
"{}",
250-
case.id
251-
);
252-
assert_eq!(
253-
error.reason().unwrap().as_str(),
254-
expected.reason,
255-
"{}",
256-
case.id
257-
);
258-
assert_eq!(error.field(), Some(corpus.field.as_str()));
227+
refusal(&error, &expected.code, &expected.reason, &corpus.field);
259228
assert_eq!(generates.load(Ordering::SeqCst), 0);
260229
assert_eq!(retrieves.load(Ordering::SeqCst), 0);
261230
continue;
@@ -267,7 +236,7 @@ async fn shared_record_scalar_profile() {
267236
.unwrap()
268237
.await
269238
.unwrap();
270-
let stored_term = stored_equality(&fresh, &corpus.field);
239+
let stored_term = stored_outputs(&fresh, &corpus.field).unwrap().0;
271240
let opened = record::decrypt(Scope::Keyset(keyset.clone()), fresh, &plan, None)
272241
.unwrap()
273242
.await

0 commit comments

Comments
 (0)