Skip to content

Commit 67bbb82

Browse files
committed
Merge feat/go-opaque-storage-profile into feat/json-tree-data-plan
Brings in next's JSON tree cascade and random storage order (#1379). Conflicts resolved to #1379's names and wording, keeping this branch's data plan target, its JSON text parser and the JsonSearch rename: JsonProjection is JsonSelectorSet in the fuzz table and the JsonQuery docs, the leakage page takes #1379's opening under the eql_v4_json_search name, json_tree_path_query's comment says stored order under the json_entry domain, and the oracle's hostile-row updates find each entry's slot (the order is random) and cast to eql_v4_json_search. The changelog keeps both sides' entries. Claude-Session: https://claude.ai/code/session_01N7bCqa3ZVJMcEE99RzVqEu
2 parents 3513753 + 0bccc05 commit 67bbb82

52 files changed

Lines changed: 5588 additions & 1564 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"@cipherstash/eql": patch
3+
---
4+
5+
EQL v4 `json_tree` (unreleased): a stored document's entries are now in a random order, root first, an object's members get a random permutation as their ordinals (the `o` the server probes a node's children at, which was a keyed rank), and its containment projection `q` is shuffled the same way, so the stored order no longer shows each row's tree, the size of every subtree or the byte order of sibling key names (#1357). A containment or equality needle is sent in a random order of its own too, so a query no longer shows sibling key order either. Every entry's ciphertext now also commits to its children's AEAD tags, so the client checks a subtree read's rows by the chain rather than by their positions, and a row swapped, dropped, added, reordered or moved beneath the node read is refused by the cipher. Nothing stored changes shape; `json_tree_children` returns a node's children in ordinal order (an array's elements by position), where it returned them by stored slot, and no other SQL function changes (three doc comments stop promising document order); documents sealed by the earlier build of this unreleased family no longer open. `->`, `->>` and an ordering on `eql_v4.ord_term(col -> sel)` at a wildcard path now return an unspecified element of the row (the same one every time), not element 0; read every element with `json_tree_path_query`. `->` on an object or array with members no longer opens on the client (the entry's ciphertext commits to its children, which `->` does not return): Stack Encrypt reports `JsonNotALeaf`, which is also what an altered entry looks like through `->`; read a container with a subtree read. `docs/reference/json-tree-leakage.md` says what the stored row still shows and what a read now proves.

‎Cargo.lock‎

Lines changed: 3 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎docs/fuzzing.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ Current targets:
2929
| `fuzz:check-record` | `stack-encrypt` | `check_record` | `dynamic::record::check_record` over an `Arbitrary`-derived plan and tree, against a model of the record rules (structure-aware) |
3030
| `fuzz:plan-build` | `stack-encrypt` | `plan_build` | `plan::FieldsBuilder::build` over `Arbitrary`-derived untrusted names (context, fields, identities); a built field's label must render and parse back, and passthrough names are never refused (structure-aware) |
3131
| `fuzz:json-text` | `stack-encrypt` | `json_text` | `json::parse_text`, the boundary parser for a language SDK's JSON document, against serde_json: it reads what serde_json reads to the same value, refuses beyond it only repeated members and out-of-range integers, and what it reads writes back to itself |
32-
| `fuzz:json-wire` | `stack-encrypt` | `json_wire_decode` | the JSON index's wire decoders (`JsonDocument`, `JsonNode`, `JsonProjection`, `JsonLeafQuery` from JSON text; `JsonHeader::from_bytes`), over base64 tags and base85 ciphertexts; an accepted document or node must round-trip through its wire form |
32+
| `fuzz:json-wire` | `stack-encrypt` | `json_wire_decode` | the JSON index's wire decoders (`JsonDocument`, `JsonNode`, `JsonSelectorSet`, `JsonLeafQuery` from JSON text; `JsonHeader::from_bytes`), over base64 tags and base85 ciphertexts; an accepted document or node must round-trip through its wire form |
3333

3434
Each target is a few lines — `libfuzzer-sys` hands a `&str` (or `&[u8]`
3535
for the byte decoders) to the parser via the `arbitrary` crate:

‎docs/plans/2026-10-17-json-tree-index.md‎

Lines changed: 24 additions & 2 deletions
Large diffs are not rendered by default.

‎docs/plans/2026-10-25-json-tree-leakage-before-and-after.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -115,7 +115,7 @@ The "before" column is the profile at `9a7ca372c`; the "after" column is the por
115115
| `selector_precedence` | kept | which of two selectors comes first across rows: sibling key order |
116116
| `depth_recovery` | kept | an attack that reconstructs every row's depth sequence from the stored order alone |
117117
| `token_reach` | kept | per read, row and token: the node it finds and everything its walk links, with parents and depths |
118-
| `ordinal_patterns` | kept | elements' ordinals in position order; members ranked consistently under one token |
118+
| `ordinal_patterns` | replaced by `member_order_shared_across_rows` | whether one token's objects give their members one order of ordinals across rows, reported only below a `2⁻⁴⁰` chance bound: members' ordinals are now a random permutation per row |
119119
| `member_ordinals_follow_key_order` | kept | whether members' ordinals follow key order, reported only below a `2⁻⁴⁰` chance bound |
120120
| `token_derivability` | kept | which reads' tokens the database can derive from which others' (now by the member rule and by the element rule at an index below 4, from the tokens that find a node in some row) |
121121
| `transcripts` | kept | every query term with its tags named: determinism, shared selectors, needles; a read is now its token list and budget instead of a token and gates |

‎languages/golang/encrypt/guest/Cargo.lock‎

Lines changed: 3 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎languages/golang/encrypt/guest/src/ops.rs‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ use stack_encrypt::sem::MatchOptions;
5151
use stack_encrypt::target::IndexSpec;
5252
use stack_encrypt::ErrorPayload;
5353
use stack_encrypt::{CipherText, KeysetCipher, Label, Node, SealedValue, StackCipherText};
54-
use stack_kms::DataKeySource;
54+
use stack_kms::{DataKeySource, IndexKeySource};
5555
use zeroize::Zeroizing;
5656

5757
use stack_guest_abi::last_error;
@@ -101,7 +101,7 @@ pub async fn term<K>(
101101
kind: u32,
102102
) -> Result<Vec<u8>, u32>
103103
where
104-
K: DataKeySource + Sync + 'static,
104+
K: DataKeySource + IndexKeySource + Sync + 'static,
105105
{
106106
// The same proof every stack-encrypt leaf demands: an empty context is
107107
// `STATUS_ENCODING` here, before any derivation.
@@ -178,7 +178,7 @@ pub async fn encrypt_record<K>(
178178
plan: &[u8],
179179
) -> Result<Vec<u8>, u32>
180180
where
181-
K: DataKeySource + Sync + 'static,
181+
K: DataKeySource + IndexKeySource + Sync + 'static,
182182
{
183183
let plan = parse_plan(plan)?;
184184
let tree =
@@ -213,7 +213,7 @@ pub async fn decrypt_record<K>(
213213
expected: Option<Label>,
214214
) -> Result<Vec<u8>, u32>
215215
where
216-
K: DataKeySource + Sync + 'static,
216+
K: DataKeySource + IndexKeySource + Sync + 'static,
217217
{
218218
let plan = parse_plan(plan)?;
219219
let value =
@@ -255,7 +255,7 @@ pub async fn query_term<K>(
255255
operator: &[u8],
256256
) -> Result<Vec<u8>, u32>
257257
where
258-
K: DataKeySource + Sync + 'static,
258+
K: DataKeySource + IndexKeySource + Sync + 'static,
259259
{
260260
let (plan, field, operator) = parse_query(plan, field, operator)?;
261261
let term = dynamic::query_term(

‎languages/golang/encrypt/guest/src/status.rs‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -129,6 +129,9 @@ pub fn status_for_error(error: &stack_encrypt::Error) -> u32 {
129129
// retrieved, like a foreign keyset and unlike a wrong-context open.
130130
stack_encrypt::Error::ContextMismatch { .. } => STATUS_CONTEXT_MISMATCH,
131131
stack_encrypt::Error::Kms(kms) => status_for_kms(kms),
132+
// A keyset a JSON document needed could not be loaded: the load's
133+
// own error says what the host can act on.
134+
stack_encrypt::Error::KeysetLoad { source, .. } => status_for_error(source),
132135
// A context that renders past ZeroKMS's descriptor limit is the
133136
// caller's input, refused before any request is sent.
134137
stack_encrypt::Error::DescriptorTooLong { .. } => STATUS_ENCODING,
@@ -141,6 +144,10 @@ pub fn status_for_error(error: &stack_encrypt::Error) -> u32 {
141144
// one it was opened as: sealed as that type, not tampered with, so
142145
// the caller's input and not STATUS_AUTH.
143146
stack_encrypt::Error::OpenedType { .. } => STATUS_ENCODING,
147+
// The entry `->` returned did not open alone: a container with
148+
// children asked for as a scalar, or an altered entry. The client
149+
// cannot tell which, so neither STATUS_AUTH nor a claim of either.
150+
stack_encrypt::Error::JsonNotALeaf => STATUS_ENCODING,
144151
_ => STATUS_INTERNAL,
145152
}
146153
}
@@ -283,13 +290,30 @@ mod tests {
283290
ViturRequestError::new(kind, "stubbed", std::io::Error::other("boom"))
284291
}
285292

293+
#[test]
294+
fn a_failed_keyset_load_maps_as_its_cause() {
295+
let load = stack_encrypt::Error::KeysetLoad {
296+
keyset_id: uuid::Uuid::from_u128(1),
297+
source: std::sync::Arc::new(stack_encrypt::Error::Kms(stack_kms::Error::LoadKeyset(
298+
stack_kms::LoadKeysetError::Unauthorized(vitur(
299+
ViturRequestErrorKind::Unauthorized,
300+
)),
301+
))),
302+
};
303+
assert_eq!(status_for_error(&load), STATUS_KMS_UNAUTHORIZED);
304+
}
305+
286306
#[test]
287307
fn aead_and_composition_errors_map_to_the_vitaminc_codes() {
288308
assert_eq!(status_for_error(&stack_encrypt::Error::Aead), STATUS_AUTH);
289309
assert_eq!(
290310
status_for_error(&stack_encrypt::Error::OpenedType { expected: "i32" }),
291311
STATUS_ENCODING
292312
);
313+
assert_eq!(
314+
status_for_error(&stack_encrypt::Error::JsonNotALeaf),
315+
STATUS_ENCODING
316+
);
293317
assert_eq!(
294318
status_for_error(&stack_encrypt::Error::DescriptorTooLong { len: 513 }),
295319
STATUS_ENCODING

‎packages/eql/Cargo.lock‎

Lines changed: 3 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎packages/eql/crates/eql-bindings/src/v4/json_tree.rs‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,8 @@ pub struct JsonTreeKeyHeader(pub Base64Text);
7777
/// `public.eql_v4_json_search`: a JSON tree document, `{v, i, h, sv:
7878
/// [entry], q: group}`. Strict. `h` is the document's key header, stored
7979
/// once (every entry is sealed under the document's one data key); `sv`
80-
/// holds one entry per node in depth-first order, the root first; `q` is
80+
/// holds one entry per node, the root first and the rest in a random order
81+
/// drawn when the document was sealed; `q` is
8182
/// the document's containment projection, one `{s, vs}` pair per node
8283
/// nested by array element, which the client writes beside the entries
8384
/// and which is the only place the value selectors live: the server's

0 commit comments

Comments
 (0)