Skip to content

Commit 26294a8

Browse files
committed
feat(stack-encrypt): derive element-scoped predicates for a JSON document field
`element_match` takes `{"path", "wildcard_bound"?, "predicates"}`: an array's elements by a path with a wildcard, and predicates on fields of an element by a member path relative to it. Its operand is `eql_v4.query_json_elements`: per predicate, the `subtree` tokens of the member's full path at the bound, and the leaf operands the orderings derive for its values. `eq` is one leaf as both `gte` and `lte`, so its equality is the ordering term's (folded text, numbers as float64), and `between` is `between_at`'s two. Each member's tokens are the subtree key of its full path, so the k-th token of every member names a field of one element, and the last the probe. No new derivation. Refused before anything is derived: a path with no wildcard, a `wildcard_bound` of 0, a position in the path or a member, a wildcard in a member, an empty predicate list, an unknown `op`, a value no ordering takes, and a member whose full path the field does not order. The operator is listed only on a field that orders a path and whose type names the elements domain (`TargetDescriptor::with_elements_query_sql_domain`), so a host that does not name it, such as the Go guest today, sees no new operator. The heap hygiene suite watches the operator too. Its spy now searches for the data key where its lock holds it, and records the key in place: it used to copy the key onto the stack to compare, on every free during a watch, and a future built later at that depth carried the copy into its box. On this branch that made the `json_open (subtree)` watch fail on the spy's own copy, in `load_keysets`' future box (macOS arm64, debug); with the copy gone the watch passes and `load_keysets` is unchanged. Refs #1440
1 parent 5975599 commit 26294a8

6 files changed

Lines changed: 1086 additions & 15 deletions

File tree

‎packages/stack-encrypt/src/dynamic/json_query.rs‎

Lines changed: 290 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@
1616
//! | `selector` | `{"path"}` | text: the selector `->` takes |
1717
//! | `subtree` | `{"path", "wildcard_bound"?}` | text array: the tokens `json_tree_subtree` takes |
1818
//! | `children`, `child_count` | `{"path"}`: a path with no wildcard | text: the node's token |
19+
//! | `element_match` | `{"path", "wildcard_bound"?, "predicates"}`: an array's elements, by a path with a wildcard, and predicates on fields of an element at ordered paths | EQL: per predicate, the field's read tokens and its bounds (`eql_v4.query_json_elements`), compared as the orderings compare, so its `eq` is of folded values |
1920
//!
2021
//! Every JSON value in an input is UTF-8 JSON text, parsed by
2122
//! [`parse_text`](crate::json::parse_text)'s rules. A path is JSONPath text
@@ -32,6 +33,52 @@
3233
//! hold of the same array element; one containment query of a document
3334
//! holding both does.
3435
//!
36+
//! # Element predicates
37+
//!
38+
//! `element_match` asks whether some one element of an array satisfies
39+
//! every predicate, which two `eq_at` or ordering operands cannot: the
40+
//! jsonpath `exists($.contacts[*] ? (@.relationship == "Sibling" &&
41+
//! @.calls > 5))`. Its input names the array's elements by a path with a
42+
//! wildcard (`$.contacts[*]`), and each predicate a field of an element by
43+
//! a `member` path relative to it (`relationship`, `address.postcode`,
44+
//! `["first name"]`), an `op` and its value:
45+
//!
46+
//! ```json
47+
//! {"path": "$.contacts[*]", "wildcard_bound": 16, "predicates": [
48+
//! {"member": "relationship", "op": "eq", "json": "\"Sibling\""},
49+
//! {"member": "calls", "op": "gt", "json": "5"},
50+
//! {"member": "age", "op": "between", "low": "18", "high": "40"}
51+
//! ]}
52+
//! ```
53+
//!
54+
//! `op` is `eq`, `lt`, `lte`, `gt` or `gte`, with `json`, or `between`,
55+
//! with `low` and `high`, inclusive. Each member's full path, the path and
56+
//! the member joined, must be one the field orders. The operand holds, per
57+
//! predicate, the tokens of a `subtree` read of the full path at the bound,
58+
//! and the bounds as the orderings derive them: `gt` the leaf operand
59+
//! `gt_at` derives, `between` the two `between_at` derives as `gte` and
60+
//! `lte`, and `eq` one leaf operand as both `gte` and `lte`. So equality
61+
//! here is the ordering term's: text compares case- and accent-folded, and
62+
//! numbers as float64, so integers past 2^53 can tie. Exact equality at a
63+
//! path is `eq_at`, of any element.
64+
//!
65+
//! Refused before anything is derived, each by its reason: a path with no
66+
//! wildcard ([`Reason::JsonPathNoWildcard`]), a `wildcard_bound` of `0`,
67+
//! whose operand could never be true ([`Reason::JsonWildcardBoundZero`]),
68+
//! an empty list of predicates ([`Reason::JsonNoPredicates`]), an `op`
69+
//! that is none of the six ([`Reason::JsonUnknownPredicateOp`]), a
70+
//! predicate whose keys do not fit its `op` ([`Reason::JsonInput`]), a
71+
//! wildcard in a member ([`Reason::JsonPathWildcard`]); and by the index's
72+
//! own refusals, a position in the path or a member
73+
//! ([`TermError::JsonOrderingAtPosition`]), a member whose full path the
74+
//! field does not order ([`TermError::JsonPathNotOrdered`]), a value that
75+
//! is not a number or a string ([`TermError::JsonLeafNotOrdered`]), and a
76+
//! member that is not a path ([`TermError::JsonPath`], at an offset into
77+
//! the member's own text). The
78+
//! database refuses a row whose array is longer than the bound, as a
79+
//! subtree read of it opens to a refusal: each member's tokens end with
80+
//! the probe, and `json_tree_element_match` raises when it names a node.
81+
//!
3582
//! # Reads
3683
//!
3784
//! [`json_open`] opens the rows a read returned, against the request that
@@ -76,6 +123,8 @@ pub(super) enum Binds {
76123
Text,
77124
/// `text[]`.
78125
TextArray,
126+
/// The target's element predicate domain (`eql_v4.query_json_elements`).
127+
Elements,
79128
}
80129

81130
/// What a JSON operator's operand binds as, or `None` for an operator
@@ -90,6 +139,7 @@ pub(super) fn binds(op: OpSpec) -> Option<Binds> {
90139
}
91140
OpSpec::Selector | OpSpec::Children | OpSpec::ChildCount => Some(Binds::Text),
92141
OpSpec::Subtree => Some(Binds::TextArray),
142+
OpSpec::ElementMatch => Some(Binds::Elements),
93143
OpSpec::Eq
94144
| OpSpec::Matches
95145
| OpSpec::Lt
@@ -108,6 +158,7 @@ pub(super) fn domain(target: &super::TargetDescriptor, binds: Binds) -> Option<S
108158
Binds::Leaf => target.leaf_query_sql_domain.clone(),
109159
Binds::Text => Some("text".to_owned()),
110160
Binds::TextArray => Some("text[]".to_owned()),
161+
Binds::Elements => target.elements_query_sql_domain.clone(),
111162
}
112163
}
113164

@@ -199,6 +250,53 @@ struct ReadInput {
199250
wildcard_bound: Option<u32>,
200251
}
201252

253+
#[derive(Deserialize)]
254+
#[serde(deny_unknown_fields)]
255+
struct ElementsInput {
256+
path: String,
257+
#[serde(default)]
258+
wildcard_bound: Option<u32>,
259+
predicates: Vec<PredicateInput>,
260+
}
261+
262+
#[derive(Deserialize)]
263+
#[serde(deny_unknown_fields)]
264+
struct PredicateInput {
265+
member: String,
266+
op: String,
267+
#[serde(default)]
268+
json: Option<JsonText>,
269+
#[serde(default)]
270+
low: Option<JsonText>,
271+
#[serde(default)]
272+
high: Option<JsonText>,
273+
}
274+
275+
/// How a predicate of `element_match` compares its field with one value.
276+
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
277+
enum Compare {
278+
Eq,
279+
Lt,
280+
Lte,
281+
Gt,
282+
Gte,
283+
}
284+
285+
/// A predicate's bounds, as its input gives them.
286+
enum Bounds {
287+
/// One value, compared by the predicate's `op`.
288+
One(Compare, Parsed),
289+
/// `between`: the low bound then the high, both inclusive.
290+
Between(Parsed, Parsed),
291+
}
292+
293+
/// One predicate of `element_match`, checked: the full path of the
294+
/// element's field, which the field orders, and its bounds.
295+
struct Predicate {
296+
path: JsonPath,
297+
bounds: Bounds,
298+
}
299+
202300
/// An operator's input, read from its frame and checked: everything
203301
/// [`derive`] would refuse before it derives anything.
204302
enum Input {
@@ -207,6 +305,7 @@ enum Input {
207305
Between(JsonPath, Parsed, Parsed),
208306
Path(JsonPath),
209307
Read(JsonPath, u32),
308+
Elements(u32, Vec<Predicate>),
210309
}
211310

212311
fn read_frame<T: for<'de> Deserialize<'de>>(input: &ValueFrame) -> Result<T, Error> {
@@ -280,10 +379,97 @@ fn read_input(op: OpSpec, options: &JsonOptions, input: &ValueFrame) -> Result<I
280379
read.wildcard_bound.unwrap_or(JsonSubtreeKey::DEFAULT_BOUND),
281380
))
282381
}
382+
OpSpec::ElementMatch => read_elements(options, input),
283383
_ => Err(Error::Internal),
284384
}
285385
}
286386

387+
/// `element_match`'s input: an array's elements, by a path with a wildcard
388+
/// and no position, and one or more predicates on their fields.
389+
fn read_elements(options: &JsonOptions, input: &ValueFrame) -> Result<Input, Error> {
390+
let elements: ElementsInput = read_frame(input)?;
391+
let base = path(&elements.path)?;
392+
if base.is_positional() {
393+
return Err(Error::bad_source(Reason::JsonPathNoWildcard));
394+
}
395+
if base.names_a_position() {
396+
return Err(term(TermError::JsonOrderingAtPosition));
397+
}
398+
let bound = elements
399+
.wildcard_bound
400+
.unwrap_or(JsonSubtreeKey::DEFAULT_BOUND);
401+
if bound == 0 {
402+
return Err(Error::bad_source(Reason::JsonWildcardBoundZero));
403+
}
404+
if elements.predicates.is_empty() {
405+
return Err(Error::bad_source(Reason::JsonNoPredicates));
406+
}
407+
let predicates = elements
408+
.predicates
409+
.iter()
410+
.map(|predicate| read_predicate(options, &base, predicate))
411+
.collect::<Result<Vec<_>, _>>()?;
412+
Ok(Input::Elements(bound, predicates))
413+
}
414+
415+
/// One predicate of `element_match`: its member's full path below `base`,
416+
/// which the field must order, and the bounds its `op` takes.
417+
fn read_predicate(
418+
options: &JsonOptions,
419+
base: &JsonPath,
420+
predicate: &PredicateInput,
421+
) -> Result<Predicate, Error> {
422+
let compare = match predicate.op.as_str() {
423+
"eq" => Some(Compare::Eq),
424+
"lt" => Some(Compare::Lt),
425+
"lte" => Some(Compare::Lte),
426+
"gt" => Some(Compare::Gt),
427+
"gte" => Some(Compare::Gte),
428+
"between" => None,
429+
_ => return Err(Error::bad_source(Reason::JsonUnknownPredicateOp)),
430+
};
431+
let path = member_path(options, base, &predicate.member)?;
432+
let value = |text: &JsonText| -> Result<Parsed, Error> {
433+
let value = text.parse()?;
434+
leaf(&value.0)?;
435+
Ok(value)
436+
};
437+
let bounds = match (compare, &predicate.json, &predicate.low, &predicate.high) {
438+
(Some(compare), Some(json), None, None) => Bounds::One(compare, value(json)?),
439+
(None, None, Some(low), Some(high)) => Bounds::Between(value(low)?, value(high)?),
440+
_ => return Err(Error::bad_source(Reason::JsonInput)),
441+
};
442+
Ok(Predicate { path, bounds })
443+
}
444+
445+
/// A member's full path: `base` and then the member's steps, read as a
446+
/// path relative to an element (`name`, `address.city`, `["a b"]`). A
447+
/// member names one field of each element, so it has no wildcard of its
448+
/// own, and the full path is an ordering's: no position, and ordered. A
449+
/// member that does not parse is refused at an offset into the member's
450+
/// own text.
451+
fn member_path(options: &JsonOptions, base: &JsonPath, member: &str) -> Result<JsonPath, Error> {
452+
let prefix = if member.starts_with('[') { "$" } else { "$." };
453+
let relative = JsonPath::parse(&format!("{prefix}{member}")).map_err(|error| {
454+
let error = crate::json::JsonPathError {
455+
at: error.at.saturating_sub(prefix.len()),
456+
..error
457+
};
458+
Error::Cipher(TermError::JsonPath(error).into())
459+
})?;
460+
if !relative.is_positional() {
461+
return Err(Error::bad_source(Reason::JsonPathWildcard));
462+
}
463+
let full = relative
464+
.segments()
465+
.iter()
466+
.fold(base.clone(), |at, segment| at.then(segment.clone()));
467+
if full.segments().len() > crate::json::MAX_DEPTH {
468+
return Err(term(TermError::JsonTooDeep));
469+
}
470+
ordered_path_of(options, full)
471+
}
472+
287473
/// `eq_at`'s input: a scalar at a path read through a selector.
288474
fn read_eq_at(input: &ValueFrame) -> Result<Input, Error> {
289475
let at: AtInput = read_frame(input)?;
@@ -324,7 +510,11 @@ fn read_between(options: &JsonOptions, input: &ValueFrame) -> Result<Input, Erro
324510
/// position as the wildcard), so the data plan's own,
325511
/// [`Reason::JsonPathPosition`], is what refuses one there.
326512
fn ordered_path(options: &JsonOptions, text: &str) -> Result<JsonPath, Error> {
327-
let path = path(text)?;
513+
ordered_path_of(options, path(text)?)
514+
}
515+
516+
/// [`ordered_path`] of a path already parsed.
517+
fn ordered_path_of(options: &JsonOptions, path: JsonPath) -> Result<JsonPath, Error> {
328518
if path.names_a_position() {
329519
Err(term(TermError::JsonOrderingAtPosition))
330520
} else if options.orders(&path) {
@@ -421,9 +611,108 @@ pub(super) fn derive<'a, K: 'static>(
421611
let key = keyset.json_subtree_key_now(&path, options, bound, column())?;
422612
Pending::ready(keyset, Ok(Operand::TextList(key.tokens())))
423613
}
614+
Input::Elements(bound, predicates) => elements(keyset, label, options, bound, &predicates)?,
424615
})
425616
}
426617

618+
/// One member of an `element_match` operand: the tokens of a read of the
619+
/// member's full path, probes included, and its bounds, each a leaf
620+
/// operand. `eql_v4.query_json_elements` holds a list of them as `{"m"}`.
621+
#[derive(serde::Serialize)]
622+
struct MemberOperand {
623+
t: Vec<String>,
624+
#[serde(skip_serializing_if = "Option::is_none")]
625+
gt: Option<JsonLeafQuery>,
626+
#[serde(skip_serializing_if = "Option::is_none")]
627+
gte: Option<JsonLeafQuery>,
628+
#[serde(skip_serializing_if = "Option::is_none")]
629+
lt: Option<JsonLeafQuery>,
630+
#[serde(skip_serializing_if = "Option::is_none")]
631+
lte: Option<JsonLeafQuery>,
632+
}
633+
634+
/// `element_match`'s operand, `eql_v4.query_json_elements`.
635+
#[derive(serde::Serialize)]
636+
struct ElementsOperand {
637+
m: Vec<MemberOperand>,
638+
}
639+
640+
impl MemberOperand {
641+
/// The member reading `tokens` that compares as `compare`, with the
642+
/// leaf operands of its values in order: one for a comparison, the
643+
/// low's then the high's for `between` (`compare` `None`). Equality is
644+
/// both inclusive bounds at its one value.
645+
fn new(tokens: Vec<String>, compare: Option<Compare>, leaves: Vec<JsonLeafQuery>) -> Self {
646+
let mut member = Self {
647+
t: tokens,
648+
gt: None,
649+
gte: None,
650+
lt: None,
651+
lte: None,
652+
};
653+
let mut leaves = leaves.into_iter();
654+
match compare {
655+
Some(Compare::Eq) => {
656+
member.gte = leaves.next();
657+
member.lte = member.gte.clone();
658+
}
659+
Some(Compare::Lt) => member.lt = leaves.next(),
660+
Some(Compare::Lte) => member.lte = leaves.next(),
661+
Some(Compare::Gt) => member.gt = leaves.next(),
662+
Some(Compare::Gte) => member.gte = leaves.next(),
663+
None => {
664+
member.gte = leaves.next();
665+
member.lte = leaves.next();
666+
}
667+
}
668+
member
669+
}
670+
}
671+
672+
/// `element_match`'s operand: for each predicate, the tokens of a subtree
673+
/// read of its full path at `bound`, as `subtree` derives them, and the
674+
/// leaf operand of each of its values, as the orderings derive them. The
675+
/// reads share the path and the bound, so the `k`-th token of every member
676+
/// names a field of one element, and each member's last names the element
677+
/// at the bound: the probe.
678+
fn elements<'a, K: 'static>(
679+
keyset: &'a KeysetCipher<'_, K>,
680+
label: &Label,
681+
options: &JsonOptions,
682+
bound: u32,
683+
predicates: &[Predicate],
684+
) -> Result<Pending<'a, Operand, K>, Error> {
685+
let mut members = Vec::with_capacity(predicates.len());
686+
for predicate in predicates {
687+
let tokens = keyset
688+
.json_subtree_key_now(
689+
&predicate.path,
690+
options,
691+
bound,
692+
NonEmpty::from(label.clone()),
693+
)?
694+
.tokens();
695+
let leaf = |value: &Parsed| {
696+
terms(
697+
keyset,
698+
options,
699+
label,
700+
predicate.path.holding(value.0.clone()),
701+
)
702+
.map(|terms| JsonLeafQuery::of_leaf(&terms))
703+
};
704+
let (compare, leaves) = match &predicate.bounds {
705+
Bounds::One(compare, value) => (Some(*compare), vec![leaf(value)]),
706+
Bounds::Between(low, high) => (None, vec![leaf(low), leaf(high)]),
707+
};
708+
members.push(
709+
Pending::all(keyset, leaves)
710+
.map(move |leaves| MemberOperand::new(tokens, compare, leaves)),
711+
);
712+
}
713+
Ok(Pending::all(keyset, members).try_map(|m| eql(&ElementsOperand { m }).map(Operand::One)))
714+
}
715+
427716
/// A read to open, as the request that prepared it names it: the
428717
/// operator, the path and, for a subtree read, the wildcard bound. What a
429718
/// binding keeps from preparing the read and hands back with its rows; the

‎packages/stack-encrypt/src/dynamic/mod.rs‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -657,6 +657,18 @@ reasons! {
657657
/// finds a null by its term. A `"null": "sql"` field's nulls are SQL
658658
/// `NULL`, which `IS NULL` finds.
659659
NullQuery => "null_query", r#"a null is found by its term only with eq on a field with "null": "sealed"; a "null": "sql" field's nulls are SQL NULL"#;
660+
/// An element predicate's path has no wildcard: the predicate is
661+
/// over the elements of an array, which `[*]` names.
662+
JsonPathNoWildcard => "json_path_no_wildcard", "an element predicate is over an array's elements, and the path names no wildcard";
663+
/// An element predicate has no predicates: it would be true of
664+
/// every row with an element.
665+
JsonNoPredicates => "json_no_predicates", "an element predicate needs at least one predicate";
666+
/// An element predicate's `wildcard_bound` is `0`: its reads would
667+
/// send the probe alone, so the predicate could never be true.
668+
JsonWildcardBoundZero => "json_wildcard_bound_zero", "an element predicate reads at least one element, and a wildcard_bound of 0 reads none";
669+
/// A predicate of an element predicate has an `op` that is not
670+
/// `eq`, `lt`, `lte`, `gt`, `gte` or `between`.
671+
JsonUnknownPredicateOp => "json_unknown_predicate_op", r#"a predicate's "op" is not eq, lt, lte, gt, gte or between"#;
660672
}
661673
}
662674

0 commit comments

Comments
 (0)