Skip to content

Commit e65fca9

Browse files
committed
docs(go): show the element match in the references and the JSON example
The stashgen README's "Encrypted JSON documents" table gets the Match row and its SQL, with when Elements and Paths are generated, how encrypt.AtPath makes a predicate from a path, what a path's type checks, the run-time Member form, what is refused before the engine, which arrays get no accessor, the folded equality and the 22023 bound. Its list of what stashgen writes, and -name, name Paths. The SDK README and package doc show the call, and the CHANGELOG records it under Unreleased. The typed-document example orders each visit's at and place, and finds the customer with one visit at home after 15, with the paths from Paths: Ada has a visit at home and a visit after 15, but no one visit that is both. Its generated file is regenerated from the new tag. Refs #1442 Claude-Session: https://claude.ai/code/session_01MrxG73fnX1p7oiizh3GRG5
1 parent 586a67e commit e65fca9

8 files changed

Lines changed: 279 additions & 16 deletions

File tree

‎languages/golang/CHANGELOG.md‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,39 @@ listed under **Breaking**. Releases are tagged `languages/golang/vX.Y.Z`.
1212

1313
### Added
1414

15+
- **Test several members of one array element of a `JsonSearch` field
16+
(#1442).** A field whose `ordered=` paths name members beneath an array's
17+
wildcard (`$.contacts[*].relationship;$.contacts[*].calls`) gets
18+
`Fields.<Field>.Elements.<Array>.Match`, one accessor per array, named as
19+
`At` names a path (`Elements.Contacts`). `Match(ctx, cipher, opts,
20+
predicates...)` returns an `eql.JSONSearchElementsQuery` for
21+
`eql_v4.json_tree_element_match(col, $1::eql_v4.query_json_elements)`,
22+
which finds a row where one element passes every predicate. An `EqAt` and
23+
an ordering at the array's paths can each be answered by a different
24+
element. The generated `Paths` names each array's ordered members, nested
25+
as the JSON nests them (`Paths.Profile.Contacts.Relationship`), and
26+
`encrypt.AtPath(path)` makes the predicates: `Eq`, `Lt`, `Lte`, `Gt`,
27+
`Gte` and `Between`. A path's type, `encrypt.Path[E, T]`, carries its
28+
array and the member's Go type from the document's struct (`any` for a
29+
`json.RawMessage`), so a value of another type, or a predicate on another
30+
array's member, does not compile. `Paths` holds only fields, so no JSON
31+
key is reserved. For a member named at run time, the array accessor's
32+
`Member("relationship")` makes the same predicates with `any` values.
33+
Each comparison is of the member's ordering term, so `Eq` compares text
34+
case- and accent-folded and integers past 2^53 can tie; `EqAt` stays
35+
exact. A member named at run time that the field does not order is
36+
refused before the engine is asked, with `encrypt.ErrTerm`, naming the
37+
member and the array; no predicates, a zero `ElementPredicate` and a
38+
member with a wildcard are refused with `encrypt.ErrEncoding`. An array
39+
whose members `Paths` cannot name (a key with no Go name, or one name for
40+
two paths) gets no accessor and a notice, as two arrays with one
41+
accessor name do. A row whose array is
42+
longer than `JSONReadOptions.WildcardBound` (64 by default) fails the
43+
query with SQLSTATE 22023. An `ElementPredicate` prints and logs its
44+
member and comparison, never its values. The embedded EQL guest now names
45+
the `eql_v4.query_json_elements` domain, so `Checker.Operators` lists
46+
`element_match` (input `path_predicates`) on a field that orders a path.
47+
1548
- **A two-letter word is searchable in `TextMatch`, `TextSearch` and
1649
`TextSearchOre` (#1309).** The EQL v4 match types index each two-letter
1750
word whole, beside the 3-grams of the text, so a two-letter surname such

‎languages/golang/cmd/stashgen/README.md‎

Lines changed: 47 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -178,6 +178,7 @@ The field's `Fields` entry has a method per operation:
178178
| `Selector(ctx, c, path)` | `encrypt.JSONSelection`: `Selector()` and `Open` | `col -> $1::text` |
179179
| `Subtree(ctx, c, path, opts)` | `encrypt.JSONRead`: `Tokens()`, `Open` and `OpenOne` | `eql_v4.json_tree_subtree(col, $1::text[], 64)` |
180180
| `Children(ctx, c, path)`, `ChildCount(ctx, c, path)` | `encrypt.JSONToken` | `eql_v4.json_tree_children(col, $1::text)`, `eql_v4.json_tree_child_count(col, $1::text)` |
181+
| `Elements.Visits.Match(ctx, c, opts, predicates...)` | `eql.JSONSearchElementsQuery` | `eql_v4.json_tree_element_match(col, $1::eql_v4.query_json_elements)` |
181182

182183
`At` has one accessor per ordered path, named for its keys (`$.visits[*].at` is `At.VisitsAt`, `$.a.1` is `At.A1`); two paths with one name, or one path given twice, stop `stashgen`.
183184
An ordering compares the value `->` extracts at its path, and EQL's `<` compares ordering terms alone, so take the selector from the same accessor (`At.Age.Selector`) rather than spelling the path again.
@@ -193,6 +194,50 @@ SELECT id FROM customers ORDER BY (
193194
```
194195

195196
with `$1` from `At.VisitsAt.Selector(ctx, c)` and `$2` from `At.VisitsAt.Gt(ctx, c, v)`; for the latest visit, order by `ORDER BY 1 DESC NULLS LAST` inside and `DESC` outside.
197+
198+
Two queries at a wildcard path, an `EqAt` and an ordering say, can each be answered by a different element.
199+
To test several members of one element together, use `Match` under `Elements`.
200+
`Elements` has one accessor per array whose elements have an ordered member, named for the array's keys as `At` names a path: with `ordered=$.visits[*].at;$.visits[*].place`, `$.visits[*]` is `Elements.Visits`, with the members `at` and `place`.
201+
A root array (`ordered=$[*].name`) is `Elements.Root`.
202+
An ordered path that ends at its wildcard (`$.tags[*]`), or has none, gives no accessor, and a field without one has no `Elements`.
203+
Two arrays whose keys give one name (`$.a_b[*]` and `$.aB[*]` are both `AB`), and an array whose keys give no name, get no accessor: `stashgen` still writes the field and its `At`, and prints a notice naming the ordered paths, which the generated file repeats in a comment.
204+
205+
The generated `Paths` names each accessor's members, nested as the JSON nests them: `$.visits[*].place` is `Paths.Profile.Visits.Place`.
206+
`encrypt.AtPath` makes a predicate from one, with `Eq`, `Lt`, `Lte`, `Gt`, `Gte` or `Between` (inclusive).
207+
208+
```go
209+
// rows with one visit that is at home and after 15
210+
q, err := customers.Fields.Profile.Elements.Visits.Match(ctx, cipher, encrypt.JSONReadOptions{},
211+
encrypt.AtPath(customers.Paths.Profile.Visits.Place).Eq("home"),
212+
encrypt.AtPath(customers.Paths.Profile.Visits.At).Gt(15),
213+
)
214+
```
215+
216+
```sql
217+
SELECT id FROM customers WHERE eql_v4.json_tree_element_match(profile, $1::eql_v4.query_json_elements)
218+
```
219+
220+
A path is an `encrypt.Path[E, T]`.
221+
`E` is a type `stashgen` writes for the array (`ProfileFieldElementsVisits`), and `Match` takes only `encrypt.ElementPredicate[E]` of its own array, so a predicate on another array's member does not compile.
222+
`T` is the member's Go type in the document's struct, read as `encoding/json` reads the document (through pointers, promoted fields, maps and slices), so a value of another type does not compile either.
223+
It is `any` where the type names none: a `json.RawMessage` document, a path the struct does not hold, or a member inside a type with its own `MarshalJSON`.
224+
`Paths` holds only fields, and its paths' methods are the SDK's, so any JSON key can name a member.
225+
With `-name Customer`, it is `CustomerPaths`, as `Fields` is `CustomerFields`.
226+
227+
For a filter built at run time, from text such as `visits:(place = "home" AND at > 15)`, name the member with the array's `Member`, written as it is below the array in the tag (`place`, or `address.city`):
228+
229+
```go
230+
visits := customers.Fields.Profile.Elements.Visits
231+
q, err := visits.Match(ctx, cipher, encrypt.JSONReadOptions{}, visits.Member("place").Eq("home"), visits.Member("at").Gt(15))
232+
```
233+
234+
Its values are `any`, as the orderings take them.
235+
A member the field does not order is refused before the engine is asked, with `encrypt.ErrTerm`, naming the member, the array and the members it orders; no predicates, a zero `ElementPredicate`, and a member with a wildcard, are refused with `encrypt.ErrEncoding`.
236+
`Paths` names each key on its own, so an array whose members it cannot name gets no accessor and the same notice: a key that gives no Go name there (`$.a.1[*].x`, whose `1` `At` joins to `A1X`), or one name for two paths (`$.a[*].b` beside `$.a[*].b.c`, or `$.x.a_b[*].m` beside `$.x.aB.n[*].o`).
237+
Each comparison is of the member's ordering term, so `Eq` is an equality of folded values: text compares case- and accent-folded (`"home"` equals `"Home"`), and integers past 2^53 can tie.
238+
`EqAt` at the member's full path is exact, of any element.
239+
`Match` reads each array to `opts.WildcardBound` elements, 64 when it is 0, and a row whose array is longer fails the query with SQLSTATE 22023 rather than answer over part of it: set the bound past the longest array.
240+
The function runs on each row a scan gives it, so narrow the rows first with an indexed predicate, such as `@>` on the same values.
196241
`OpenOne` reads one value and refuses a read whose path has a wildcard before anything is opened; open a wildcard read with `Open`.
197242
A needle and a value are Go values: a `json.RawMessage` of JSON text crosses as it is, and anything else is marshalled with `encoding/json`.
198243
`EqAt` and the orderings take a scalar: a string, a number, a bool, or `nil` for JSON `null`.
@@ -253,6 +298,7 @@ For `-type User`, the file `user_stash.go` holds:
253298
`encrypt_into=TextMatch` answers `Matches`, which returns an `encrypt.EQLMatchQuery`: the `eql.TextMatchQuery` operand, with `Verify` and `TargetLimit`.
254299
The query of `Matches` carries `Verify` and `TargetLimit`, because its answer can hold rows that do not match; an exact operator's query carries neither.
255300
See the encrypt package's README for each method's result and the query with `database/sql`.
301+
- `Paths`, when a `JsonSearch` field orders members of an array's elements: those members, nested as the JSON nests them, for `encrypt.AtPath` and the array's `Match` (see [Encrypted JSON documents](#encrypted-json-documents)).
256302
- `String` and `LogValue` on `EncryptedUser`, which print the passthrough fields and hide the sealed ones.
257303
- A copy of the fields of `User`, so a change to them stops the build.
258304

@@ -263,7 +309,7 @@ Generated code uses no reflection, and no function in it panics.
263309
| Flag | Meaning |
264310
|---|---|
265311
| `-type T` | The struct that carries the `stash` tags. Required. |
266-
| `-name N` | Write `EncryptN`, `DecryptN` and `NFields`. A package holds one `Encrypt`; the second struct in a package needs a name. |
312+
| `-name N` | Write `EncryptN`, `DecryptN` and `NFields` (and `NPaths`). A package holds one `Encrypt`; the second struct in a package needs a name. |
267313
| `-for P.F` | `T` declares the tags for `F`, a type in another package. Each field of `T` names a field of `F` with the same name and type, and every exported field of `F` is named. |
268314
| `-model Name=R` | A model `R` for separate columns: one field for each column, each tagged with the output it holds, `stash:"email"` or `stash:"email,equality"`. Writes `EncryptName` and `DecryptName`. Any number. |
269315
| `-model Name=R:D` | The same, for an `R` that cannot carry tags. The struct `D` in your package declares them. |

‎languages/golang/encrypt/README.md‎

Lines changed: 30 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -520,7 +520,7 @@ The EQL types the engine produces, and the Go field types each takes:
520520
| `Numeric` | `encrypt.Decimal` | none |
521521
| `NumericEq` | `encrypt.Decimal` | `=`, `<>` (`1.5 = 1.50`) |
522522
| `NumericOrd`, `NumericOrdOpe`, `NumericOrdOre` | `encrypt.Decimal` | `<`, `<=`, `>`, `>=`, `ORDER BY` |
523-
| `JsonSearch` (`eql.JSONSearch`) | struct, map, slice, or `json.RawMessage` | containment, equality at a path, ordering at declared paths, selection, subtree reads; see [Encrypted JSON documents](#encrypted-json-documents) |
523+
| `JsonSearch` (`eql.JSONSearch`) | struct, map, slice, or `json.RawMessage` | containment, equality at a path, ordering at declared paths, several members of one array element, selection, subtree reads; see [Encrypted JSON documents](#encrypted-json-documents) |
524524

525525
Any built-in Go integer field takes an integer EQL type (ADR-0002): the field
526526
is sealed at the type's width, and opens back into the field's type. A type
@@ -658,6 +658,35 @@ element of each row, so an ordering over every element is written over
658658
`eql_v4.json_tree_path_query(col, $sel)`, its matches (the stashgen README
659659
shows the SQL).
660660

661+
Two such queries can each be answered by a different element. To test
662+
several members of one array element together, use `Match` on the array's
663+
accessor under `Elements`. A field has one for each array whose elements
664+
have an ordered member (`ordered=$.visits[*].at;$.visits[*].place`). The
665+
generated `Paths` names those members, nested as the JSON nests them, and
666+
`encrypt.AtPath` makes a predicate from one:
667+
668+
```go
669+
q, err := users.Fields.Profile.Elements.Visits.Match(ctx, cipher, encrypt.JSONReadOptions{},
670+
encrypt.AtPath(users.Paths.Profile.Visits.Place).Eq("home"),
671+
encrypt.AtPath(users.Paths.Profile.Visits.At).Gt(15),
672+
)
673+
rows, err := db.QueryContext(ctx, `SELECT id FROM users WHERE eql_v4.json_tree_element_match(profile, $1::eql_v4.query_json_elements)`, q)
674+
```
675+
676+
A path carries its array and the member's Go type from the document's
677+
struct, so a value of another type, or a predicate on another array's
678+
member, does not compile. For a filter built at run time, name the member
679+
with the array's `Member`:
680+
`users.Fields.Profile.Elements.Visits.Member("place").Eq("home")`. A member
681+
named so that the field does not order is refused before the engine is
682+
asked, naming it.
683+
684+
Each comparison is of the member's ordering term, so `Eq` compares text
685+
case- and accent-folded, and integers past 2^53 can tie; `EqAt` is exact.
686+
A row whose array is longer than
687+
`JSONReadOptions.WildcardBound` (64 by default) fails the query with
688+
SQLSTATE 22023.
689+
661690
## Building
662691

663692
The package embeds `wasm/stack_encrypt_guest.wasm`, a build artefact of the

‎languages/golang/encrypt/doc.go‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -242,6 +242,29 @@
242242
// A selection or a subtree read opens values as JSON text ([JSONValue],
243243
// [JSONRead.Open]). [DecodeJSON] reads one into a Go type.
244244
//
245+
// Two queries at an array's paths can each be answered by a different
246+
// element. To test several members of one element together, use Match on
247+
// the array's accessor under Elements, which a field has for each array
248+
// whose elements have an ordered member. The generated Paths names those
249+
// members, nested as the JSON nests them, and [AtPath] makes a predicate
250+
// from one:
251+
//
252+
// q, err := customers.Fields.Profile.Elements.Visits.Match(ctx, cipher, encrypt.JSONReadOptions{},
253+
// encrypt.AtPath(customers.Paths.Profile.Visits.Place).Eq("home"),
254+
// encrypt.AtPath(customers.Paths.Profile.Visits.At).Gt(15),
255+
// )
256+
// // WHERE eql_v4.json_tree_element_match(profile, $1::eql_v4.query_json_elements)
257+
//
258+
// A [Path] carries its array and the member's Go type, so a value of
259+
// another type, or a predicate on another array's member, does not
260+
// compile. For a filter built at run time, the array accessor's Member
261+
// names the member as text: Visits.Member("place").Eq("home").
262+
//
263+
// Each comparison is of the member's ordering term, so [MemberTest.Eq]
264+
// compares text case- and accent-folded, and integers past 2^53 can tie. A row whose
265+
// array is longer than [JSONReadOptions.WildcardBound] (64 by default)
266+
// fails the query with SQLSTATE 22023.
267+
//
245268
// # Cancellation
246269
//
247270
// Every call takes a [context.Context], and a call whose context is

‎languages/golang/encrypt/example/json/README.md‎

Lines changed: 25 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
11
# Typed encrypted JSON document example
22

33
This program stores a Go `Profile` struct as one encrypted EQL v4 JSON column.
4-
It finds a row by containment, compares an ordered path, decrypts a whole
5-
profile, and opens only the visit times from another query.
4+
It finds a row by containment, compares an ordered path, finds a row where one
5+
visit passes two tests, decrypts a whole profile, and opens only the visit
6+
times from another query.
67

78
## Run
89

@@ -35,8 +36,28 @@ its `stash` tags. After changing the struct or tags, run this from
3536
go generate ./encrypt/example/json
3637
```
3738

38-
`Profile` uses `encoding/json` for the document shape. The `ordered=$.age`
39-
tag adds an ordering term for that path. A containment needle made from a
39+
`Profile` uses `encoding/json` for the document shape. The
40+
`ordered=$.age;$.visits[*].at;$.visits[*].place` tag adds an ordering term
41+
for each of those paths. Because `at` and `place` are ordered members of
42+
each visit, the generated `Fields.Profile.Elements.Visits.Match` tests both
43+
on one visit. The generated `Paths` names them as the JSON nests them:
44+
45+
```go
46+
q, err := Fields.Profile.Elements.Visits.Match(ctx, cipher, encrypt.JSONReadOptions{},
47+
encrypt.AtPath(Paths.Profile.Visits.Place).Eq("home"),
48+
encrypt.AtPath(Paths.Profile.Visits.At).Gt(15),
49+
)
50+
// SELECT id FROM customers WHERE eql_v4.json_tree_element_match(profile, $1::eql_v4.query_json_elements)
51+
```
52+
53+
`Paths.Profile.Visits.At` has the Go type of `Visit.At`, `int64`, so
54+
`Gt("15")` does not compile, and neither does a predicate on another
55+
array's member. Ada has a visit at home and a visit after 15, but no one
56+
visit that is both, so only Bea's row matches. Each comparison is of the
57+
ordering term, so text
58+
compares case- and accent-folded. A row with more visits than
59+
`JSONReadOptions.WildcardBound` (64 by default) fails the query with SQLSTATE
60+
22023. A containment needle made from a
4061
struct includes every field without `omitempty`, including zero values;
4162
`main.go` uses a map to name only the member being queried. The prepared
4263
subtree read returns JSON text, which `encrypt.DecodeJSON[int64]` decodes.

0 commit comments

Comments
 (0)