Skip to content

Commit f3539d6

Browse files
committed
fix(eql): disable the ORE range domains where the ORE opclass is absent
A non-superuser install skips the ore_block_256 btree operator class, and ore_fallback.sql then poisons every single-bound ORE domain so a value cast to one fails loudly with an OPE hint. It could not reach the nine query_<T>_range_ore domains: it is shared with EQL v3 and they are v4-only. So on such an install a range operand was accepted, and eql_v4.between ran as an unindexed seq scan with no hint. Each v4-only range types file now adds the same NOT VALID poison CHECK to its own ORE range, behind the same opclass check, calling the function ore_fallback.sql creates, and names the family's OPE range in the hint. A new PostgreSQL test installs the built v4 bundle as a NOSUPERUSER role in a database of its own and checks every ORE range is refused with that hint while its OPE range is accepted. Stripping the new ALTER DOMAINs from the bundle makes it fail. Also: the generated ORE between note now lists the family's own OPE column domains and OPE range instead of a fixed `_ord`, `_ord_ope`, `_search` (only text has `_search`), and the inverted-range wording no longer says the client refuses such a range, which #1235 makes untrue; it says what holds either way: such a range matches no row and the SQL does not reorder it.
1 parent b7a793c commit f3539d6

24 files changed

Lines changed: 615 additions & 63 deletions

‎.changeset/eql-v4-bindings.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,6 @@
44

55
**The `eql-bindings` crate has an `eql_bindings::v4` module, with the `stack-encrypt` feature: payload types for the EQL v4 domains.** EQL v4 is the v3 SQL installed under a second name (ADR-0002), for values written by Stack Encrypt, so the module exists only with that feature and its types derive Stack Encrypt's traits without a further gate. The module is generated from the same catalog as `v3`, with `eql_v4_*` domain names and `"v": 4`. `SchemaVersion` is now generic over the version it admits: `v3::SchemaVersion` is `SchemaVersion<3>` and `v4::SchemaVersion` is `SchemaVersion<4>`, each refusing the other's payloads, and plain `SchemaVersion` still means v3. Only v4 types derive Stack Encrypt's `EncryptFrom` and `DecryptInto`, and the Stack Encrypt target table (now `eql_bindings::v4::targets`) names v4 domains. Besides `TextEq`, Stack Encrypt now produces the storage-only types `Text`, `Integer`, `Bigint`, `Boolean`, `Real` and `Double`, and `TextMatch` (3-grams, `k = 6`, a 2048-bit filter, pinned as `eql_bindings::encryption::EqlMatch`; text shorter than three characters is refused, stored or queried, and only `stored @@ query` matches until #1171 is fixed), from the Rust primitives `i32`, `i64`, `bool`, `f32` and `f64`; the Go SDK's `encrypt/eql` table marks them producible. Stack Encrypt also produces the number equality types `IntegerEq`, `BigintEq`, `RealEq` and `DoubleEq`, whose `=` compares as PostgreSQL compares the plain types: an integer has one term whatever its width, and for floats `-0.0 = 0.0` and `NaN = NaN` (ADR-0002's semantic baseline). Every type it cannot produce yet names the issue that tracks its blocker. The by-name target entry points (`eql_bindings::encryption::targets::encrypt`, `decrypt` and `query`) exchange the plaintext as Stack Encrypt's `dynamic::ValueFrame`, one value as its transport frame, rather than a value tree. They, and the generated `v4::targets` table, need the new `targets` feature, which turns on `stack-encrypt` and Stack Encrypt's `dynamic` feature. `stack-encrypt` alone no longer builds Stack Encrypt's `dynamic` layer, so a Rust program using the typed v4 types through `encrypt_as` doesn't compile it.
66

7-
**The EQL v4 SQL bundle has range query domains and `eql_v4.between`, so a `BETWEEN` query is one operand.** The v4 bundle (`cipherstash-encrypt-v4.sql`) is built from this source and reaches users as the `eql-v4-install.sql` attached to each Go release; neither the npm package nor the crate ships it, and the v3 SQL they ship does not change. Every ordered family has one range query domain per order term, beside its single-bound ones: `eql_v4.query_integer_range_ore` beside `eql_v4.query_integer_ord_ore`, `eql_v4.query_integer_range_ope` beside `eql_v4.query_integer_ord_ope`, and the same for `smallint`, `bigint`, `date`, `timestamp`, `numeric`, `text`, `real` and `double`. A range is `{"v": 4, "i": …, "lo": <term>, "hi": <term>}`: each bound is the single-bound domain's `ob` (ORE) or `op` (OPE) value, in a `{v, i, lo, hi}` envelope. Stack Encrypt does not build that envelope yet; it will need to when the ordered types become producible. `eql_v4.between(column, range)` is true when the column's order term lies between the bounds, both included, and has an overload for every ordered column domain with the range of the term it stores (`_ord_ore` and `_search_ore` take the ORE range; `_ord`, `_ord_ope` and `_search` the OPE one). It is an inlinable SQL function, so a btree on the column's order term (`eql_v4.ord_term_ore(col)` or `eql_v4.ord_term(col)`) serves it as one index range scan. Write it schema-qualified (`BETWEEN` is an SQL keyword, so a bare `between(...)` does not parse), and cast the range to its exact domain (`$1::eql_v4.query_integer_range_ore`): an uncast `$1`, a `jsonb` parameter, or a range of another term or family fails with SQLSTATE 42725 ("function … is not unique"), unlike the single-bound operators, which resolve an untyped `$1`. In the crate, `eql_bindings::v4` (`stack-encrypt` feature) has a type for each range domain (`IntegerRangeOreQuery`, `TextRangeOpeQuery`, …, listed by `v4::all_range_query()`). EQL v3 has no range domains: its SQL and types are unchanged (ADR-0009, decision 7).
7+
**The EQL v4 SQL bundle has range query domains and `eql_v4.between`, so a `BETWEEN` query is one operand.** The v4 bundle (`cipherstash-encrypt-v4.sql`) is built from this source and reaches users as the `eql-v4-install.sql` attached to each Go release; neither the npm package nor the crate ships it, and the v3 SQL they ship does not change. Every ordered family has one range query domain per order term, beside its single-bound ones: `eql_v4.query_integer_range_ore` beside `eql_v4.query_integer_ord_ore`, `eql_v4.query_integer_range_ope` beside `eql_v4.query_integer_ord_ope`, and the same for `smallint`, `bigint`, `date`, `timestamp`, `numeric`, `text`, `real` and `double`. A range is `{"v": 4, "i": …, "lo": <term>, "hi": <term>}`: each bound is the single-bound domain's `ob` (ORE) or `op` (OPE) value, in a `{v, i, lo, hi}` envelope. Stack Encrypt does not build that envelope yet; it will need to when the ordered types become producible. `eql_v4.between(column, range)` is true when the column's order term lies between the bounds, both included, and has an overload for every ordered column domain with the range of the term it stores (`_ord_ore` and `_search_ore` take the ORE range; `_ord`, `_ord_ope` and `_search` the OPE one). It is an inlinable SQL function, so a btree on the column's order term (`eql_v4.ord_term_ore(col)` or `eql_v4.ord_term(col)`) serves it as one index range scan. Write it schema-qualified (`BETWEEN` is an SQL keyword, so a bare `between(...)` does not parse), and cast the range to its exact domain (`$1::eql_v4.query_integer_range_ore`): an uncast `$1`, a `jsonb` parameter, or a range of another term or family fails with SQLSTATE 42725 ("function … is not unique"), unlike the single-bound operators, which resolve an untyped `$1`. An ORE range scan needs the `ore_block_256` btree operator class, which the installer creates only as a superuser; on an install without it (Supabase and other non-superuser roles), the ORE range domains are disabled like the single-bound ORE domains, and a value cast to one raises `feature_not_supported` naming the family's OPE range. In the crate, `eql_bindings::v4` (`stack-encrypt` feature) has a type for each range domain (`IntegerRangeOreQuery`, `TextRangeOpeQuery`, …, listed by `v4::all_range_query()`). EQL v3 has no range domains: its SQL and types are unchanged (ADR-0009, decision 7).
88

99
Nothing a released version shipped has changed. The v3 types, the v3 SQL and the TypeScript package are as they were; the `stack-encrypt` derives on the v3 text types and the `v3::targets` module were added after the last release and are moved to v4 here, not removed from a release.

‎packages/eql/crates/eql-codegen/src/range.rs‎

Lines changed: 145 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@
3131
use eql_domains::{DomainFamily, Term, RANGE_BOUND_KEYS};
3232

3333
use crate::consts::{sql_str, INTERNAL_SCHEMA};
34-
use crate::context::{domain_name, environment};
34+
use crate::context::{domain_name, environment, query_domain_name};
3535
use crate::render::{EqlVersion, ONLY_DIRECTIVE};
3636

3737
/// The version the range surface belongs to.
@@ -94,23 +94,59 @@ struct RangeDomainBlock {
9494
comment: String,
9595
}
9696

97+
/// An ORE range domain the file disables when the install has no ORE
98+
/// operator class, as `ore_fallback.sql` disables the single-bound ORE
99+
/// domains: `name` for the identifier position, `name_literal` sql_str-escaped
100+
/// for the poison error's message, `alternatives` for its hint.
101+
#[derive(serde::Serialize)]
102+
struct OreRangeFallback {
103+
name: String,
104+
name_literal: String,
105+
alternatives: String,
106+
}
107+
97108
#[derive(serde::Serialize)]
98109
struct RangeTypesContext {
99110
only_directive: String,
100111
family_name: String,
112+
requires: Vec<String>,
101113
domains: Vec<RangeDomainBlock>,
114+
ore_ranges: Vec<OreRangeFallback>,
115+
}
116+
117+
/// The file whose poison function an ORE range's fallback calls.
118+
const ORE_FALLBACK: &str = "src/v3/scalars/ore_fallback.sql";
119+
120+
/// The poison hint for an ORE range: the family's OPE range, which needs no
121+
/// operator class. Every family with an ORE range has one today; a future
122+
/// family without must still render a sentence.
123+
fn ore_range_alternatives(spec: &DomainFamily) -> String {
124+
if spec.range_terms().contains(&Term::Ope) {
125+
format!(
126+
"{} (range)",
127+
query_domain_name(&spec.range_query_name(Term::Ope))
128+
)
129+
} else {
130+
"a non-ORE encrypted domain".to_string()
131+
}
102132
}
103133

104134
/// Body of `query_<T>_range_types.sql`, or `None` for a family with no
105135
/// ordering term.
136+
///
137+
/// An ORE range domain is disabled on an install without the ORE operator
138+
/// class, as the single-bound ORE domains are. `ore_fallback.sql` cannot do
139+
/// it, since it is shared with EQL v3 and the range domains are v4 only, so
140+
/// this file adds the same NOT VALID poison CHECK to its own ORE range,
141+
/// calling the function `ore_fallback.sql` creates under the same condition.
106142
pub fn render_range_types_file(spec: &DomainFamily) -> Option<String> {
107143
let terms = spec.range_terms();
108144
if terms.is_empty() {
109145
return None;
110146
}
111147
let domains = terms
112-
.into_iter()
113-
.map(|term| {
148+
.iter()
149+
.map(|&term| {
114150
let name = spec.range_query_name(term);
115151
RangeDomainBlock {
116152
typname: sql_str(&name),
@@ -129,10 +165,28 @@ pub fn render_range_types_file(spec: &DomainFamily) -> Option<String> {
129165
}
130166
})
131167
.collect();
168+
let ore_ranges: Vec<OreRangeFallback> = terms
169+
.iter()
170+
.filter(|&&t| t == Term::Ore)
171+
.map(|&t| {
172+
let name = query_domain_name(&spec.range_query_name(t));
173+
OreRangeFallback {
174+
name_literal: sql_str(&name),
175+
name,
176+
alternatives: sql_str(&ore_range_alternatives(spec)),
177+
}
178+
})
179+
.collect();
180+
let mut requires = vec!["src/v3/schema.sql".to_string()];
181+
if !ore_ranges.is_empty() {
182+
requires.push(ORE_FALLBACK.to_string());
183+
}
132184
let ctx = RangeTypesContext {
133185
only_directive: only_directive(),
134186
family_name: spec.name.to_string(),
187+
requires,
135188
domains,
189+
ore_ranges,
136190
};
137191
Some(
138192
environment()
@@ -161,6 +215,10 @@ struct Between {
161215
/// An ORE range, whose index needs the opclass a non-superuser
162216
/// install skips: the generated docs say so.
163217
ore: bool,
218+
/// For an ORE range, the family's OPE column domains and OPE range,
219+
/// which the generated docs point to instead. Empty otherwise.
220+
ope_columns: String,
221+
ope_range: String,
164222
}
165223

166224
#[derive(serde::Serialize)]
@@ -196,6 +254,17 @@ pub fn render_range_functions_file(spec: &DomainFamily) -> Option<String> {
196254
];
197255
let mut accessors = Vec::new();
198256
let mut betweens = Vec::new();
257+
let ope_columns = spec
258+
.range_columns(Term::Ope)
259+
.iter()
260+
.map(|d| format!("`{}`", domain_name(&d.full_name(family_name))))
261+
.collect::<Vec<_>>()
262+
.join(", ");
263+
let ope_range = if terms.contains(&Term::Ope) {
264+
format!("`{}`", query_domain_name(&spec.range_query_name(Term::Ope)))
265+
} else {
266+
String::new()
267+
};
199268
for &term in &terms {
200269
let range = spec.range_query_name(term);
201270
let returns = format!("{INTERNAL_SCHEMA}.{}", term.ctor());
@@ -228,6 +297,16 @@ pub fn render_range_functions_file(spec: &DomainFamily) -> Option<String> {
228297
range: range.clone(),
229298
extractor: term.extractor(),
230299
ore: term == Term::Ore,
300+
ope_columns: if term == Term::Ore {
301+
ope_columns.clone()
302+
} else {
303+
String::new()
304+
},
305+
ope_range: if term == Term::Ore {
306+
ope_range.clone()
307+
} else {
308+
String::new()
309+
},
231310
});
232311
}
233312
}
@@ -445,6 +524,69 @@ mod tests {
445524
}
446525
}
447526

527+
/// The ORE note names the family's own OPE columns and OPE range, not a
528+
/// fixed list: only text has a `_search` domain.
529+
#[test]
530+
fn the_ore_note_names_the_familys_own_ope_columns() {
531+
let integer = render_range_functions_file(integer()).unwrap();
532+
assert!(
533+
integer.contains(
534+
"use an OPE column (`public.{{prefix}}_integer_ord`, \
535+
`public.{{prefix}}_integer_ord_ope`)\n--! and `{{prefix}}.query_integer_range_ope`"
536+
),
537+
"{integer}"
538+
);
539+
assert!(!integer.contains("_search"), "{integer}");
540+
let text = render_range_functions_file(&TEXT).unwrap();
541+
assert!(text.contains("`public.{{prefix}}_text_search`"), "{text}");
542+
}
543+
544+
/// An ORE range is disabled where the single-bound ORE domains are: the
545+
/// file requires `ore_fallback.sql`, whose poison function it calls, and
546+
/// adds the poison CHECK to the ORE range alone, behind the same opclass
547+
/// check, with the OPE range as the hint.
548+
#[test]
549+
fn the_ore_range_is_disabled_without_the_ore_opclass() {
550+
for f in CATALOG.iter().filter(|f| !f.range_terms().is_empty()) {
551+
let body = render_range_types_file(f).unwrap();
552+
let ore = f.range_terms().contains(&Term::Ore);
553+
assert_eq!(
554+
body.contains("-- REQUIRE: src/v3/scalars/ore_fallback.sql"),
555+
ore,
556+
"{}",
557+
f.name
558+
);
559+
let v4 = render_str(&body, EqlVersion::V4).unwrap();
560+
let alters: Vec<&str> = v4
561+
.lines()
562+
.filter(|l| l.contains("ADD CONSTRAINT eql_ore_unavailable"))
563+
.collect();
564+
if !ore {
565+
assert!(alters.is_empty(), "{}", f.name);
566+
continue;
567+
}
568+
let name = format!("eql_v4.query_{}_range_ore", f.name);
569+
assert_eq!(
570+
alters,
571+
[format!(
572+
" ALTER DOMAIN {name} ADD CONSTRAINT eql_ore_unavailable"
573+
)],
574+
"{}",
575+
f.name
576+
);
577+
assert!(
578+
v4.contains(&format!(
579+
"CHECK (eql_v4_internal.ore_domain_unavailable(VALUE, '{name}', \
580+
'eql_v4.query_{}_range_ope (range)')) NOT VALID;",
581+
f.name
582+
)),
583+
"{v4}"
584+
);
585+
assert!(v4
586+
.contains("AND c.opcintype = 'eql_v4_internal.ore_block_256'::pg_catalog.regtype"));
587+
}
588+
}
589+
448590
#[test]
449591
fn every_function_is_documented() {
450592
for f in CATALOG.iter().filter(|f| !f.range_terms().is_empty()) {

‎packages/eql/crates/eql-codegen/templates/range_functions.sql.j2‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -43,8 +43,9 @@ AS $$ SELECT {{ a.body }} $$;
4343
--! serves as one range scan.{% if b.ore %} For this ORE range that index needs the
4444
--! default btree operator class on {{ internal_schema }}.ore_block_256, which the
4545
--! installer creates only with superuser privileges. Without it (on Supabase,
46-
--! for instance) the installer disables the ORE column domains, so use an OPE
47-
--! column (`_ord`, `_ord_ope`, `_search`) and its OPE range, which index everywhere.{% endif %}
46+
--! for instance) the installer disables the ORE column domains and this ORE range,
47+
--! so use an OPE column ({{ b.ope_columns }})
48+
--! and {{ b.ope_range }}, which index everywhere.{% endif %}
4849
--! @note `BETWEEN` is an SQL keyword, so call it schema-qualified:
4950
--! `{{ schema }}.between(...)` parses, a bare `between(...)` does not.
5051
--! @note Cast the range to its exact domain, `$1::{{ schema }}.{{ b.range }}`.

‎packages/eql/crates/eql-codegen/templates/range_types.sql.j2‎

Lines changed: 37 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
11
-- AUTOMATICALLY GENERATED FILE.
22
{{ only_directive }}
3-
-- REQUIRE: src/v3/schema.sql
4-
3+
{% for r in requires -%}
4+
-- REQUIRE: {{ r }}
5+
{% endfor %}
56
--! @file v3/scalars/{{ family_name }}/query_{{ family_name }}_range_types.sql
67
--! @brief Range query-operand domains for {{ family_name }}: the operand of `{{ schema }}.between`.
78
--! @note A range holds both bounds of one `BETWEEN` as one value:
@@ -17,9 +18,14 @@
1718
--! @note EQL version {{ eql_version }} only: the `-- ONLY EQL VERSION:` line above keeps this
1819
--! file out of every other version's installer.
1920
--! @note Like every query operand, a range lives in `{{ schema }}`, not `public`,
20-
--! and holds no ciphertext `c`. The client refuses a range whose low
21-
--! bound is above its high bound; one that reaches the database anyway
22-
--! matches no row.
21+
--! and holds no ciphertext `c`. A range whose low bound is above its
22+
--! high bound matches no row, as with SQL `BETWEEN`; the SQL does not
23+
--! reorder the bounds.{% if ore_ranges %}
24+
--! @note On an install without the ORE operator class (a non-superuser
25+
--! install, on Supabase for instance), the ORE range domain is disabled
26+
--! as the single-bound ORE domains are: a value cast to it raises
27+
--! `feature_not_supported`, naming the OPE range. See
28+
--! `v3/scalars/ore_fallback.sql`.{% endif %}
2329

2430
DO $$
2531
BEGIN
@@ -47,3 +53,29 @@ BEGIN
4753
{% endfor -%}
4854
END
4955
$$;
56+
{%- if ore_ranges %}
57+
58+
-- Without the default btree operator class on {{ internal_schema }}.ore_block_256
59+
-- (the installer skips it on insufficient_privilege), ore_fallback.sql has
60+
-- created {{ internal_schema }}.ore_domain_unavailable and poisoned the
61+
-- single-bound ORE domains. It cannot reach this ORE range, which exists in
62+
-- EQL v4 only, so the same NOT VALID poison CHECK is added here.
63+
DO $do$
64+
BEGIN
65+
IF EXISTS (
66+
SELECT 1
67+
FROM pg_catalog.pg_opclass c
68+
JOIN pg_catalog.pg_am am ON am.oid = c.opcmethod
69+
WHERE am.amname = 'btree'
70+
AND c.opcdefault
71+
AND c.opcintype = '{{ internal_schema }}.ore_block_256'::pg_catalog.regtype
72+
) THEN
73+
RETURN;
74+
END IF;
75+
{% for e in ore_ranges %}
76+
ALTER DOMAIN {{ e.name }} ADD CONSTRAINT eql_ore_unavailable
77+
CHECK ({{ internal_schema }}.ore_domain_unavailable(VALUE, '{{ e.name_literal }}', '{{ e.alternatives }}')) NOT VALID;
78+
{%- endfor %}
79+
END;
80+
$do$;
81+
{%- endif %}

0 commit comments

Comments
 (0)