Skip to content

Commit 2d5be82

Browse files
rustyconoverclaude
andcommitted
test(filter-v2): cover the bare boolean predicate where it can actually fail
The fix alone is not a guard. Two places had to gain a case, and neither was the obvious one. `bool_filter_echo` is a table function with a nullable BOOLEAN column (`flag` cycling TRUE, FALSE, NULL) that echoes the SQL rendering of what was pushed. It exists because nothing else could express the shape: - `filter_echo` has no boolean column, so `WHERE flag` cannot be written against it at all. - The table-in-out `example.echo` path is not handed a bare boolean column as a pushed predicate. Cases written there pass whether or not the worker can decode one -- I checked, by reverting the fix and watching them stay green. A test that cannot fail is worse than no test. It takes a TABLE FUNCTION with a boolean column to see the shape at all, which is how a filter-pushdown suite of 348 files -- including one dedicated to booleans, added to close a boolean coverage gap -- missed a decoder that refused every `WHERE flag` there is. Echoing `pushed_filters` covers both halves: the rows prove the decode, and the rendering proves `to_sql()` emits something. A shape that parses but renders no SQL reads as "(none)" there, and for a worker that builds a WHERE clause from it that is silently wrong rows -- DuckDB does not re-apply a predicate it pushed into a table function. The conformance corpus gains `structural.boolean-column-predicate`, a positive case for a bare `column_ref` root and for `not` over one. That is the cross-port contract, and its absence is why this is not a Python bug so much as a Python *sighting*: every port's decoder is free to have the same gap, and none had a case that would say so. `bool_filter_echo` is excluded from function_registration.test's portable baseline for now -- counting it would fail every SDK fixture worker that has not shipped it. It belongs in the baseline (167) once they do. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 2b73435 commit 2d5be82

5 files changed

Lines changed: 164 additions & 0 deletions

File tree

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
{
2+
"encoding": "vgi.filters.v2",
3+
"semantics": "vgi.duckdb.standard.v1",
4+
"kind": "snapshot",
5+
"predicates": [
6+
{
7+
"id": "query:bare-boolean-column",
8+
"revision": 0,
9+
"mode": "required",
10+
"source": "query",
11+
"expression": {
12+
"node": "column_ref",
13+
"column_index": 0,
14+
"column_name": "flag"
15+
}
16+
},
17+
{
18+
"id": "query:negated-boolean-column",
19+
"revision": 0,
20+
"mode": "required",
21+
"source": "query",
22+
"expression": {
23+
"node": "not",
24+
"expression": {
25+
"node": "column_ref",
26+
"column_index": 1,
27+
"column_name": "other_flag"
28+
}
29+
}
30+
}
31+
]
32+
}

‎conformance/filter-v2/manifest.json‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,11 @@
3030
"document": "cases/positive/runtime-filter-snapshot.json",
3131
"expected": "valid"
3232
},
33+
{
34+
"id": "structural.boolean-column-predicate",
35+
"document": "cases/positive/boolean-column-predicate.json",
36+
"expected": "valid"
37+
},
3338
{
3439
"id": "structural.and-too-few-children",
3540
"document": "cases/negative/and-too-few-children.json",

‎vgi/_test_fixtures/table/__init__.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,7 @@
6565
ProjectsScanFunction,
6666
)
6767
from vgi._test_fixtures.table.filters import (
68+
BoolFilterEchoFunction,
6869
DictFilterEchoFunction,
6970
DynamicFilterEchoFunction,
7071
ExpressionFilterTestFunction,
@@ -216,6 +217,7 @@
216217
"ConstantColumnsFunction",
217218
"CountryPartitionedSalesFunction",
218219
"DepartmentsScanFunction",
220+
"BoolFilterEchoFunction",
219221
"DictFilterEchoFunction",
220222
"DisjointRangePartitionedFunction",
221223
"DoubleSequenceFunction",

‎vgi/_test_fixtures/table/filters.py‎

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -191,6 +191,129 @@ def process(
191191
state.remaining -= size
192192

193193

194+
# ============================================================================
195+
# BoolFilterEchoFunction — a table function with a BOOLEAN column, so the
196+
# bare-column predicate shapes (`WHERE flag`, `WHERE NOT flag`) can be driven
197+
# end to end.
198+
#
199+
# filter_echo cannot cover these: its schema has no boolean column, so the
200+
# predicate cannot even be written. The table-in-out `echo` path cannot either
201+
# — DuckDB does not hand it a bare boolean column as a pushed predicate, so a
202+
# test written there passes whether or not the worker can decode one. It takes
203+
# a TABLE FUNCTION with a boolean column to see the shape at all, which is why
204+
# a decoder that refused every one of them ("predicate root must resolve to
205+
# BOOLEAN", vgi-python < 0.36.2) went unnoticed by a filter-pushdown suite of
206+
# 348 files including one dedicated to booleans.
207+
#
208+
# `pushed_filters` echoes the SQL rendering, so this covers both halves: the
209+
# decode (the rows are auto-applied and must be right) and `to_sql()` (a shape
210+
# that parses but renders nothing would show up here as "(none)").
211+
# ============================================================================
212+
213+
214+
@dataclass(slots=True, frozen=True)
215+
class _BoolFilterEchoArgs:
216+
"""Arguments for BoolFilterEchoFunction."""
217+
218+
count: Annotated[int, Arg(0, doc="Number of rows to generate", ge=0)]
219+
220+
221+
@dataclass(kw_only=True)
222+
class _BoolFilterEchoState(ArrowSerializableDataclass):
223+
"""Remaining rows plus the echoed SQL rendering of the pushed filters."""
224+
225+
remaining: int
226+
current_index: int = 0
227+
filter_str: str = "(none)"
228+
229+
230+
@init_single_worker
231+
@bind_fixed_schema
232+
@_cardinality_from_count
233+
class BoolFilterEchoFunction(TableFunctionGenerator[_BoolFilterEchoArgs, _BoolFilterEchoState]):
234+
"""Rows with a nullable BOOLEAN column, echoing the pushed-down filters.
235+
236+
SCHEMA
237+
------
238+
Output: {"n": int64, "flag": bool, "pushed_filters": string}
239+
240+
``flag`` cycles TRUE, FALSE, NULL so every predicate is exercised against
241+
all three truth values -- the NULL row is what distinguishes ``WHERE flag``
242+
from ``WHERE flag IS NOT FALSE``, and ``WHERE NOT flag`` from
243+
``WHERE flag IS NOT TRUE``.
244+
245+
Attributes:
246+
FIXED_SCHEMA: The fixed Arrow output schema this function always produces.
247+
248+
"""
249+
250+
class Meta:
251+
"""Metadata for BoolFilterEchoFunction."""
252+
253+
name = "bool_filter_echo"
254+
description = "Rows with a nullable BOOLEAN column, echoing pushed-down filters"
255+
categories = ["generator", "diagnostic"]
256+
filter_pushdown = True
257+
auto_apply_filters = True
258+
projection_pushdown = True
259+
examples = [
260+
FunctionExample(
261+
sql="SELECT * FROM bool_filter_echo(6) WHERE flag",
262+
description="A boolean column is a predicate on its own",
263+
),
264+
]
265+
266+
FIXED_SCHEMA: ClassVar[pa.Schema] = schema({"n": pa.int64(), "flag": pa.bool_(), "pushed_filters": pa.utf8()})
267+
268+
@classmethod
269+
def initial_state(cls, params: ProcessParams[_BoolFilterEchoArgs]) -> _BoolFilterEchoState:
270+
"""Create initial state, caching the SQL rendering of the pushed filters."""
271+
assert params.init_call is not None
272+
pf = params.init_call.pushdown_filters
273+
filters = (
274+
cls.pushdown_filters(
275+
pf,
276+
join_keys=params.init_call.join_keys,
277+
output_schema=params.init_call.output_schema,
278+
)
279+
if pf is not None
280+
else None
281+
)
282+
return _BoolFilterEchoState(
283+
remaining=params.args.count,
284+
filter_str=_format_pushed_filters(filters),
285+
)
286+
287+
@classmethod
288+
def process(
289+
cls,
290+
params: ProcessParams[_BoolFilterEchoArgs],
291+
state: _BoolFilterEchoState,
292+
out: OutputCollector,
293+
) -> None:
294+
"""Emit rows whose ``flag`` cycles TRUE, FALSE, NULL."""
295+
if state.remaining <= 0:
296+
out.finish()
297+
return
298+
299+
n_values = list(range(state.current_index, state.current_index + state.remaining))
300+
flags: list[bool | None] = [(True, False, None)[i % 3] for i in n_values]
301+
302+
out.emit(
303+
pa.RecordBatch.from_pydict(
304+
{
305+
"n": n_values,
306+
"flag": flags,
307+
"pushed_filters": [state.filter_str] * len(n_values),
308+
},
309+
schema=params.output_schema,
310+
)
311+
)
312+
313+
state.current_index += state.remaining
314+
state.remaining = 0
315+
316+
194317
# ============================================================================
195318
# ValuePruneFunction — exercises PushdownFilters.get_column_values('n'), the
196319
# partition-pruning idiom (resolve the discrete value set up front, fetch only

‎vgi/_test_fixtures/worker.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -140,6 +140,7 @@
140140
RFF_SIMPLE_COLUMNS,
141141
RFF_STRUCT_COLUMNS,
142142
BatchIndexOverflowFunction,
143+
BoolFilterEchoFunction,
143144
BrokenMissingPartitionValuesFunction,
144145
BrokenPartitionColumnAbsentFromBatchFunction,
145146
BrokenPartitionMinNeqMaxFunction,
@@ -486,6 +487,7 @@ def _build_fixture_stats() -> tuple[
486487
# TableFunctionGenerator - generate output without input
487488
ConstantColumnsFunction,
488489
SlowCancellableFunction,
490+
BoolFilterEchoFunction,
489491
FilterEchoFunction,
490492
FilterEchoPartitionedFunction,
491493
FilterEchoTableScanFunction,

0 commit comments

Comments
 (0)