Skip to content

sql: preserve filters during distinct preprocessing - #174721

Open
Alignyx wants to merge 1 commit into
cockroachdb:masterfrom
Alignyx:fix-158770-filtered-distinct-placeholder
Open

sql: preserve filters during distinct preprocessing#174721
Alignyx wants to merge 1 commit into
cockroachdb:masterfrom
Alignyx:fix-158770-filtered-distinct-placeholder

Conversation

@Alignyx

@Alignyx Alignyx commented Sep 5, 2026

Copy link
Copy Markdown

Fixes #158770.

Root cause

When every aggregate in a stage is DISTINCT, physical planning adds an early distinct processor keyed by the union of the aggregate argument columns. Filter columns were not included in that key. Rows with the same aggregate argument but different filter results could therefore be collapsed before each aggregate evaluated its own filter, causing multiple filtered DISTINCT aggregates over the same argument to observe the wrong subset of rows. Placeholders made the issue visible by preventing optimizer deduplication, but they were not the source of the error; two different literal filters failed in the same way.

How I tracked it down

I compared the reported prepared query with direct literal filters, reordered aggregates, a single filtered DISTINCT aggregate, non-DISTINCT controls, identical filters, different aggregate arguments, and COUNT plus SUM over the same DISTINCT argument. Row/vectorized and local/forced-distributed execution agreed on the failure, which placed it before either aggregation implementation. The physical plan contained both aggregate specs and both Boolean filter projections, but the shared all-DISTINCT preprocessing stage deduplicated only on the argument column. For a repeated argument, that stage could retain a false-filter row and discard the true-filter row before either aggregate ran.

Fix

The early distinct key now includes every non-nil aggregate filter column in addition to the existing union of argument columns. This preserves rows that are distinguishable by any later filter while retaining the shared preprocessing optimization and leaving individual aggregate implementations unchanged.

Test coverage

  • Added a direct SQL regression with two literal filters over the same DISTINCT argument, expecting counts 2,1.
  • Added a prepared regression with one literal and one parameterized filter, covering both equal thresholds (2,2) and different thresholds (2,1).
  • Verified the focused regression in five local/fake-distributed row, vectorized, and disk-spilling configurations; ran the complete aggregate logic suite locally and fake-distributed, the full 16-shard SQL test target, and the five-node distributed aggregation suite containing the earlier distinct-preprocessing regression.

When every aggregate is distinct, physical planning adds an early distinct
stage keyed by the union of aggregate argument columns. For filtered
aggregates, this can discard rows that have equal arguments but different
filter results before each aggregate applies its own filter.

Include every aggregate filter column in the early distinct key. Add direct
and prepared SQL regressions with multiple filtered distinct aggregates.

Fixes cockroachdb#158770

Release note (bug fix): Fixed incorrect results from multiple filtered
DISTINCT aggregates over the same argument.
@Alignyx
Alignyx requested a review from a team as a code owner September 5, 2026 01:24
@Alignyx
Alignyx requested review from mw5h and removed request for a team September 5, 2026 01:24
@blathers-crl

blathers-crl Bot commented Sep 5, 2026

Copy link
Copy Markdown

Thank you for contributing to CockroachDB. Please ensure you have followed the guidelines for creating a PR.

My owl senses detect your PR is good for review. Please keep an eye out for any test failures in CI.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@blathers-crl blathers-crl Bot added the O-community Originated from the community label Sep 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

O-community Originated from the community

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Wrong result from COUNT(DISTINCT ..) FILTER (..) aggregate function

1 participant