Skip to content

bench(compress): run the full dataset suite under --gpu-decompress - #9148

Closed
joseph-isaacs wants to merge 3 commits into
developfrom
claude/compression-benchmarks-cpu-gpu-66iuyw
Closed

bench(compress): run the full dataset suite under --gpu-decompress#9148
joseph-isaacs wants to merge 3 commits into
developfrom
claude/compression-benchmarks-cpu-gpu-66iuyw

Conversation

@joseph-isaacs

Copy link
Copy Markdown
Contributor

Rationale for this change

compress-bench runs fifteen datasets on CPU but the GPU decompression path was allow-listed to a single benchmark name, TPC-H l_comment canonical. The comment above the allow-list said a name goes in only after its CUDA compress and decompress have been verified end to end — but the allow-list filter ran before the --datasets regex, so --gpu-decompress --datasets taxi selected zero benchmarks. There was no way to run a candidate dataset on GPU to do the verification that would promote it.

This opens the GPU path up to the whole suite so we can find out where each dataset actually stands.

What changes are included in this PR?

  • Removes the gpu_decompress_benchmarks allow-list, so --gpu-decompress covers the same datasets as a CPU run.
  • Sets FLAT_LAYOUT_INLINE_ARRAY_NODE to "0" in gpu-compress-bench-pr.yml, so the job writes the array node out of line. The flat layout treats the variable as enabled only when it is exactly "1".
  • Updates the README to describe the new behaviour and how to narrow a run.

Expected failures

Not every dataset decodes on GPU yet. A dataset works only if every encoding only_cuda_compatible() selects for it has a kernel registered in initialize_cuda — there is no CPU fallback, an unregistered encoding is a hard error from hybrid_dispatch.

Encoding the wide table datasets through the GPU write path locally gives columns of vortex.list, with vortex.primitive elements and vortex.sequence offsets. vortex.list has no registered kernel, so those six benchmarks are expected to fail with No CUDA kernel for encoding vortex.list. Since benchmark_decompress propagates with ?, the first failure aborts the run — this PR is a draft so we can see how far the GPU job gets before deciding whether to skip-and-report instead of aborting.

The remaining datasets (taxi, the six Public BI tables, TPC-H l_comment chunked) are untested; their data could not be fetched locally.

Separately, string::NullDominatedSparseScheme (vortex.string.sparse) is missing from the only_cuda_compatible() exclusion list even though it produces Sparse, which has no CUDA kernel. The integer and float variants are both excluded. Null-heavy string columns are exactly what the Public BI tables have, so this is a likely second failure source. Not fixed here.

What APIs are changed? Are there any user-facing changes?

No API changes. Benchmark-only: --gpu-decompress now runs the full suite instead of one dataset, and the GPU CI job runs with the inline flat-layout array node disabled.


Generated by Claude Code

claude added 3 commits August 3, 2026 10:09
The GPU decompress suite is allow-listed to a single benchmark name, while the
CPU suite runs thirteen datasets. Promoting one requires verifying its CUDA
decode end to end, but the allow-list filter ran before the `--datasets` regex,
so `--gpu-decompress --datasets taxi` selected nothing and there was no way to
run a candidate dataset on GPU in the first place.

An explicit `--datasets` filter now overrides the allow-list. Unfiltered runs
still use the allow-list, so CI behaviour is unchanged.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
GPU decompression was allow-listed to a single benchmark name while the CPU suite
runs fifteen. Removes the allow-list so --gpu-decompress covers the same datasets
as a CPU run.

Not every dataset decodes on GPU yet. A dataset works only if every encoding
only_cuda_compatible() selects for it has a kernel registered in initialize_cuda;
there is no CPU fallback, an unregistered encoding is a hard error. The `wide
table` datasets encode their columns as vortex.list, which has no kernel, so
those six fail with "No CUDA kernel for encoding vortex.list". --datasets narrows
the run to the datasets that decode.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Sets FLAT_LAYOUT_INLINE_ARRAY_NODE to "0" so the GPU compression benchmark
writes the array node out of line. The flat layout treats the variable as
enabled only when it is exactly "1", so this restores the pre-inline-node
behaviour for this job.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@joseph-isaacs joseph-isaacs added the action/bench-gpu-compress Run only the GPU compression benchmark on this PR label Aug 3, 2026 — with Claude
@github-actions github-actions Bot removed the action/bench-gpu-compress Run only the GPU compression benchmark on this PR label Aug 3, 2026
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

BENCHMARK FAILED

GPU Compression failed. Check the workflow run for details.

@github-actions

Copy link
Copy Markdown
Contributor

This PR has been marked as stale because it has been open for 14 days with no activity. Please comment or remove the stale label if you wish to keep it active, otherwise it will be closed in 7 days

@github-actions github-actions Bot added the stale This PR is stale and will be auto-closed soon label Aug 21, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This PR was closed because it has been inactive for 7 days since being marked as stale.

@github-actions github-actions Bot closed this Aug 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stale This PR is stale and will be auto-closed soon

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants