bench(compress): run the full dataset suite under --gpu-decompress - #9148
Closed
joseph-isaacs wants to merge 3 commits into
Closed
bench(compress): run the full dataset suite under --gpu-decompress#9148joseph-isaacs wants to merge 3 commits into
joseph-isaacs wants to merge 3 commits into
Conversation
The GPU decompress suite is allow-listed to a single benchmark name, while the CPU suite runs thirteen datasets. Promoting one requires verifying its CUDA decode end to end, but the allow-list filter ran before the `--datasets` regex, so `--gpu-decompress --datasets taxi` selected nothing and there was no way to run a candidate dataset on GPU in the first place. An explicit `--datasets` filter now overrides the allow-list. Unfiltered runs still use the allow-list, so CI behaviour is unchanged. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
GPU decompression was allow-listed to a single benchmark name while the CPU suite runs fifteen. Removes the allow-list so --gpu-decompress covers the same datasets as a CPU run. Not every dataset decodes on GPU yet. A dataset works only if every encoding only_cuda_compatible() selects for it has a kernel registered in initialize_cuda; there is no CPU fallback, an unregistered encoding is a hard error. The `wide table` datasets encode their columns as vortex.list, which has no kernel, so those six fail with "No CUDA kernel for encoding vortex.list". --datasets narrows the run to the datasets that decode. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Sets FLAT_LAYOUT_INLINE_ARRAY_NODE to "0" so the GPU compression benchmark writes the array node out of line. The flat layout treats the variable as enabled only when it is exactly "1", so this restores the pre-inline-node behaviour for this job. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Contributor
BENCHMARK FAILEDGPU Compression failed. Check the workflow run for details. |
Contributor
|
This PR has been marked as stale because it has been open for 14 days with no activity. Please comment or remove the stale label if you wish to keep it active, otherwise it will be closed in 7 days |
Contributor
|
This PR was closed because it has been inactive for 7 days since being marked as stale. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale for this change
compress-benchruns fifteen datasets on CPU but the GPU decompression path was allow-listed to a single benchmark name,TPC-H l_comment canonical. The comment above the allow-list said a name goes in only after its CUDA compress and decompress have been verified end to end — but the allow-list filter ran before the--datasetsregex, so--gpu-decompress --datasets taxiselected zero benchmarks. There was no way to run a candidate dataset on GPU to do the verification that would promote it.This opens the GPU path up to the whole suite so we can find out where each dataset actually stands.
What changes are included in this PR?
gpu_decompress_benchmarksallow-list, so--gpu-decompresscovers the same datasets as a CPU run.FLAT_LAYOUT_INLINE_ARRAY_NODEto"0"ingpu-compress-bench-pr.yml, so the job writes the array node out of line. The flat layout treats the variable as enabled only when it is exactly"1".Expected failures
Not every dataset decodes on GPU yet. A dataset works only if every encoding
only_cuda_compatible()selects for it has a kernel registered ininitialize_cuda— there is no CPU fallback, an unregistered encoding is a hard error fromhybrid_dispatch.Encoding the
wide tabledatasets through the GPU write path locally gives columns ofvortex.list, withvortex.primitiveelements andvortex.sequenceoffsets.vortex.listhas no registered kernel, so those six benchmarks are expected to fail withNo CUDA kernel for encoding vortex.list. Sincebenchmark_decompresspropagates with?, the first failure aborts the run — this PR is a draft so we can see how far the GPU job gets before deciding whether to skip-and-report instead of aborting.The remaining datasets (
taxi, the six Public BI tables,TPC-H l_comment chunked) are untested; their data could not be fetched locally.Separately,
string::NullDominatedSparseScheme(vortex.string.sparse) is missing from theonly_cuda_compatible()exclusion list even though it producesSparse, which has no CUDA kernel. The integer and float variants are both excluded. Null-heavy string columns are exactly what the Public BI tables have, so this is a likely second failure source. Not fixed here.What APIs are changed? Are there any user-facing changes?
No API changes. Benchmark-only:
--gpu-decompressnow runs the full suite instead of one dataset, and the GPU CI job runs with the inline flat-layout array node disabled.Generated by Claude Code