Repository navigation
[ntuple] Use stored bit width when recompressing merged pages - #23752
Open
Elvand-Lie wants to merge 1 commit into
Open
Elvand-Lie wants to merge 1 commit into
Elvand-Lie wants to merge 1 commit into
Conversation
Apply the source column's stored bit width before computing packed page sizes during recompression. This allows truncated and quantized float pages to be merged with a different output compression. Compare both fields against their stored source values in a regression. Assisted-by: codex:gpt-6.1-sol Assisted-by: claude-code:opus
Contributor
|
Thanks. I created #23755 on top of your changes as a more general solution to the issue which should prevent this kind of bug going forward. |
Test Results 20 files 20 suites 2d 22h 24m 21s ⏱️ For more details on these failures, see this check. Results for commit ff9d65f. |
Contributor
Author
|
@silverweed Cool. Should I close this one, or would you rather merge it first? |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This Pull request:
Fix recompression of truncated and quantized float pages during RNTuple merging, including
hadd -f1when the inputs use a different compression.Changes or fixes:
Source column elements used their default bit width when calculating packed page sizes. Apply the column descriptor's stored bit width before recompression. The pages stay packed, so no value-range setup is needed.
Add one regression next to the compression tests: merge both 20-bit fields from compression 505 to 101 and compare every value against the source file's stored values.
Validation
Base: upstream master
7494cd79d74531ae35ca4831f0843b345865b3c9.zip buffer too short(exit 1).-j3; fullntuple_merger: 341 tests passed.hadd -f1 out.root in.root: master exits 1; fixed build exits 0, with all 1000 entries matching both stored source fields on readback.AI assistance: Codex (gpt-6.1-sol) implemented and tested this change using defect analysis from Claude Code (opus). Review stays with author before a push.
Checklist:
This PR fixes low-precision page recompression during RNTuple merging.