Skip to content

Canonicalize chunked nested types through the builder [builders-child-stack] - #8967

Open
robert3005 wants to merge 5 commits into
developfrom
claude/chunked-canonical-via-builder-9ze0t6
Open

Canonicalize chunked nested types through the builder [builders-child-stack]#8967
robert3005 wants to merge 5 commits into
developfrom
claude/chunked-canonical-via-builder-9ze0t6

Conversation

@robert3005

@robert3005 robert3005 commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Always use builders when canonicalizing chunked arrays

@codspeed-hq

codspeed-hq Bot commented Jul 25, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 18.45%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

❌ 8 regressed benchmarks
✅ 1973 untouched benchmarks
⏩ 42 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation canonicalize[1024, 32] 56.2 µs 75.1 µs -25.18%
Simulation canonicalize[256, 32] 56.3 µs 74.5 µs -24.44%
Simulation canonicalize[16, 32] 58.3 µs 74.9 µs -22.12%
Simulation canonicalize[1024, 8] 43.8 µs 54.4 µs -19.52%
Simulation canonicalize[256, 8] 43.8 µs 53.1 µs -17.64%
Simulation canonicalize[16, 8] 47 µs 55.1 µs -14.68%
WallTime words_gather_scalar[65536] 8.2 µs 9.4 µs -12.39%
Simulation canonicalize[1024, 2] 41.5 µs 46.3 µs -10.29%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing claude/chunked-canonical-via-builder-9ze0t6 (0746486) with develop (adfdd77)2

Open in CodSpeed

Footnotes

  1. 42 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on develop (b052a06) during the generation of this report, so adfdd77 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@claude claude Bot changed the title refactor(array): canonicalize chunked structs and FSLs through the builder refactor(array): canonicalize chunked structs and FSLs through the builder [builders-child-stack] Jul 25, 2026
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch 2 times, most recently from afed52f to 79f350a Compare July 29, 2026 07:35
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch 2 times, most recently from 1145029 to 8f39b06 Compare July 29, 2026 14:19
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch from 8f39b06 to 825ab2e Compare July 31, 2026 13:35
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch from 825ab2e to ce2eb1d Compare July 31, 2026 15:29
Comment thread vortex-array/src/arrays/chunked/vtable/canonical.rs Outdated
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch from ce2eb1d to cefbc8e Compare July 31, 2026 16:54
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch from cefbc8e to 0d4eb9a Compare July 31, 2026 17:22
@robert3005 robert3005 changed the title refactor(array): canonicalize chunked structs and FSLs through the builder [builders-child-stack] Canonicalize chunked nested types through the builder [builders-child-stack] Jul 31, 2026
@robert3005 robert3005 added the changelog/chore A trivial change label Jul 31, 2026
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch from 0d4eb9a to 1e8346a Compare July 31, 2026 18:39
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch from 1e8346a to 0431fa7 Compare July 31, 2026 18:43
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch 2 times, most recently from a9f7d31 to 9c8574c Compare July 31, 2026 22:45
@robert3005
robert3005 changed the base branch from claude/builders-lazy-validity-9ze0t6 to claude/constant-fast-paths-9ze0t6 July 31, 2026 22:47
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch from 9c8574c to 0b003e4 Compare August 1, 2026 09:43
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch 2 times, most recently from 876aa4e to a66422f Compare August 5, 2026 12:51
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch from a66422f to dac04cb Compare August 7, 2026 10:40
@robert3005
robert3005 changed the base branch from claude/constant-fast-paths-9ze0t6 to claude/builders-bulk-loop-callers-9ze0t6 August 7, 2026 11:09
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch from dac04cb to 3f93504 Compare August 7, 2026 11:14
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch 2 times, most recently from 697f31a to 5f173bf Compare August 7, 2026 14:36
Base automatically changed from claude/builders-bulk-loop-callers-9ze0t6 to develop August 20, 2026 12:26
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch from 5f173bf to 15bd75e Compare August 20, 2026 12:26
`pack_struct_chunks`, `swizzle_fixed_size_list_chunks` and
`swizzle_list_chunks` existed because `append_to_builder` used to decode a
builder's children: without them, canonicalizing a `ChunkedArray` would
concatenate every chunk's children instead of reusing them. Their doc
comments describe what the builders now do on their own, so all three are
dead weight - the generic builder path produces the same swizzled array,
with each chunk's children kept as chunks of the combined child.

Only `Variant` still needs a hand-written pack, because there is no variant
builder.

This drops the one canonicalization path that allocated its buffers through
the session allocator: `swizzle_list_chunks` allocated its offsets and sizes
with `ctx.allocator()`, whereas `builder_with_capacity_in` still ignores the
allocator it is handed, so chunked primitives, structs and FSLs already went
around it. `list_canonicalize_uses_memory_session_allocator` guarded that
one path and goes with it; restoring the property means teaching the
builders to allocate through a `HostAllocatorRef`, not keeping a bespoke
list swizzle alive.

Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: Robert Kruszewski <robert@spiraldb.com>

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@robert3005
robert3005 force-pushed the claude/chunked-canonical-via-builder-9ze0t6 branch from 15bd75e to f01c3f9 Compare August 20, 2026 12:45
Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
I, Robert Kruszewski <github@robertk.io>, hereby add my Signed-off-by to this commit: 5c8401d

Signed-off-by: Robert Kruszewski <github@robertk.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/chore A trivial change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants