Skip to content

refactor(array): allocate execution outputs through context - #9671

Draft
gatesn wants to merge 3 commits into
ngates/buffer-allocator-buildersfrom
ngates/buffer-allocator-execution
Draft

refactor(array): allocate execution outputs through context#9671
gatesn wants to merge 3 commits into
ngates/buffer-allocator-buildersfrom
ngates/buffer-allocator-execution

Conversation

@gatesn

@gatesn gatesn commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Summary

Allocate execution outputs through the execution context.

Changes

  • Propagate allocators through filter, take, interleave, and patches.
  • Use context allocators in dictionary and scalar functions.
  • Add tests for allocator propagation.

@codspeed-hq

codspeed-hq Bot commented Aug 27, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 8.82%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
❌ 6 regressed benchmarks
✅ 2083 untouched benchmarks
⏩ 206 skipped benchmarks1
🗄️ 4 archived benchmarks run2

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation take_search[(0.005, 0.05)] 20 µs 23.1 µs -13.7%
Simulation take_search[(0.01, 0.05)] 20.7 µs 23.9 µs -13.26%
WallTime words_gather_scalar_avx2[65536] 8.3 µs 9.4 µs -12.05%
Simulation take_search[(0.005, 0.1)] 26.2 µs 29.4 µs -10.78%
Simulation take_search_chunked[(0.01, 0.05)] 31 µs 34.5 µs -10.16%
Simulation take_search_chunked[(0.005, 0.05)] 31 µs 34.5 µs -10.13%
Simulation encode_varbin[(2000, 2)] 483.8 µs 437.8 µs +10.51%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ngates/buffer-allocator-execution (cd00ce3) with ngates/buffer-allocator-builders (0ce40e7)

Open in CodSpeed

Footnotes

  1. 206 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. 4 benchmarks were run, but are now archived. If they were deleted in another branch, consider rebasing to remove them from the report. Instead if they were added back, click here to restore them.

@gatesn gatesn mentioned this pull request Aug 27, 2026
5 tasks
@gatesn
gatesn force-pushed the ngates/buffer-allocator-execution branch from f8000e4 to 9a8b653 Compare August 27, 2026 19:38
@gatesn
gatesn force-pushed the ngates/buffer-allocator-execution branch 2 times, most recently from 6931802 to ed2fbe7 Compare August 27, 2026 21:37
@gatesn
gatesn force-pushed the ngates/buffer-allocator-execution branch from ed2fbe7 to 3c868b7 Compare August 27, 2026 21:38
@gatesn
gatesn force-pushed the ngates/buffer-allocator-execution branch 2 times, most recently from 09d740d to 3e127b9 Compare August 28, 2026 14:56
@gatesn
gatesn force-pushed the ngates/buffer-allocator-execution branch 2 times, most recently from 8d4cbfb to 92397b9 Compare August 28, 2026 17:03
@gatesn
gatesn force-pushed the ngates/buffer-allocator-execution branch from 92397b9 to da81af8 Compare August 28, 2026 17:35
@gatesn
gatesn force-pushed the ngates/buffer-allocator-execution branch from da81af8 to 6210f3b Compare August 28, 2026 18:02
@gatesn
gatesn force-pushed the ngates/buffer-allocator-execution branch 2 times, most recently from 375111f to af6234b Compare August 28, 2026 18:24
@gatesn
gatesn force-pushed the ngates/buffer-allocator-execution branch from af6234b to aa6801b Compare August 28, 2026 20:41
@gatesn
gatesn force-pushed the ngates/buffer-allocator-execution branch 2 times, most recently from 903c310 to 2898fb0 Compare August 28, 2026 20:54
@gatesn gatesn added the changelog/feature A new feature label Aug 28, 2026
@gatesn
gatesn force-pushed the ngates/buffer-allocator-execution branch from 776037e to 8df467b Compare August 29, 2026 00:07
@codecov

codecov Bot commented Aug 29, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 81.06169% with 132 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.24%. Comparing base (4a232a6) to head (8df467b).

Files with missing lines Patch % Lines
...rray/src/arrays/filter/execute/take/fixed_width.rs 58.82% 21 Missing ⚠️
...ex-array/src/scalar_fn/fns/binary/compare/bytes.rs 75.60% 20 Missing ⚠️
vortex-array/src/scalar_fn/fns/like/mod.rs 73.91% 18 Missing ⚠️
...-array/src/scalar_fn/fns/binary/compare/decimal.rs 58.06% 13 Missing ⚠️
vortex-array/src/scalar_fn/fns/binary/boolean.rs 64.00% 9 Missing ⚠️
...ortex-array/src/arrays/filter/execute/take/rank.rs 58.82% 7 Missing ⚠️
vortex-array/src/arrays/filter/execute/take.rs 25.00% 6 Missing ⚠️
...x-array/src/arrays/interleave/execute/primitive.rs 75.00% 6 Missing ⚠️
vortex-array/src/patches.rs 70.00% 6 Missing ⚠️
...ortex-array/src/scalar_fn/fns/list_contains/mod.rs 60.00% 6 Missing ⚠️
... and 13 more

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@gatesn
gatesn force-pushed the ngates/buffer-allocator-execution branch from 8df467b to 41308ed Compare August 29, 2026 01:01
@gatesn
gatesn force-pushed the ngates/buffer-allocator-execution branch from 41308ed to 00bd4e5 Compare August 29, 2026 01:27
@gatesn
gatesn force-pushed the ngates/buffer-allocator-execution branch 2 times, most recently from eecf43a to 187bae1 Compare August 29, 2026 02:45
@gatesn
gatesn force-pushed the ngates/buffer-allocator-execution branch from 187bae1 to 2b0d9f0 Compare August 29, 2026 03:17
gatesn added 3 commits August 31, 2026 09:49
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
@gatesn
gatesn force-pushed the ngates/buffer-allocator-execution branch from 2b0d9f0 to cd00ce3 Compare August 31, 2026 13:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant