Skip to content

Commit a810e59

Browse files
claudeclaude
committed
Merge branch 'develop' into ct/fix-normalized
Both sides added benchmarks to `vortex-tensor`, so the `[[bench]]` list and the dev-dependencies conflicted. Resolved to develop's four bench targets. `develop`'s `benches/normalized.rs` calls the three-argument `Normalized::try_new` and builds its `nullable` arm by wrapping the normalized child in a `MaskedArray`. Both are gone here: the constructor takes an explicit `Validity` and rejects a nullable child. The arm now passes the validity alongside a non-nullable child, which keeps the names `non_nullable` and `nullable` stable for CodSpeed. Folds the encode benchmark from this branch into that file as `encode_non_nullable` and `encode_nullable` rather than keeping a separate `benches/normalize.rs`, whose name was one letter from `normalized.rs`. The arms reuse the file's `ELEMENTS` budget, which is sized for CodSpeed's CPU simulation rather than a desktop, so they are far smaller than the standalone bench was. Signed-off-by: Connor Tsui <connor@spiraldb.com>
2 parents 81cb02c + a1057db commit a810e59

237 files changed

Lines changed: 11809 additions & 5559 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/bench-dispatch.yml‎

Lines changed: 0 additions & 83 deletions
This file was deleted.

‎.github/workflows/codspeed.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,7 @@ jobs:
5757
- { shard: 6, name: "Encodings 3", packages: "vortex-pco vortex-runend vortex-sequence" }
5858
- { shard: 7, name: "Encodings 4", packages: "vortex-sparse vortex-zigzag vortex-zstd" }
5959
- { shard: 8, name: "Storage formats & row encoding", packages: "vortex-flatbuffers vortex-proto vortex-btrblocks vortex-row" }
60+
- { shard: 9, name: "Tensor & geo", packages: "vortex-tensor vortex-geo" }
6061
name: "Benchmark with Codspeed (Shard #${{ matrix.shard }})"
6162
timeout-minutes: 30
6263
runs-on: >-

‎.github/workflows/commit-metadata.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,13 +22,13 @@ jobs:
2222
with:
2323
fetch-depth: 2
2424

25-
# v4 (Postgres) ingest -- REQUIRED (see bench.yml rationale). Empty records:
25+
# v4 (Postgres) ingest -- REQUIRED (see develop-bench.yml rationale). Empty records:
2626
# post-ingest.py --postgres upserts the commit row only. Gated on the
2727
# ingest-role ARN var (the assume-role input that MUST exist for OIDC to
2828
# succeed).
2929
#
3030
# `sync: false` -- the ingest runs `uv run --no-project --with`, which needs only
31-
# the uv binary, never the synced workspace (see bench.yml rationale).
31+
# the uv binary, never the synced workspace (see develop-bench.yml rationale).
3232
- name: Install uv for v4 ingest
3333
if: vars.GH_BENCH_INGEST_ROLE_ARN != ''
3434
uses: spiraldb/actions/.github/actions/setup-uv@a746510eafaa926484c354541cfc49b2ec06cc63 # 0.18.6

.github/workflows/bench.yml renamed to .github/workflows/develop-bench.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Runs after every commit to `develop` (or in other words, _after_ every pull request merges).
22

3-
name: Benchmarks
3+
name: Develop Benchmarks
44

55
on:
66
push:
@@ -188,7 +188,7 @@ jobs:
188188
deduplication-key: ci-bench-${{ matrix.benchmark.id }}-failure
189189

190190
sql:
191-
uses: ./.github/workflows/sql-benchmarks.yml
191+
uses: ./.github/workflows/sql-bench-matrix.yml
192192
secrets: inherit
193193
with:
194194
mode: "develop"

‎.github/workflows/nightly-bench.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ permissions:
1818

1919
jobs:
2020
sql:
21-
uses: ./.github/workflows/sql-benchmarks.yml
21+
uses: ./.github/workflows/sql-bench-matrix.yml
2222
secrets: inherit
2323
with:
2424
mode: "develop"

‎.github/workflows/pr-bench-compress.yml‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Runs the compression benchmark for a pull request.
2+
3+
name: PR Compression Benchmark
4+
5+
on:
6+
workflow_call: { }
7+
workflow_dispatch: { }
8+
9+
permissions:
10+
contents: read
11+
pull-requests: write # for commenting on PRs
12+
id-token: write # enables AWS-GitHub OIDC
13+
14+
jobs:
15+
bench:
16+
uses: ./.github/workflows/pr-bench-runner.yml
17+
secrets: inherit
18+
with:
19+
benchmark_id: compress-bench
20+
benchmark_name: Compression

‎.github/workflows/pr-bench-dispatch.yml‎

Lines changed: 153 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,153 @@
1+
# Dispatches benchmark workflows when benchmark labels are added to pull requests.
2+
# This is a separate workflow so that non-benchmark label events don't create
3+
# phantom check suites that obscure in-progress benchmark runs on the PR.
4+
5+
name: PR Benchmark Dispatch
6+
run-name: >-
7+
PR #${{ github.event.pull_request.number }} benchmark:
8+
${{ github.event.label.name }}
9+
10+
on:
11+
pull_request:
12+
types: [labeled]
13+
branches: ["develop"]
14+
15+
permissions:
16+
actions: write
17+
contents: read
18+
pull-requests: write # for label removal and PR comments
19+
id-token: write # enables AWS-GitHub OIDC
20+
21+
jobs:
22+
remove-all-label:
23+
runs-on: ubuntu-latest
24+
timeout-minutes: 10
25+
if: github.event.label.name == 'action/bench-all'
26+
steps:
27+
- uses: actions-ecosystem/action-remove-labels@2ce5d41b4b6aa8503e285553f75ed56e0a40bae0 # v1
28+
if: github.event.pull_request.head.repo.full_name == 'vortex-data/vortex'
29+
with:
30+
labels: action/bench-all
31+
fail_on_error: true
32+
33+
all-random-access-bench:
34+
needs: remove-all-label
35+
uses: ./.github/workflows/pr-bench-random-access.yml
36+
secrets: inherit
37+
38+
all-compression-bench:
39+
needs: remove-all-label
40+
uses: ./.github/workflows/pr-bench-compress.yml
41+
secrets: inherit
42+
43+
all-string-bench:
44+
needs: remove-all-label
45+
uses: ./.github/workflows/pr-bench-string.yml
46+
secrets: inherit
47+
48+
all-sql-bench:
49+
needs: remove-all-label
50+
uses: ./.github/workflows/pr-bench-sql.yml
51+
secrets: inherit
52+
with:
53+
matrix_preset: "pr-all"
54+
55+
remove-random-access-label:
56+
runs-on: ubuntu-latest
57+
timeout-minutes: 10
58+
if: github.event.label.name == 'action/bench-random-access'
59+
steps:
60+
- uses: actions-ecosystem/action-remove-labels@2ce5d41b4b6aa8503e285553f75ed56e0a40bae0 # v1
61+
if: github.event.pull_request.head.repo.full_name == 'vortex-data/vortex'
62+
with:
63+
labels: action/bench-random-access
64+
fail_on_error: true
65+
66+
random-access-bench:
67+
needs: remove-random-access-label
68+
uses: ./.github/workflows/pr-bench-random-access.yml
69+
secrets: inherit
70+
71+
remove-compress-label:
72+
runs-on: ubuntu-latest
73+
timeout-minutes: 10
74+
if: github.event.label.name == 'action/bench-compress'
75+
steps:
76+
- uses: actions-ecosystem/action-remove-labels@2ce5d41b4b6aa8503e285553f75ed56e0a40bae0 # v1
77+
if: github.event.pull_request.head.repo.full_name == 'vortex-data/vortex'
78+
with:
79+
labels: action/bench-compress
80+
fail_on_error: true
81+
82+
compression-bench:
83+
needs: remove-compress-label
84+
uses: ./.github/workflows/pr-bench-compress.yml
85+
secrets: inherit
86+
87+
remove-string-label:
88+
runs-on: ubuntu-latest
89+
timeout-minutes: 10
90+
if: github.event.label.name == 'action/bench-string'
91+
steps:
92+
- uses: actions-ecosystem/action-remove-labels@2ce5d41b4b6aa8503e285553f75ed56e0a40bae0 # v1
93+
if: github.event.pull_request.head.repo.full_name == 'vortex-data/vortex'
94+
with:
95+
labels: action/bench-string
96+
fail_on_error: true
97+
98+
string-bench:
99+
needs: remove-string-label
100+
uses: ./.github/workflows/pr-bench-string.yml
101+
secrets: inherit
102+
103+
remove-gpu-compress-label:
104+
runs-on: ubuntu-latest
105+
timeout-minutes: 10
106+
if: github.event.label.name == 'action/bench-gpu-compress'
107+
steps:
108+
- uses: actions-ecosystem/action-remove-labels@2ce5d41b4b6aa8503e285553f75ed56e0a40bae0 # v1
109+
if: github.event.pull_request.head.repo.full_name == 'vortex-data/vortex'
110+
with:
111+
labels: action/bench-gpu-compress
112+
fail_on_error: true
113+
114+
gpu-compress-bench:
115+
needs: remove-gpu-compress-label
116+
uses: ./.github/workflows/pr-bench-gpu-compress.yml
117+
secrets: inherit
118+
119+
remove-sql-label:
120+
runs-on: ubuntu-latest
121+
timeout-minutes: 10
122+
if: github.event.label.name == 'action/bench-sql'
123+
steps:
124+
- uses: actions-ecosystem/action-remove-labels@2ce5d41b4b6aa8503e285553f75ed56e0a40bae0 # v1
125+
if: github.event.pull_request.head.repo.full_name == 'vortex-data/vortex'
126+
with:
127+
labels: action/bench-sql
128+
fail_on_error: true
129+
130+
sql-bench:
131+
needs: remove-sql-label
132+
uses: ./.github/workflows/pr-bench-sql.yml
133+
secrets: inherit
134+
with:
135+
matrix_preset: "pr"
136+
137+
remove-sql-compact-label:
138+
runs-on: ubuntu-latest
139+
timeout-minutes: 10
140+
if: github.event.label.name == 'action/bench-sql-compact'
141+
steps:
142+
- uses: actions-ecosystem/action-remove-labels@2ce5d41b4b6aa8503e285553f75ed56e0a40bae0 # v1
143+
if: github.event.pull_request.head.repo.full_name == 'vortex-data/vortex'
144+
with:
145+
labels: action/bench-sql-compact
146+
fail_on_error: true
147+
148+
sql-compact-bench:
149+
needs: remove-sql-compact-label
150+
uses: ./.github/workflows/pr-bench-sql.yml
151+
secrets: inherit
152+
with:
153+
matrix_preset: "pr-compact"

.github/workflows/gpu-compress-bench-pr.yml renamed to .github/workflows/pr-bench-gpu-compress.yml

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
# Runs the GPU-enabled compression decompression benchmarks for a pull request.
2-
# Called from bench-dispatch.yml when the `action/benchmark-gpu-compress` label is added.
2+
# Called from pr-bench-dispatch.yml when the `action/bench-gpu-compress` label is added.
33

4-
name: GPU Compression Benchmarks
4+
name: PR GPU Compression Benchmark
55

66
concurrency:
7-
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
7+
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}-gpu-compress
88
cancel-in-progress: false
99

1010
on:
@@ -71,13 +71,13 @@ jobs:
7171
} > comment.md
7272
cat comment.md >> "$GITHUB_STEP_SUMMARY"
7373
- name: Comment PR
74-
if: github.event.pull_request.head.repo.fork == false
74+
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == false
7575
uses: thollander/actions-comment-pull-request@24bffb9b452ba05a4f3f77933840a6a841d1b32b # v3
7676
with:
7777
file-path: comment.md
7878
comment-tag: bench-pr-comment-gpu-compress
7979
- name: Comment PR on failure
80-
if: failure() && github.event.pull_request.head.repo.fork == false
80+
if: failure() && github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == false
8181
uses: thollander/actions-comment-pull-request@24bffb9b452ba05a4f3f77933840a6a841d1b32b # v3
8282
with:
8383
message: |

‎.github/workflows/pr-bench-random-access.yml‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# Runs the random-access benchmark for a pull request.
2+
3+
name: PR Random Access Benchmark
4+
5+
on:
6+
workflow_call: { }
7+
workflow_dispatch: { }
8+
9+
permissions:
10+
contents: read
11+
pull-requests: write # for commenting on PRs
12+
id-token: write # enables AWS-GitHub OIDC
13+
14+
jobs:
15+
bench:
16+
uses: ./.github/workflows/pr-bench-runner.yml
17+
secrets: inherit
18+
with:
19+
benchmark_id: random-access-bench
20+
benchmark_name: Random Access
21+
with_lance: true

0 commit comments

Comments
 (0)