Skip to content

Commit 904cdaa

Browse files
rustyconoverclaude
andcommitted
ci: run the integration suite with the extension's no_error_skip config
Given no --test-config, DuckDB's sqllogictest runner turns any error whose text contains "HTTP" or "Unable to connect" into a SKIP and exits 0. Over the http transport every worker error carries "HTTP", so the http lane could report a real failure as a skip. The extension (vgi main f5aa489) ships test/configs/no_error_skip.json and wires its own scripts to it; vgi-python's two direct unittest callers did not: - ci/run-integration.sh (every CI lane, both invocations) now passes --test-config "$VGI_SRC/test/configs/no_error_skip.json" and fails fast if the pinned vgi ref lacks it. - scripts/run_all_tests.sh passes it for the local integration job. - CLAUDE.md's direct-invocation examples pass it, with a note why. Run on aarch64 with haybarn-unittest against vgi main f5aa489, the three CI lanes each show the same 3 failures, all version skew between the vgi main test tree and the community-published extension CI installs, not masked errors and not worker bugs: attach_ticket/unsupported.test (vgi_export_session missing), reflection/vgi_protocols.test (vgi_protocols missing), table/logging_generator.test (attach_opaque_data_sha256 log field, vgi ac09b3a). Everything else passes: launch/shm 353/356, http 349/352. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
1 parent 649007a commit 904cdaa

3 files changed

Lines changed: 19 additions & 5 deletions

File tree

‎CLAUDE.md‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -96,9 +96,13 @@ e.g. to pass an unusual filter to `unittest`):
9696
```bash
9797
cd ~/Development/vgi
9898
VGI_TEST_WORKER="uv run --project ~/Development/vgi-python vgi-fixture-worker" \
99-
./build/release/test/unittest "test/sql/integration/*"
99+
./build/release/test/unittest --test-config test/configs/no_error_skip.json "test/sql/integration/*"
100100
```
101101

102+
Always pass `--test-config test/configs/no_error_skip.json` to `unittest`: without
103+
it, DuckDB's runner reports any error mentioning "HTTP" or "Unable to connect" as
104+
a SKIP and exits 0.
105+
102106
**HTTP transport** (worker as an HTTP server):
103107
```bash
104108
cd ~/Development/vgi
@@ -142,7 +146,7 @@ with elapsed milliseconds. This helps identify slow queries and bottlenecks.
142146
# Timing output goes to stderr, grep for the bracket-prefixed lines:
143147
cd ~/Development/vgi
144148
VGI_TEST_WORKER="uv run --project ~/Development/vgi-python vgi-fixture-worker" \
145-
./build/release/test/unittest "test/sql/integration/table/writable_table*" \
149+
./build/release/test/unittest --test-config test/configs/no_error_skip.json "test/sql/integration/table/writable_table*" \
146150
2>&1 | grep "^\[stmt\|^\[query" | sort -t']' -k2 -rn
147151
```
148152

@@ -166,7 +170,7 @@ uv run coverage run -m pytest --no-cov -n auto
166170
# 3. Run DuckDB subprocess integration tests with coverage
167171
cd ~/Development/vgi
168172
VGI_TEST_WORKER="/tmp/vgi-coverage-worker.sh" \
169-
./build/release/test/unittest "test/sql/integration/*"
173+
./build/release/test/unittest --test-config test/configs/no_error_skip.json "test/sql/integration/*"
170174

171175
# 4. Run DuckDB HTTP integration tests with coverage
172176
cd ~/Development/vgi

‎ci/run-integration.sh‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -336,6 +336,13 @@ EOF
336336
"$HAYBARN_UNITTEST" "test/_warm.test" >/dev/null 2>&1 || echo "::warning::extension warm step did not fully succeed"
337337
rm -f "$STAGE/test/_warm.test"
338338

339+
# DuckDB's sqllogictest runner, given no --test-config, turns any error whose
340+
# text contains "HTTP" or "Unable to connect" into a SKIP and exits 0. Over the
341+
# http transport every worker error carries "HTTP", so real failures read as
342+
# skips. The extension ships a config that disables that; every run below uses it.
343+
TEST_CONFIG="$VGI_SRC/test/configs/no_error_skip.json"
344+
[ -f "$TEST_CONFIG" ] || { echo "::error::missing $TEST_CONFIG (VGI_REF predates it?)"; exit 1; }
345+
339346
# Run the lane, streaming the native sqllogictest report (a progress line per
340347
# file + the final "All tests passed (.. N assertions ..)" summary). Out-of-scope
341348
# tests were dropped at staging; any failed assertion exits non-zero.
@@ -366,7 +373,7 @@ run_unittest() {
366373
# `set +e` rather than `|| true`: the latter runs before PIPESTATUS is read and
367374
# overwrites it with true's 0, silently swallowing every real test failure.
368375
set +e
369-
"$HAYBARN_UNITTEST" "$@" 2>&1 | tee "$log"
376+
"$HAYBARN_UNITTEST" --test-config "$TEST_CONFIG" "$@" 2>&1 | tee "$log"
370377
rc="${PIPESTATUS[0]}"
371378
set -e
372379
if grep -q 'due to a fatal error condition' "$log"; then

‎scripts/run_all_tests.sh‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -204,11 +204,14 @@ run_integration_job() {
204204
return 2
205205
;;
206206
esac
207+
# --test-config: without it the runner turns any error mentioning "HTTP" or
208+
# "Unable to connect" into a SKIP, so a failure can pass as a skip.
207209
# `~[.]` excludes Catch2-hidden tests, i.e. `*.test_slow` files. The
208210
# launcher keeps one Python worker warm while unittest advances serially,
209211
# avoiding both repeated startup and cross-test state contention.
210212
VGI_TEST_WORKER="$worker" \
211-
timeout 600 ./build/release/test/unittest "test/sql/integration/*" "~[.]" >"$INT_LOG" 2>&1
213+
timeout 600 ./build/release/test/unittest --test-config test/configs/no_error_skip.json \
214+
"test/sql/integration/*" "~[.]" >"$INT_LOG" 2>&1
212215
local rc=$?
213216
summarize_integration "$INT_LOG" "$INT_SUMMARY" "$INT_FAILURES"
214217
echo "exit=$rc" >>"$INT_SUMMARY"

0 commit comments

Comments
 (0)