Skip to content

Commit 1e44bb4

Browse files
janniklasroseIsaac
andauthored
Install Ruff and Python prerequisites in bump-sdk-post (#7389)
## Changes - Add `install-ruff`, which installs Ruff 0.9.1 with `uv tool install ruff==0.9.1`. - Run `install-pythons` and `install-ruff` in `bump-sdk-post` before `generate-check` and `test-update`. Add uv’s tool executable directory to `PATH` for `test-update`. - Update the bump-sdk skill documentation. The acceptance runner, scripts, and goldens are unchanged. Ruff remains an environment prerequisite; the runner does not install it. ## Why The SDK bumper’s image has uv and Python 3.12 but no Ruff, so `test-update` stops at the Ruff prerequisite. It also lacks the managed Python versions needed by the offline acceptance tests, which can cause missing-interpreter errors to be written into goldens. Prepare these prerequisites outside the runner, as CI already does. The bumper reaches this setup through the existing `generate-clijson-post` → `bump-sdk-post` call chain. ## Tests - `install-ruff` succeeds on initial and repeated runs with isolated tool directories; the existing local Ruff installation is unchanged. - `go test ./acceptance/internal -count=1` passes. - With externally installed Ruff 0.9.1 and `UV_OFFLINE=true`, both pass: - `go test ./acceptance -run "^TestAccept$/prerequisites" -count=1` - `go test ./acceptance -run "^TestAccept$/bundle/templates/pydabs/check-formatting" -count=1` - `./task --dry bump-sdk-post`, YAML and Go formatting checks, `./task ws`, and `git diff --check` pass. Downloads inside the actual SDK bumper pod have not been verified. This pull request and its description were written by Isaac. --------- Co-authored-by: Isaac <no-reply@databricks.com>
1 parent d1fd7f0 commit 1e44bb4

2 files changed

Lines changed: 17 additions & 4 deletions

File tree

‎.agents/skills/bump-sdk/SKILL.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ The SDK bumper (eng-dev-ecosystem automation) does steps 1-4 for every new stabl
2020
It opens a PR from the `automation/go-sdk-vX.Y.Z` branch of `databricks/cli`, titled `Update Go SDK to vX.Y.Z`, containing the `go get` + `go mod tidy` result, `.codegen/_openapi_sha`, `.codegen/cli.json` (genkit, followed by `./task generate-clijson-post`), and a changelog fragment without a PR link.
2121
`generate-clijson-post` ends with `bump-sdk-post` (step 4), whose error it ignores so that the bumper, which aborts on any error there, still opens a PR. The PR therefore carries whatever got regenerated, and stays red while the bump needs hand fixes (step 5).
2222
Nothing logs the task's output in the bumper, so judge from the PR what got regenerated.
23-
The bumper's environment has `uv` and Python but no `ruff`, which the acceptance tests require, so `test-update` stops at `TestAccept/prerequisites` there and bumper PRs carry no golden updates.
23+
The bumper's image has `uv` and Python 3.12 but no `ruff`; `bump-sdk-post` runs `install-pythons` and `install-ruff` before refreshing the acceptance goldens, so `test-update` can run there. Bumper PRs up to #7277 predate that and carry no golden updates.
2424
Bumper PRs opened before `generate-clijson-post` ran `bump-sdk-post` (e.g. #7138) have nothing downstream of `cli.json`: their stale command stubs usually fail to compile, which fails lint and every test job.
2525
The bumper does not merge CLI PRs, and it reuses an open PR for the same release on later runs, keeping commits pushed to its branch.
2626

@@ -52,13 +52,14 @@ Decide with the user whether to keep the tagging-producer drift or restore those
5252
The task ends with step 4 (`generate-clijson-post` runs `bump-sdk-post`) and ignores its error, so read the end of its output, or go on with step 4 directly.
5353

5454
**4. Regenerate everything downstream: `./task bump-sdk-post`.**
55-
This is the whole mechanical part of the bump: `generate-check` (command stubs, refschema, schema, validation, direct-engine YAML, pydabs), then `test-update` for the acceptance goldens.
55+
This is the whole mechanical part of the bump: `install-pythons` and `install-ruff`, then `generate-check` (command stubs, refschema, schema, validation, direct-engine YAML, pydabs), then `test-update` for the acceptance goldens.
5656
Everything after it needs judgment.
5757
Run directly, it fails on errors: `generate-refschema` and `test-update` build the CLI, so it fails while the bump still breaks compilation; fix that (step 5) and rerun the task.
5858

5959
Run it in CI's environment, or the goldens pick up local drift:
6060
- `uv` index: the acceptance tests build wheels offline from the vendored packages (`UV_OFFLINE`, `UV_FIND_LINKS`), but `uv` still reads its default index's pages from the shared cache. If a `uv.toml` (e.g. `/etc/uv/uv.toml`, which applies even under the tests' temporary `HOME`) makes an internal PyPI proxy the default index, `uv` picks versions from cached proxy pages that it cannot download offline and fails with `No solution found when resolving: hatchling`; `-update` then bakes that error into dozens of template goldens and deletes their `output/` trees. The same failure on `main` tells it apart from bump fallout. Set `UV_DEFAULT_INDEX=https://pypi.org/simple` for the acceptance runs only, so `uv` stops reading cached proxy pages (CI's test jobs point `UV_INDEX_URL` at a JFrog mirror instead): the generators fetch tools such as `ruff` and need the proxy where `pypi.org` is unreachable. On such a machine, run the task's two halves separately: `./task generate-check`, then `UV_DEFAULT_INDEX=https://pypi.org/simple ./task test-update` (and the same prefix for `./task test` later). If `generate-clijson` already ran `test-update` without it, first revert the `acceptance/` churn (`git checkout -- acceptance`, plus any stray untracked files it left).
61-
- Python: CI installs the managed interpreters with `./task install-pythons`; do the same.
61+
- Python: the managed interpreters from `./task install-pythons`, which `bump-sdk-post` runs first (CI runs it in setup-build-environment).
62+
- Ruff: `./task install-ruff` installs CI's version (0.9.1) via uv; `bump-sdk-post` adds uv's tool executable directory to `PATH` for `test-update`. The acceptance runner only checks that Ruff is available.
6263
- Go: the `toolchain` in `go.mod`. A newer local `go` also changes goldens and generated `gofmt` output; use `GOTOOLCHAIN=<pinned>` if they differ.
6364

6465
Before going on, account for every changed file outside the generated set, including the bumper's (its environment is not CI's either): a bump without fallout changes `cmd/**`, `bundle/schema/jsonschema.json`, `bundle/internal/validation/generated/*`, `python/databricks/bundles/**`, `acceptance/bundle/refschema/output` and help-text goldens (`acceptance/help/output.txt`, `acceptance/cmd/account/account-help/output.txt`).

‎Taskfile.yml‎

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -320,6 +320,11 @@ tasks:
320320
cmds:
321321
- "uv python install 3.9 3.10 3.11 3.12 3.13"
322322

323+
install-ruff:
324+
desc: Install Ruff 0.9.1 via uv
325+
cmds:
326+
- uv tool install ruff==0.9.1
327+
323328
# --- Building ---
324329

325330
# The root binary only imports bundle/, cmd/, experimental/, internal/, libs/,
@@ -967,8 +972,15 @@ tasks:
967972
bump-sdk-post:
968973
desc: Regenerate everything downstream of an SDK bump and refresh acceptance goldens
969974
cmds:
975+
# The acceptance tests run uv offline, so the managed interpreters they
976+
# pin (3.10 for bundle tests, 3.9-3.13 in one template matrix) must exist
977+
# up front. CI installs them in setup-build-environment; the SDK bumper's
978+
# image ships only Python 3.12, and a missing version would otherwise be
979+
# baked into the goldens as a uv error.
980+
- task: install-pythons
981+
- task: install-ruff
970982
- task: generate-check
971-
- task: test-update
983+
- PATH="$(uv tool dir --bin):$PATH" "{{.TASK_EXE}}" test-update
972984

973985
# Refreshes acceptance/bundle/refschema/output/*.txt, which record the field paths / types emitted by
974986
# `bundle debug refschema` (see cmd/bundle/debug/refschema.go). It reflects

0 commit comments

Comments
 (0)