Skip to content

Commit 663e1f7

Browse files
andrewnesterIsaac
andauthored
pr-review skill: ask whether API changes were run on a real workspace (#7386)
## Changes Adds one bullet to pass 9 (Tests) of `.agents/skills/pr-review/SKILL.md`, placed after the bullet on whether each new or changed test shows the effect it claims. When a change sends a new field, value format or API call, and no covering test is `Cloud = true` and the description does not mention a real-workspace run, the review asks whether it was verified against one. Nothing else in the skill changes. No `.nextchanges/` fragment is added, matching #7344, which added the skill. ## Why The fake server in `libs/testserver` only returns what its handlers were taught. A test that records the request, or a handler that echoes a field back, shows what the CLI sends, not what the API accepts. Pass 9 never asked whether such a change had been checked against a real workspace, and maintainers ask that question often. The gap showed up in a review of #7184 ("Support file selectors in pipelines run"). The change started sending new `refresh_selection_by_path` values (workspace paths, notebook extension stripped) to the pipelines StartUpdate API. The only coverage was the fake server echoing them back. The test was not `Cloud = true`, and the PR description said no live-workspace run was done. The first review question was whether it had been run against a real workspace, and the skill as written would not have raised it. The bullet asks a question rather than requiring a cloud run. Every acceptance test still runs locally, and `Cloud = true` only adds a run against a real workspace. ## Tests - Markdown-only change. No Go code or acceptance tests changed. - Ran the whitespace validator that `./task ws` runs, without `--fix`: 10787 checked, 0 failed. - Checked the bullet's claims against `.agents/rules/testing.md`: `Cloud = true` adds a cloud run and never removes the local one, `Cloud` is inherited (so `out.test.toml` holds the effective value), and the fake server lives in `libs/testserver`. This pull request and its description were written by Isaac. Co-authored-by: Isaac <no-reply@databricks.com>
1 parent 119ba53 commit 663e1f7

1 file changed

Lines changed: 8 additions & 0 deletions

File tree

‎.agents/skills/pr-review/SKILL.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -139,6 +139,14 @@ on code the change adds or modifies, not on pre-existing issues next to it.
139139
should include the state after the action (list or print it again after
140140
a sync, a deploy or a migration) and the exact value the change is
141141
about, not just a successful exit.
142+
- The fake server in `libs/testserver` only returns what its handlers were
143+
taught, so a test that records the request, or a handler that echoes a
144+
field back, shows what the CLI sends, not what the API accepts. When the
145+
change sends a new field, a new value format (paths, identifiers,
146+
durations) or a new call to a real API, and no test covering it has
147+
`Cloud = true` (`Cloud` is inherited, so check `out.test.toml`) and the
148+
description does not mention a run against a real workspace, ask whether
149+
it was verified against one.
142150
- Are fixtures readable? Prefer generating archives and other binary
143151
inputs in the test script over committing opaque files.
144152
- Does the test reuse the repo's existing test helpers and scripts, and

0 commit comments

Comments
 (0)