Repository navigation
[WIP] Add the databricks pipelines test command - #7271
Draft
tharunbhuk-db wants to merge 10 commits into
Draft
tharunbhuk-db wants to merge 10 commits into
tharunbhuk-db wants to merge 10 commits into
Conversation
Submits a test-only pipeline update with optional test selections and pytest arguments. Relative selections resolve to deployed workspace paths through the bundle sync root, or against the pipeline's root_path when the pipeline is identified with --pipeline-id. Only -k and -m are accepted in --pytest-arg. Co-authored-by: Isaac <no-reply@databricks.com>
Cover each rejected pytest option with a unit test, and add an acceptance case for `--pytest-arg VALUE`, which keeps commas and spaces. Co-authored-by: Isaac <no-reply@databricks.com>
The service names a test case relative to pytest's rootdir: the deepest folder that holds the pipeline's root_path and every test source, or, without a root_path, the folder of every test source. A test ID copied from the results now runs that test again in any project layout. A relative selection that exists locally still maps to its deployed path. Any other relative selection resolves against that folder, which the command reads from the deployed pipeline. With --pipeline-id, all relative selections resolve against it. Co-authored-by: Isaac <no-reply@databricks.com>
…nd preview A local test path mapped to workspace.file_path in every sync mode. A source-linked deployment runs the source files, so a local path now maps under the sync root, as TranslatePaths maps resource paths. An immutable deployment runs a snapshot whose path the command does not know, so a local path is an error there that points to test IDs and workspace paths, which still work. The folder of test IDs is read once, and only when a selection is a relative path. Paths in errors use forward slashes. The Short text starts with *Preview*, like other preview commands. Tests: the option list covers each rejected form once, and a folder test source without root_path has its parent as the folder of test IDs, as the service computes it. Co-authored-by: Isaac <no-reply@databricks.com>
This was referenced Oct 30, 2026
Co-authored-by: Isaac <no-reply@databricks.com>
Contributor
|
Thanks for the contribution! This pull request was opened from a fork. If you're a member of the Databricks organization, please request access to the If you're an external contributor, no action is needed; a maintainer will review your pull request. |
tharunbhuk-db
changed the title
Contributor
|
An authorized user can trigger integration tests manually by following the instructions below: Trigger: Inputs:
Checks will be approved automatically on success. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack 1/4: #7271 command · #7272 output · #7273 JUnit · #7274 bundles. Builds on
main.This PR's changes: tharunbhuk-db/cli@databricks:main...tharun/pipelines-test-pr1-command
Design doc: https://docs.google.com/document/d/1atW-JG_hQu-G-YX-m1M7OodNZdH7LMc9H7CBlQAv5gw/edit
Changes
databricks pipelines test [PIPELINE_KEY] [TEST_SELECTION...], which runs a pipeline's tests in a test-only update. The help marks it*Preview*. Without a selection, all of the pipeline's tests run.tests/test_orders.py::test_backfill. The API accepts only absolute workspace paths, so the CLI converts relative paths: in a bundle, a relative path is a local path from the working directory, must exist, and maps to its deployed path; with--pipeline-id, it is relative to the pipeline'sroot_path.--pytest-argpasses-kand-mto pytest. A value in the next--pytest-argcannot start with-. Other pytest options are rejected with guidance.--pipeline-idnames the pipeline.Why
Pipeline tests run only from the editor today. CI jobs and agents need a command that runs them.
Tests
acceptance/pipelines/test/{arguments,selection,pipeline-id,immutable}.This pull request and its description were written by Isaac.