[tharunbhuk-db]

🥞 Stacked PR

• stack/pipelines-test-pr1-command [Files changed]
  • stack/pipelines-test-bundle-key [Files changed] ← this PR
    • stack/pipelines-test-read-events [Files changed]
      • stack/pipelines-test-print-results [Files changed]
        • stack/pipelines-test-pr2-output [Files changed]
          • stack/pipelines-test-a4-run-control [Files changed]
            • stack/pipelines-test-pr3-junit [Files changed]
              • stack/pipelines-test-junit-flag [Files changed]

---

• In a bundle, identify the pipeline by KEY: pipelines test [flags] [KEY] [TEST_SELECTION...].
• Relative selections are local paths, mapped to their deployed paths.
• A first argument that is not a KEY gets an error that names the bundle's pipelines and the working command.

Tests: unit tests; acceptance in acceptance/pipelines/test/{selection,immutable}.

This pull request and its description were written by Isaac.

[github-actions]

Commit: 2a9fc8107 | Workflow run

Regression test report

Selected 4 changed tests (limit=20, 0 not selected)

4 tests: regression=4 unreleased=0 coverage=0 failing=0

test	branch	main (b00fe63)	latest (v1.25.0)
TestAccept/pipelines/test/arguments	✅	❌	❌
TestAccept/pipelines/test/immutable	✅	❌	❌
TestAccept/pipelines/test/pipeline-id	✅	❌	❌
TestAccept/pipelines/test/selection	✅	❌	❌

TestAccept/pipelines/test/arguments: branch ✅ | main (b00fe63) ❌ | latest (v1.25.0) ❌

main (b00fe63):

acceptance_test.go:1345: Diff:
        --- pipelines/test/arguments/output.txt
        +++ /tmp/TestAcceptpipelinestestarguments1247089293/002/output.txt
        @@ -4,16 +4,10 @@
         
         This command is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it.
         
        -KEY is the unique name of the pipeline, as defined in its YAML file.
        -It is required when test selections are given. If there is only one pipeline
        -in the project and no arguments are given, it is selected automatically.
        -Outside a project, identify the pipeline with --pipeline-id; all positional
        -arguments are then test selections.
        +Identify the pipeline with --pipeline-id.
         
         TEST_SELECTION is a test directory, file, or pytest test ID, for example
        -tests/test_orders.py::test_backfill. In a project, a relative path is a local
        -path, relative to the working directory, and is converted to its deployed
        -workspace path. With --pipeline-id, a relative path is relative to the
        +tests/test_orders.py::test_backfill. A relative path is relative to the
         pipeline's root_path. Paths that start with / are workspace paths. Without a
         selection, all of the pipeline's tests run.
         
        @@ -21,13 +13,13 @@
         and -m are supported, each followed by its value in the next --pytest-arg, for
         example:
         
        -  databricks pipelines test my_pipeline --pytest-arg=-k --pytest-arg='orders and not slow'
        +  databricks pipelines test --pipeline-id <id> --pytest-arg=-k --pytest-arg='orders and not slow'
         
         Usage:
        -  databricks pipelines test [flags] [KEY] [TEST_SELECTION...]
        +  databricks pipelines test [flags] [TEST_SELECTION...]
         
         Pipeline Test Flags:
        -      --pipeline-id string       ID of the pipeline to test, instead of a KEY.
        +      --pipeline-id string       ID of the pipeline to test.
               --pytest-arg stringArray   Argument to pass to pytest (repeatable). Only -k and -m are supported.
         
         Flags:

latest (v1.25.0):

acceptance_test.go:1345: Diff:
        --- pipelines/test/arguments/output.txt
        +++ /tmp/TestAcceptpipelinestestarguments2596189000/002/output.txt
        @@ -1,67 +1,434 @@
         
         >>> [CLI] pipelines test --help
        -Run the tests of a pipeline in a test-only update.
        +The Lakeflow Spark Declarative Pipelines API allows you to create, edit,
        +  delete, start, and view details about pipelines.
         
        -This command is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it.
        +  Spark Declarative Pipelines is a framework for building reliable,
        +  maintainable, and testable data processing pipelines. You define the
        +  transformations to perform on your data, and Spark Declarative Pipelines
        +  manages task orchestration, cluster management, monitoring, data quality, and
        +  error handling.
         
        -KEY is the unique name of the pipeline, as defined in its YAML file.
        -It is required when test selections are given. If there is only one pipeline
        -in the project and no arguments are given, it is selected automatically.
        -Outside a project, identify the pipeline with --pipeline-id; all positional
        -arguments are then test selections.
        +  Instead of defining your data pipelines using a series of separate Apache
        +  Spark tasks, Spark Declarative Pipelines manages how your data is transformed
        +  based on a target schema you define for each processing step. You can also
        +  enforce data quality with Spark Declarative Pipelines expectations.
        +  Expectations allow you to define expected data quality and specify how to
        +  handle records that fail those expectations.
         
        -TEST_SELECTION is a test directory, file, or pytest test ID, for example
        -tests/test_orders.py::test_backfill. In a project, a relative path is a local
        -path, relative to the working directory, and is converted to its deployed
        -workspace path. With --pipeline-id, a relative path is relative to the
        -pipeline's root_path. Paths that start with / are workspace paths. Without a
        -selection, all of the pipeline's tests run.
        +Usage:
        +  databricks pipelines [flags]
        +  databricks pipelines [command]
         
        -Each --pytest-arg passes one argument to pytest, in the order given. Only -k
        -and -m are supported, each followed by its value in the next --pytest-arg, for
        -example:
        +Main Commands
        +  deploy                Deploy pipelines
        +  destroy               Destroy a pipelines project
        +  dry-run               Validate correctness of the pipeline's graph
        +  generate              Generate pipeline configuration
        +  history               Retrieve past runs for a pipeline
        +  init                  Initialize a new pipelines project
        +  logs                  Retrieve events for a pipeline
        +  open                  Open a pipeline in the browser
        +  run                   Run a pipeline
        +  show                  Preview a table's columns and sample rows
        +  stop                  Stop a pipeline
         
        -  databricks pipelines test my_pipeline --pytest-arg=-k --pytest-arg='orders and not slow'
        +Management Commands
        +  apply-environment     *Public Preview* Apply the latest environment to the pipeline.
        +  clone                 Clone a pipeline.
        +  create                Create a pipeline.
        +  delete                Delete a pipeline.
        +  get                   Get a pipeline.
        +  get-update            Get a pipeline update.
        +  list-pipeline-events  List pipeline events.
        +  list-pipelines        List pipelines.
        +  list-updates          List pipeline updates.
        +  start-update          Start a pipeline.
        +  update                Edit a pipeline.
         
        -Usage:
        -  databricks pipelines test [flags] [KEY] [TEST_SELECTION...]
        +Permission Commands
        +  get-permission-levels Get pipeline permission levels.
        +  get-permissions       Get pipeline permissions.
        +  set-permissions       Set pipeline permissions.
        +  update-permissions    Update pipeline permissions.
         
        -Pipeline Test Flags:
        -      --pipeline-id string       ID of the pipeline to test, instead of a KEY.
        -      --pytest-arg stringArray   Argument to pass to pytest (repeatable). Only -k and -m are supported.
        -
         Flags:
        -  -h, --help   help for test
        +  -h, --help          help for pipelines
        +      --var strings   set values for variables defined in project config. Example: --var="foo=bar"
         
         Global Flags:
               --debug            enable debug logging
           -o, --output type      output type: text or json (default text)
           -p, --profile string   ~/.databrickscfg profile
           -t, --target string    bundle target to use (if applicable)
        -      --var strings      set values for variables defined in project config. Example: --var="foo=bar"
         
        +Use "databricks pipelines [command] --help" for more information about a command.
        +
         === The command is experimental and off by default
         >>> musterr [CLI] pipelines test --pipeline-id 1234
        -Error: pipelines test is experimental. Set DATABRICKS_CLI_PIPELINES_TEST=true to enable it
        +Error: unknown flag: --pipeline-id
         
        +Usage:
        +  databricks pipelines [flags]
        +  databricks pipelines [command]
        +
        +Main Commands
        +  deploy                Deploy pipelines
        +  destroy               Destroy a pipelines project
        +  dry-run               Validate correctness of the pipeline's graph
        +  generate              Generate pipeline configuration
        +  history               Retrieve past runs for a pipeline
        +  init                  Initialize a new pipelines project
        +  logs                  Retrieve events for a pipeline
        +  open                  Open a pipeline in the browser
        +  run                   Run a pipeline
        +  show                  Preview a table's columns and sample rows
        +  stop                  Stop a pipeline
        +
        +Management Commands
        +  apply-environment     *Public Preview* Apply the latest environment to the pipeline.
        +  clone                 Clone a pipeline.
        +  create                Create a pipeline.
        +  delete                Delete a pipeline.
        +  get                   Get a pipeline.
        +  get-update            Get a pipeline update.
        +  list-pipeline-events  List pipeline events.
        +  list-pipelines        List pipelines.
        +  list-updates          List pipeline updates.
        +  start-update          Start a pipeline.
        +  update                Edit a pipeline.
        +
        +Permission Commands
        +  get-permission-levels Get pipeline permission levels.
        +  get-permissions       Get pipeline permissions.
        +  set-permissions       Set pipeline permissions.
        +  update-permissions    Update pipeline permissions.
        +
        +Flags:
        +  -h, --help          help for pipelines
        +      --var strings   set values for variables defined in project config. Example: --var="foo=bar"
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +
        +Use "databricks pipelines [command] --help" for more information about a command.
        +
        +
         === pytest arguments must be passed with --pytest-arg
         >>> musterr [CLI] pipelines test --pipeline-id 1234 -k orders
        -Error: -k is a pytest argument: pass pytest arguments with --pytest-arg, for example --pytest-arg=-k --pytest-arg='orders and not slow'
        +Error: unknown flag: --pipeline-id
         
        +Usage:
        +  databricks pipelines [flags]
        +  databricks pipelines [command]
        +
        +Main Commands
        +  deploy                Deploy pipelines
        +  destroy               Destroy a pipelines project
        +  dry-run               Validate correctness of the pipeline's graph
        +  generate              Generate pipeline configuration
        +  history               Retrieve past runs for a pipeline
        +  init                  Initialize a new pipelines project
        +  logs                  Retrieve events for a pipeline
        +  open                  Open a pipeline in the browser
        +  run                   Run a pipeline
        +  show                  Preview a table's columns and sample rows
        +  stop                  Stop a pipeline
        +
        +Management Commands
        +  apply-environment     *Public Preview* Apply the latest environment to the pipeline.
        +  clone                 Clone a pipeline.
        +  create                Create a pipeline.
        +  delete                Delete a pipeline.
        +  get                   Get a pipeline.
        +  get-update            Get a pipeline update.
        +  list-pipeline-events  List pipeline events.
        +  list-pipelines        List pipelines.
        +  list-updates          List pipeline updates.
        +  start-update          Start a pipeline.
        +  update                Edit a pipeline.
        +
        +Permission Commands
        +  get-permission-levels Get pipeline permission levels.
        +  get-permissions       Get pipeline permissions.
        +  set-permissions       Set pipeline permissions.
        +  update-permissions    Update pipeline permissions.
        +
        +Flags:
        +  -h, --help          help for pipelines
        +      --var strings   set values for variables defined in project config. Example: --var="foo=bar"
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +
... (270 more lines truncated, raise DATABRICKS_CLI_TEST_MAX_LOG to see more)

TestAccept/pipelines/test/immutable: branch ✅ | main (b00fe63) ❌ | latest (v1.25.0) ❌

main (b00fe63):

acceptance_test.go:1345: Diff:
        --- pipelines/test/immutable/output.txt
        +++ /tmp/TestAcceptpipelinestestimmutable1028362258/002/output.txt
        @@ -4,50 +4,14 @@
         Created pipelines.my_pipeline
         Files: 0 uploaded, 0 deleted
         Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
        -View your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID][0]?w=[WORKSPACE_ID]
        +View your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID]?w=[WORKSPACE_ID]
         
         === Local test paths are not supported with workspace.sync_mode immutable
         >>> musterr [CLI] pipelines test my_pipeline tests/test_orders.py::test_backfill
        -Recommendation: This command runs the last deployed version of the code
        +Error: identify the pipeline with --pipeline-id
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        -
        -Error: cannot resolve test selection "tests/test_orders.py::test_backfill": local test paths are not supported with workspace.sync_mode immutable; pass an absolute workspace path
        -
         === Absolute workspace paths and running all tests still work
         >>> MSYS_NO_PATHCONV=1 [CLI] pipelines test my_pipeline /Workspace/Users/someone@example.com/tests/test_orders.py::test_backfill
        -Recommendation: This command runs the last deployed version of the code
        +Error: identify the pipeline with --pipeline-id
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        -
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][1]
        -
        ->>> [CLI] pipelines test my_pipeline
        -Recommendation: This command runs the last deployed version of the code
        -
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        -
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][2]
        -
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {
        -      "test_selection": [
        -        "/Workspace/Users/someone@example.com/tests/test_orders.py::test_backfill"
        -      ]
        -    }
        -  }
        -}
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true
        -  }
        -}
        +Exit code: 1

latest (v1.25.0):

acceptance_test.go:1345: Diff:
        --- pipelines/test/immutable/output.txt
        +++ /tmp/TestAcceptpipelinestestimmutable1353462466/002/output.txt
        @@ -4,50 +4,112 @@
         Created pipelines.my_pipeline
         Files: 0 uploaded, 0 deleted
         Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
        -View your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID][0]?w=[WORKSPACE_ID]
        +View your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID]?w=[WORKSPACE_ID]
         
         === Local test paths are not supported with workspace.sync_mode immutable
         >>> musterr [CLI] pipelines test my_pipeline tests/test_orders.py::test_backfill
        -Recommendation: This command runs the last deployed version of the code
        +Error: unknown command "test" for "databricks pipelines"
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        +Usage:
        +  databricks pipelines [flags]
        +  databricks pipelines [command]
         
        -Error: cannot resolve test selection "tests/test_orders.py::test_backfill": local test paths are not supported with workspace.sync_mode immutable; pass an absolute workspace path
        +Main Commands
        +  deploy                Deploy pipelines
        +  destroy               Destroy a pipelines project
        +  dry-run               Validate correctness of the pipeline's graph
        +  generate              Generate pipeline configuration
        +  history               Retrieve past runs for a pipeline
        +  init                  Initialize a new pipelines project
        +  logs                  Retrieve events for a pipeline
        +  open                  Open a pipeline in the browser
        +  run                   Run a pipeline
        +  show                  Preview a table's columns and sample rows
        +  stop                  Stop a pipeline
         
        +Management Commands
        +  apply-environment     *Public Preview* Apply the latest environment to the pipeline.
        +  clone                 Clone a pipeline.
        +  create                Create a pipeline.
        +  delete                Delete a pipeline.
        +  get                   Get a pipeline.
        +  get-update            Get a pipeline update.
        +  list-pipeline-events  List pipeline events.
        +  list-pipelines        List pipelines.
        +  list-updates          List pipeline updates.
        +  start-update          Start a pipeline.
        +  update                Edit a pipeline.
        +
        +Permission Commands
        +  get-permission-levels Get pipeline permission levels.
        +  get-permissions       Get pipeline permissions.
        +  set-permissions       Set pipeline permissions.
        +  update-permissions    Update pipeline permissions.
        +
        +Flags:
        +  -h, --help          help for pipelines
        +      --var strings   set values for variables defined in project config. Example: --var="foo=bar"
        +
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +
        +Use "databricks pipelines [command] --help" for more information about a command.
        +
        +
         === Absolute workspace paths and running all tests still work
         >>> MSYS_NO_PATHCONV=1 [CLI] pipelines test my_pipeline /Workspace/Users/someone@example.com/tests/test_orders.py::test_backfill
        -Recommendation: This command runs the last deployed version of the code
        +Error: unknown command "test" for "databricks pipelines"
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        +Usage:
        +  databricks pipelines [flags]
        +  databricks pipelines [command]
         
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][1]
        +Main Commands
        +  deploy                Deploy pipelines
        +  destroy               Destroy a pipelines project
        +  dry-run               Validate correctness of the pipeline's graph
        +  generate              Generate pipeline configuration
        +  history               Retrieve past runs for a pipeline
        +  init                  Initialize a new pipelines project
        +  logs                  Retrieve events for a pipeline
        +  open                  Open a pipeline in the browser
        +  run                   Run a pipeline
        +  show                  Preview a table's columns and sample rows
        +  stop                  Stop a pipeline
         
        ->>> [CLI] pipelines test my_pipeline
        -Recommendation: This command runs the last deployed version of the code
        +Management Commands
        +  apply-environment     *Public Preview* Apply the latest environment to the pipeline.
        +  clone                 Clone a pipeline.
        +  create                Create a pipeline.
        +  delete                Delete a pipeline.
        +  get                   Get a pipeline.
        +  get-update            Get a pipeline update.
        +  list-pipeline-events  List pipeline events.
        +  list-pipelines        List pipelines.
        +  list-updates          List pipeline updates.
        +  start-update          Start a pipeline.
        +  update                Edit a pipeline.
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        +Permission Commands
        +  get-permission-levels Get pipeline permission levels.
        +  get-permissions       Get pipeline permissions.
        +  set-permissions       Set pipeline permissions.
        +  update-permissions    Update pipeline permissions.
         
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][2]
        +Flags:
        +  -h, --help          help for pipelines
        +      --var strings   set values for variables defined in project config. Example: --var="foo=bar"
         
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {
        -      "test_selection": [
        -        "/Workspace/Users/someone@example.com/tests/test_orders.py::test_backfill"
        -      ]
        -    }
        -  }
        -}
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true
        -  }
        -}
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
        +
        +Use "databricks pipelines [command] --help" for more information about a command.
        +
        +
        +Exit code: 1
        
FAIL

TestAccept/pipelines/test/pipeline-id: branch ✅ | main (b00fe63) ❌ | latest (v1.25.0) ❌

main (b00fe63):

acceptance_test.go:1345: Diff:
        --- pipelines/test/pipeline-id/output.txt
        +++ /tmp/TestAcceptpipelinestestpipeline-id2132709653/002/output.txt
        @@ -43,8 +43,4 @@
         
         === Outside a project, --pipeline-id is required
         >>> musterr [CLI] pipelines test tests
        -Error: Unable to locate the bundle root: databricks.yml not found.
        -
        -Run this command from a directory that contains a bundle, or set the DATABRICKS_BUNDLE_ROOT environment variable to the bundle root directory.
        -Alternatively, create a new bundle with 'databricks bundle init'.
        -
        +Error: identify the pipeline with --pipeline-id

latest (v1.25.0):

acceptance_test.go:1345: Diff:
        --- pipelines/test/pipeline-id/output.txt
        +++ /tmp/TestAcceptpipelinestestpipeline-id3554862357/002/output.txt
        @@ -1,50 +1,55 @@
         
         === Relative test selections resolve against the pipeline root_path
        ->>> MSYS_NO_PATHCONV=1 [CLI] pipelines test --pipeline-id [UUID][0] tests/test_orders.py::test_backfill /Workspace/Users/someone@example.com/other/tests
        -Pipeline: [UUID][0]
        -Update: [UUID][1]
        +>>> MSYS_NO_PATHCONV=1 [CLI] pipelines test --pipeline-id [UUID] tests/test_orders.py::test_backfill /Workspace/Users/someone@example.com/other/tests
        +Error: unknown flag: --pipeline-id
         
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {
        -      "test_selection": [
        -        "/Workspace/Users/someone@example.com/orders/tests/test_orders.py::test_backfill",
        -        "/Workspace/Users/someone@example.com/other/tests"
        -      ]
        -    }
        -  }
        -}
        +Usage:
        +  databricks pipelines [flags]
        +  databricks pipelines [command]
         
        -=== Without a root_path, test selections must be workspace paths
        ->>> musterr [CLI] pipelines test --pipeline-id [UUID][2] tests
        -Error: cannot resolve test selection "tests": the pipeline has no root_path; use an absolute workspace path such as /Workspace/Users/someone@example.com/tests
        +Main Commands
        +  deploy                Deploy pipelines
        +  destroy               Destroy a pipelines project
        +  dry-run               Validate correctness of the pipeline's graph
        +  generate              Generate pipeline configuration
        +  history               Retrieve past runs for a pipeline
        +  init                  Initialize a new pipelines project
        +  logs                  Retrieve events for a pipeline
        +  open                  Open a pipeline in the browser
        +  run                   Run a pipeline
        +  show                  Preview a table's columns and sample rows
        +  stop                  Stop a pipeline
         
        ->>> MSYS_NO_PATHCONV=1 [CLI] pipelines test --pipeline-id [UUID][2] /Workspace/Users/someone@example.com/tests/test_other.py::test_x
        -Pipeline: [UUID][2]
        -Update: [UUID][3]
        +Management Commands
        +  apply-environment     *Public Preview* Apply the latest environment to the pipeline.
        +  clone                 Clone a pipeline.
        +  create                Create a pipeline.
        +  delete                Delete a pipeline.
        +  get                   Get a pipeline.
        +  get-update            Get a pipeline update.
        +  list-pipeline-events  List pipeline events.
        +  list-pipelines        List pipelines.
        +  list-updates          List pipeline updates.
        +  start-update          Start a pipeline.
        +  update                Edit a pipeline.
         
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][2]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {
        -      "test_selection": [
        -        "/Workspace/Users/someone@example.com/tests/test_other.py::test_x"
        -      ]
        -    }
        -  }
        -}
        +Permission Commands
        +  get-permission-levels Get pipeline permission levels.
        +  get-permissions       Get pipeline permissions.
        +  set-permissions       Set pipeline permissions.
        +  update-permissions    Update pipeline permissions.
         
        -=== Outside a project, --pipeline-id is required
        ->>> musterr [CLI] pipelines test tests
        -Error: Unable to locate the bundle root: databricks.yml not found.
        +Flags:
        +  -h, --help          help for pipelines
        +      --var strings   set values for variables defined in project config. Example: --var="foo=bar"
         
        -Run this command from a directory that contains a bundle, or set the DATABRICKS_BUNDLE_ROOT environment variable to the bundle root directory.
        -Alternatively, create a new bundle with 'databricks bundle init'.
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
         
        +Use "databricks pipelines [command] --help" for more information about a command.
        +
        +
        +Exit code: 1

TestAccept/pipelines/test/selection: branch ✅ | main (b00fe63) ❌ | latest (v1.25.0) ❌

main (b00fe63):

acceptance_test.go:1345: Diff:
        --- pipelines/test/selection/output.txt
        +++ /tmp/TestAcceptpipelinestestselection1687219798/002/output.txt
        @@ -4,80 +4,10 @@
         Created pipelines.my_pipeline
         Files: 3 uploaded, 0 deleted
         Resources: 1 created, 0 changed, 0 deleted, 0 unchanged
        -View your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID][0]?w=[WORKSPACE_ID]
        +View your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID]?w=[WORKSPACE_ID]
         
         === Run all tests of the only pipeline
         >>> [CLI] pipelines test
        -Recommendation: This command runs the last deployed version of the code
        +Error: identify the pipeline with --pipeline-id
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        -
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][1]
        -
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true
        -  }
        -}
        -
        -=== Select tests by relative path and test ID, with pytest arguments in both --pytest-arg forms
        ->>> MSYS_NO_PATHCONV=1 [CLI] pipelines test my_pipeline tests tests/test_orders.py::test_backfill /Workspace/Users/someone@example.com/tests/test_other.py --pytest-arg=-k --pytest-arg=orders and not slow --pytest-arg -m --pytest-arg smoke,fast
        -Recommendation: This command runs the last deployed version of the code
        -
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        -
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][2]
        -
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {
        -      "test_selection": [
        -        "/Workspace/Users/[USERNAME]/.bundle/test-pipeline-test-selection/default/files/tests",
        -        "/Workspace/Users/[USERNAME]/.bundle/test-pipeline-test-selection/default/files/tests/test_orders.py::test_backfill",
        -        "/Workspace/Users/someone@example.com/tests/test_other.py"
        -      ],
        -      "pytest_args": [
        -        "-k",
        -        "orders and not slow",
        -        "-m",
        -        "smoke,fast"
        -      ]
        -    }
        -  }
        -}
        -
        -=== Relative paths resolve from the working directory
        ->>> [CLI] pipelines test my_pipeline test_orders.py::test_valid_order
        -Recommendation: This command runs the last deployed version of the code
        -
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        -
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][3]
        -
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {
        -      "test_selection": [
        -        "/Workspace/Users/[USERNAME]/.bundle/test-pipeline-test-selection/default/files/tests/test_orders.py::test_valid_order"
        -      ]
        -    }
        -  }
        -}
        -
        -=== Test selections require the pipeline KEY first
        ->>> musterr [CLI] pipelines test tests/test_orders.py
        -Error: "tests/test_orders.py" is not a pipeline in this bundle (pipelines: my_pipeline). Usage: databricks pipelines test my_pipeline tests/test_orders.py
        +Exit code: 1
        
FAIL

latest (v1.25.0):

acceptance_test.go:1345: Diff:
        --- pipelines/test/selection/output.txt
        +++ /tmp/TestAcceptpipelinestestselection1568603068/002/output.txt
        @@ -4,80 +4,59 @@
         Created pipelines.my_pipeline
         Files: 3 uploaded, 0 deleted
         Resources: 1 created, 0 changed, 0 deleted, 0 unchanged
        -View your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID][0]?w=[WORKSPACE_ID]
        +View your pipeline my_pipeline here: [DATABRICKS_URL]/pipelines/[UUID]?w=[WORKSPACE_ID]
         
         === Run all tests of the only pipeline
         >>> [CLI] pipelines test
        -Recommendation: This command runs the last deployed version of the code
        +Error: unknown command "test" for "databricks pipelines"
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        +Usage:
        +  databricks pipelines [flags]
        +  databricks pipelines [command]
         
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][1]
        +Main Commands
        +  deploy                Deploy pipelines
        +  destroy               Destroy a pipelines project
        +  dry-run               Validate correctness of the pipeline's graph
        +  generate              Generate pipeline configuration
        +  history               Retrieve past runs for a pipeline
        +  init                  Initialize a new pipelines project
        +  logs                  Retrieve events for a pipeline
        +  open                  Open a pipeline in the browser
        +  run                   Run a pipeline
        +  show                  Preview a table's columns and sample rows
        +  stop                  Stop a pipeline
         
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true
        -  }
        -}
        +Management Commands
        +  apply-environment     *Public Preview* Apply the latest environment to the pipeline.
        +  clone                 Clone a pipeline.
        +  create                Create a pipeline.
        +  delete                Delete a pipeline.
        +  get                   Get a pipeline.
        +  get-update            Get a pipeline update.
        +  list-pipeline-events  List pipeline events.
        +  list-pipelines        List pipelines.
        +  list-updates          List pipeline updates.
        +  start-update          Start a pipeline.
        +  update                Edit a pipeline.
         
        -=== Select tests by relative path and test ID, with pytest arguments in both --pytest-arg forms
        ->>> MSYS_NO_PATHCONV=1 [CLI] pipelines test my_pipeline tests tests/test_orders.py::test_backfill /Workspace/Users/someone@example.com/tests/test_other.py --pytest-arg=-k --pytest-arg=orders and not slow --pytest-arg -m --pytest-arg smoke,fast
        -Recommendation: This command runs the last deployed version of the code
        +Permission Commands
        +  get-permission-levels Get pipeline permission levels.
        +  get-permissions       Get pipeline permissions.
        +  set-permissions       Set pipeline permissions.
        +  update-permissions    Update pipeline permissions.
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        +Flags:
        +  -h, --help          help for pipelines
        +      --var strings   set values for variables defined in project config. Example: --var="foo=bar"
         
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][2]
        +Global Flags:
        +      --debug            enable debug logging
        +  -o, --output type      output type: text or json (default text)
        +  -p, --profile string   ~/.databrickscfg profile
        +  -t, --target string    bundle target to use (if applicable)
         
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {
        -      "test_selection": [
        -        "/Workspace/Users/[USERNAME]/.bundle/test-pipeline-test-selection/default/files/tests",
        -        "/Workspace/Users/[USERNAME]/.bundle/test-pipeline-test-selection/default/files/tests/test_orders.py::test_backfill",
        -        "/Workspace/Users/someone@example.com/tests/test_other.py"
        -      ],
        -      "pytest_args": [
        -        "-k",
        -        "orders and not slow",
        -        "-m",
        -        "smoke,fast"
        -      ]
        -    }
        -  }
        -}
        +Use "databricks pipelines [command] --help" for more information about a command.
         
        -=== Relative paths resolve from the working directory
        ->>> [CLI] pipelines test my_pipeline test_orders.py::test_valid_order
        -Recommendation: This command runs the last deployed version of the code
         
        -If you've made local changes, run 'databricks pipelines deploy' first to ensure they are included.
        -
        -Pipeline: my_pipeline ([UUID][0])
        -Update: [UUID][3]
        -
        ->>> print_requests.py //updates
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/pipelines/[UUID][0]/updates",
        -  "body": {
        -    "test_only": true,
        -    "test_options": {
        -      "test_selection": [
        -        "/Workspace/Users/[USERNAME]/.bundle/test-pipeline-test-selection/default/files/tests/test_orders.py::test_valid_order"
        -      ]
        -    }
        -  }
        -}
        -
        -=== Test selections require the pipeline KEY first
        ->>> musterr [CLI] pipelines test tests/test_orders.py
        -Error: "tests/test_orders.py" is not a pipeline in this bundle (pipelines: my_pipeline). Usage: databricks pipelines test my_pipeline tests/test_orders.py
        +Exit code: 1

[pranshupand-db]

LGTM. Follow-ups for the stack are in the #7297 review: #7297

[eng-dev-ecosystem-bot]

❌ Integration test report

Commit: ab02d65

Run: 42933958616

	Env	❌​FAIL	🔄​flaky	✅​pass	🙈​skip	Time
❌​	aws linux-2core-8gb	4		1527	1273	145:12
❌​	aws-windows-latest-4core-16gb	4	4	1442	1300	184:38
❌​	azure linux-2core-8gb	4		1507	1281	126:27
❌​	gcp linux-2core-8gb	4	2	1512	1279	140:35

10 interesting tests: 6 flaky, 4 FAIL

	Test Name	aws linux-2core-8gb	aws-windows-latest-4core-16gb	azure linux-2core-8gb	gcp linux-2core-8gb
❌​	TestAccept	❌​F	❌​F	❌​F	❌​F
🔄​	TestAccept/bundle/debug/fetch-repository-info	✅​p	✅​p	✅​p	🔄​f
🔄​	TestAccept/bundle/debug/fetch-repository-info/DMS=true	✅​p	✅​p	✅​p	🔄​f
❌​	TestAccept/bundle/resources/alerts/basic	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/resources/alerts/basic/DMS=	❌​F	❌​F	❌​F	❌​F
❌​	TestAccept/bundle/resources/alerts/basic/DMS=true	❌​F	❌​F	❌​F	❌​F
🔄​	TestAccept/bundle/resources/apps/lifecycle-started/DMS=	✅​p	🔄​f	✅​p	✅​p
🔄​	TestAccept/bundle/resources/apps/lifecycle-started/DMS=true	✅​p	🔄​f	✅​p	✅​p
🔄​	TestAccept/bundle/resources/postgres_synced_tables/recreate	✅​p	🔄​f	✅​p	✅​p
🔄​	TestAccept/bundle/resources/postgres_synced_tables/recreate/DMS=	✅​p	🔄​f	✅​p	✅​p

Top 50 slowest tests (at least 2 minutes):

duration	env	testname
13:10	azure linux-2core-8gb	TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
9:58	aws linux-2core-8gb	TestAccept/bundle/resources/apps/lifecycle-started/DMS=
9:53	azure linux-2core-8gb	TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
9:33	aws-windows-latest-4core-16gb	TestAccept/bundle/resources/apps/lifecycle-started/DMS=
9:21	aws-windows-latest-4core-16gb	TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
9:10	aws-windows-latest-4core-16gb	TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
9:02	aws-windows-latest-4core-16gb	TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
8:22	aws linux-2core-8gb	TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
8:18	gcp linux-2core-8gb	TestAccept/bundle/resources/apps/lifecycle-started/DMS=
8:09	azure linux-2core-8gb	TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
7:39	gcp linux-2core-8gb	TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=true
7:39	azure linux-2core-8gb	TestAccept/bundle/resources/apps/lifecycle-started/DMS=
7:26	gcp linux-2core-8gb	TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
7:26	aws linux-2core-8gb	TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
7:15	aws linux-2core-8gb	TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=
7:07	gcp linux-2core-8gb	TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=
7:07	aws linux-2core-8gb	TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
7:00	gcp linux-2core-8gb	TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
7:00	aws-windows-latest-4core-16gb	TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
6:58	aws-windows-latest-4core-16gb	TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
6:49	aws-windows-latest-4core-16gb	TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
6:42	gcp linux-2core-8gb	TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
6:40	gcp linux-2core-8gb	TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
6:39	aws linux-2core-8gb	TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
6:37	aws linux-2core-8gb	TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
6:34	aws linux-2core-8gb	TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
6:32	aws linux-2core-8gb	TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
6:19	aws-windows-latest-4core-16gb	TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
6:18	gcp linux-2core-8gb	TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
6:17	gcp linux-2core-8gb	TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
6:01	azure linux-2core-8gb	TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
5:40	aws linux-2core-8gb	TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
5:23	aws-windows-latest-4core-16gb	TestAccept/bundle/deploy/spark-jar-task/DMS=
5:22	aws linux-2core-8gb	TestAccept/bundle/deploy/spark-jar-task/DMS=true
5:21	aws linux-2core-8gb	TestAccept/bundle/deploy/spark-jar-task/DMS=
5:16	gcp linux-2core-8gb	TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
4:52	azure linux-2core-8gb	TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
4:43	gcp linux-2core-8gb	TestAccept/bundle/deploy/spark-jar-task/DMS=
4:26	aws linux-2core-8gb	TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
4:21	azure linux-2core-8gb	TestAccept/bundle/resources/apps/lifecycle-started-toggle/DMS=true
4:04	azure linux-2core-8gb	TestAccept/bundle/deploy/spark-jar-task/DMS=true
3:59	aws linux-2core-8gb	TestAccept/bundle/resources/clusters/deploy/simple/DMS=
3:36	aws linux-2core-8gb	TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=1
3:34	aws-windows-latest-4core-16gb	TestAccept/bundle/resources/clusters/deploy/simple/DMS=
3:31	gcp linux-2core-8gb	TestAccept/bundle/deploy/spark-jar-task/DMS=true
3:30	aws linux-2core-8gb	TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=1
3:29	aws-windows-latest-4core-16gb	TestAccept/bundle/resources/clusters/resize-terminated-fallback/DMS=true
3:27	aws-windows-latest-4core-16gb	TestAccept/bundle/resources/clusters/deploy/data_security_mode/DMS=true
3:25	aws linux-2core-8gb	TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
3:24	aws linux-2core-8gb	TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1