Skip to content

Commit fd9fedb

Browse files
tharunbhuk-dbIsaac
andcommitted
Translate the testing paths of pipelines and upload local test wheels
Relative paths in testing.sources and testing.environment.dependencies resolve to their deployed workspace paths, like libraries and environment dependencies. Local wheels in test dependencies are uploaded like other libraries, and the same-name check covers them. The testing property is only accepted with DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_PIPELINE_TESTING=true, so bundles without it are not changed. Co-authored-by: Isaac <no-reply@databricks.com>
1 parent bc013c0 commit fd9fedb

17 files changed

Lines changed: 91 additions & 3 deletions

File tree

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
bundle:
2+
name: pipeline-testing-same-name-libraries
3+
4+
resources:
5+
pipelines:
6+
orders:
7+
name: orders
8+
libraries:
9+
- file:
10+
path: ./transformations/orders.py
11+
environment:
12+
dependencies:
13+
- ./pipeline/test_helpers-0.1-py3-none-any.whl
14+
testing:
15+
sources:
16+
- glob:
17+
include: ./tests/**
18+
environment:
19+
dependencies:
20+
# A different wheel with the same file name would be uploaded to the same path.
21+
- ./tests/test_helpers-0.1-py3-none-any.whl
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Cloud = false
2+
EnvMatrix.DMS = ["", "true"]
Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
2+
>>> musterr [CLI] bundle deploy
3+
Error: Duplicate local library names: test_helpers-0.1-py3-none-any.whl
4+
at resources.pipelines.orders.environment.dependencies[0]
5+
resources.pipelines.orders.testing.environment.dependencies[0]
6+
in databricks.yml:13:13
7+
databricks.yml:21:15
8+
9+
Local library names must be unique but found libraries with the same name: pipeline/test_helpers-0.1-py3-none-any.whl, tests/test_helpers-0.1-py3-none-any.whl
10+

‎acceptance/bundle/resources/pipelines/testing-same-name-libraries/pipeline/test_helpers-0.1-py3-none-any.whl‎

Whitespace-only changes.
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
trace musterr $CLI bundle deploy
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# Runs only against the local test server.
2+
Cloud = false
3+
Env.DATABRICKS_BUNDLE_ENABLE_EXPERIMENTAL_PIPELINE_TESTING = "true"
4+
5+
[[Repls]]
6+
Old = '\\'
7+
New = '/'

‎acceptance/bundle/resources/pipelines/testing-same-name-libraries/tests/test_helpers-0.1-py3-none-any.whl‎

Whitespace-only changes.
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
def test_orders():
2+
pass
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
x = 1

‎acceptance/bundle/resources/pipelines/testing/dist/test_helpers-0.1-py3-none-any.whl‎

Whitespace-only changes.

0 commit comments

Comments
 (0)