Repository navigation
Record deployment history for bundle resource bindings #7400
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
shreyas-goenka
wants to merge
29
commits into
main
Choose a base branch
from
shreyas-goenka_data/shreyas-goenka/record-bind-history
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
29 commits
Select commit
Hold shift + click to select a range
0e9694c
Record deployment history for bundle resource bindings
shreyas-goenka 5cc96b1
Move shared deployment lookup into DMS library
shreyas-goenka a1f0527
Address bind history self-review findings
shreyas-goenka f0ad923
Keep binding registration separate from existing deploy code
shreyas-goenka 786a2bd
Move binding deployment registration into bind phase
shreyas-goenka 11f62f7
Explain the immutable binding regression test
shreyas-goenka 43303ca
Simplify the immutable binding test comment
shreyas-goenka e22b9a1
Clarify immutable binding test coverage
shreyas-goenka fddb933
Explain the observed immutable bind regression
shreyas-goenka a7fa8ff
Clarify the immutable bind regression assertion
shreyas-goenka f268f40
Use requested wording for immutable bind test comment
shreyas-goenka 0af3d00
Limit immutable history test to successful binding
shreyas-goenka 0ba8782
Record bind history without rewriting resource state
shreyas-goenka 68f395e
Parse the last recorded deployment version directly
shreyas-goenka 7a5b4e8
testserver: accept Feature Store resources in DMS versions
shreyas-goenka db0cae5
Remove feature warning workarounds after fixing the DMS stub
shreyas-goenka 9c59142
Normalize summary deployment stamps with nostamp
shreyas-goenka c9b2a70
Remove nostamp inline doctests
shreyas-goenka 7f9307f
Explain summary DMS stamp normalization
shreyas-goenka 4d376f2
Remove the DMS test-server regression test
shreyas-goenka 82ec217
Number the summary stamp normalization comment
shreyas-goenka 9fb6767
Publish bind history directly through the DMS SDK
shreyas-goenka 783f794
Remove redundant version parsing unit coverage
shreyas-goenka 076088c
Pass binding state through the bind result for history
shreyas-goenka de7b69d
Make bind history metadata optional and DMS-specific
shreyas-goenka 93d3dcf
Inline bind history metadata selection
shreyas-goenka 3456f0a
Fix moved parser test and stabilize cloud bind preview
shreyas-goenka 8177299
Record binds as updates through the shared DMS buffer
shreyas-goenka 6da40a6
Commit approved binds through the StateDB history lifecycle
shreyas-goenka File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| * Record deployment history when binding existing resources with `bundle deployment bind` when deployment history is enabled. ([#7400](https://github.com/databricks/cli/pull/7400)) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4 changes: 4 additions & 0 deletions
4
acceptance/bundle/deployment/bind/feature_spec/out.bind.plain.txt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
|
|
||
| >>> [CLI] bundle deployment bind spec1 main.fspec_[UNIQUE_NAME].boundspec --auto-approve | ||
| Successfully bound feature_spec with an id 'main.fspec_[UNIQUE_NAME].boundspec' | ||
| Run 'bundle deploy' to deploy changes to your workspace |
5 changes: 5 additions & 0 deletions
5
acceptance/bundle/deployment/bind/feature_spec/out.bind.recorded.txt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
|
|
||
| >>> [CLI] bundle deployment bind spec1 main.fspec_[UNIQUE_NAME].boundspec --auto-approve | ||
| Warn: Failed to record bind history: failed to create deployment version: operations.resource_key must have a known resource type prefix (e.g. 'jobs.', 'pipelines.'): feature_specs.spec1 | ||
| Successfully bound feature_spec with an id 'main.fspec_[UNIQUE_NAME].boundspec' | ||
| Run 'bundle deploy' to deploy changes to your workspace |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,2 +1,3 @@ | ||
| Cloud = true | ||
| EnvMatrix.DMS = ["", "true"] | ||
| EnvMatrix.OUTPUT_MODE = ["plain", "recorded"] |
2 changes: 2 additions & 0 deletions
2
acceptance/bundle/deployment/bind/feature_spec/out.unbind.plain.txt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
|
|
||
| >>> [CLI] bundle deployment unbind spec1 |
2 changes: 2 additions & 0 deletions
2
acceptance/bundle/deployment/bind/feature_spec/out.unbind.recorded.txt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
|
|
||
| >>> [CLI] bundle deployment unbind spec1 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -12,7 +12,7 @@ rm -f "$OUT_REQUESTS" | |
| trace $CLI bundle deployment bind foo $NEW_PIPELINE_ID --auto-approve &> out.bind-success.txt | ||
| trace print_requests.py '^//import-file/' '^//workspace/' | ||
|
|
||
| trace $CLI bundle summary -o json | jq .resources > out.summary.json | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. added because now the summary contains deployment and version ID |
||
| trace $CLI bundle summary -o json | jq .resources | nostamp.py > out.summary.json | ||
| trace $CLI bundle plan | ||
|
|
||
| trace musterr $CLI bundle deploy | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -9,7 +9,7 @@ rm -f "$OUT_REQUESTS" | |
| trace $CLI bundle deployment bind foo $NEW_PIPELINE_ID --auto-approve &> out.bind-success.txt | ||
| print_requests.py '^//import-file/' '^//workspace/delete' | ||
|
|
||
| trace $CLI bundle summary -o json > out.summary.json | ||
| trace $CLI bundle summary -o json | nostamp.py > out.summary.json | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. updated nostamp.py to scrub the deployment ID from the summary output as well. |
||
| trace $CLI bundle plan | ||
|
|
||
| rm -f "$OUT_REQUESTS" | ||
|
|
||
13 changes: 13 additions & 0 deletions
13
acceptance/bundle/dms/bind-history-failures/databricks.yml.tmpl
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| bundle: | ||
| name: dms-bind-history-failures-$UNIQUE_NAME | ||
|
|
||
| resources: | ||
| jobs: | ||
| create_failed: | ||
| name: create_failed | ||
| record_failed: | ||
| name: record_failed | ||
| complete_failed: | ||
| name: complete_failed | ||
| upload_failed: | ||
| name: upload_failed |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| Cloud = false | ||
| EnvMatrix.DMS = ["true"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,79 @@ | ||
|
|
||
| === A version creation failure does not prevent binding | ||
|
|
||
| >>> fault.py POST /api/2.0/bundle/deployments/*/versions 400 0 1 | ||
|
|
||
| >>> [CLI] bundle deployment bind create_failed [JOB_ID] --auto-approve | ||
| Warn: Failed to record bind history: failed to create deployment version: Fault injected by test. | ||
| Successfully bound job with an id '[JOB_ID]' | ||
| Run 'bundle deploy' to deploy changes to your workspace | ||
|
|
||
| === An operation recording failure does not prevent binding | ||
|
|
||
| >>> fault.py PATCH /api/2.0/bundle/** 400 0 1 | ||
|
|
||
| >>> [CLI] bundle deployment bind record_failed [JOB_ID] --auto-approve | ||
| Warn: Failed to record deployment history: recording operation for resources.jobs.record_failed: Fault injected by test. | ||
| Successfully bound job with an id '[JOB_ID]' | ||
| Run 'bundle deploy' to deploy changes to your workspace | ||
|
|
||
| === A version completion failure does not prevent binding | ||
|
|
||
| >>> fault.py POST /api/2.0/bundle/deployments/*/versions/*/complete 400 0 1 | ||
|
|
||
| >>> [CLI] bundle deployment bind complete_failed [JOB_ID] --auto-approve | ||
| Warn: Failed to complete deployment history version: Fault injected by test. | ||
| Successfully bound job with an id '[JOB_ID]' | ||
| Run 'bundle deploy' to deploy changes to your workspace | ||
|
|
||
| === All bindings survived the remote state round trips | ||
|
|
||
| >>> jq .state | map_values(.__id__) .databricks/bundle/default/resources.json | ||
| { | ||
| "resources.jobs.complete_failed": "[JOB_ID]", | ||
| "resources.jobs.create_failed": "[JOB_ID]", | ||
| "resources.jobs.record_failed": "[JOB_ID]" | ||
| } | ||
|
|
||
| >>> [CLI] jobs get [JOB_ID] | ||
| { | ||
| "job_id": [JOB_ID], | ||
| "name": "bind-target-[UNIQUE_NAME]" | ||
| } | ||
|
|
||
| >>> print_requests.py --dms //versions --oneline | ||
| {"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions", "q": {"version_id": "1"}, "body": {"cli_version": "[CLI_VERSION]", "operations": [{"action_type": "OPERATION_ACTION_TYPE_UPDATE", "resource_key": "jobs.create_failed"}], "version_type": "VERSION_TYPE_DEPLOY"}} | ||
| {"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions", "q": {"version_id": "2"}, "body": {"cli_version": "[CLI_VERSION]", "operations": [{"action_type": "OPERATION_ACTION_TYPE_UPDATE", "resource_key": "jobs.record_failed"}], "version_type": "VERSION_TYPE_DEPLOY"}} | ||
| {"method": "PATCH", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/2/operations/jobs.record_failed", "q": {"update_mask": "state,error_message,resource_id,status"}, "body": {"error_message": "", "resource_id": "[JOB_ID]", "sequence_id": 0, "state": "{\"state\":{\"deployment\":{\"kind\":\"BUNDLE\",\"metadata_file_path\":\"/Workspace/Users/[USERNAME]/.bundle/dms-bind-history-failures-[UNIQUE_NAME]/default/state/metadata.json\"},\"edit_mode\":\"UI_LOCKED\",\"format\":\"MULTI_TASK\",\"max_concurrent_runs\":1,\"name\":\"record_failed\",\"queue\":{\"enabled\":true}}}", "status": "OPERATION_STATUS_SUCCEEDED"}} | ||
| {"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/2/complete", "body": {"completion_reason": "VERSION_COMPLETE_FAILURE"}} | ||
| {"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions", "q": {"version_id": "3"}, "body": {"cli_version": "[CLI_VERSION]", "operations": [{"action_type": "OPERATION_ACTION_TYPE_UPDATE", "resource_key": "jobs.complete_failed"}], "previous_version_id": "2", "version_type": "VERSION_TYPE_DEPLOY"}} | ||
| {"method": "PATCH", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/3/operations/jobs.complete_failed", "q": {"update_mask": "state,error_message,resource_id,status"}, "body": {"error_message": "", "resource_id": "[JOB_ID]", "sequence_id": 0, "state": "{\"state\":{\"deployment\":{\"kind\":\"BUNDLE\",\"metadata_file_path\":\"/Workspace/Users/[USERNAME]/.bundle/dms-bind-history-failures-[UNIQUE_NAME]/default/state/metadata.json\"},\"edit_mode\":\"UI_LOCKED\",\"format\":\"MULTI_TASK\",\"max_concurrent_runs\":1,\"name\":\"complete_failed\",\"queue\":{\"enabled\":true}}}", "status": "OPERATION_STATUS_SUCCEEDED"}} | ||
| {"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/3/complete", "body": {"completion_reason": "VERSION_COMPLETE_SUCCESS"}} | ||
|
|
||
| === A remote state upload failure fails the bind version | ||
|
|
||
| >>> fault.py POST /api/2.0/workspace-files/import-file/**resources.json 400 0 1 | ||
|
|
||
| >>> musterr [CLI] bundle deployment bind upload_failed [JOB_ID] --auto-approve | ||
| Error: Fault injected by test. (400 INJECTED) | ||
|
|
||
| Endpoint: POST [DATABRICKS_URL]/api/2.0/workspace-files/import-file/Workspace%2FUsers%2F[USERNAME]%2F.bundle%2Fdms-bind-history-failures-[UNIQUE_NAME]%2Fdefault%2Fstate%2Fresources.json?overwrite=true | ||
| HTTP Status: 400 Bad Request | ||
| API error_code: INJECTED | ||
| API message: Fault injected by test. | ||
|
|
||
|
|
||
| >>> print_requests.py --dms //versions --oneline | ||
| {"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions", "q": {"version_id": "4"}, "body": {"cli_version": "[CLI_VERSION]", "operations": [{"action_type": "OPERATION_ACTION_TYPE_UPDATE", "resource_key": "jobs.upload_failed"}], "previous_version_id": "3", "version_type": "VERSION_TYPE_DEPLOY"}} | ||
| {"method": "PATCH", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/4/operations/jobs.upload_failed", "q": {"update_mask": "state,error_message,resource_id,status"}, "body": {"error_message": "", "resource_id": "[JOB_ID]", "sequence_id": 0, "state": "{\"state\":{\"deployment\":{\"kind\":\"BUNDLE\",\"metadata_file_path\":\"/Workspace/Users/[USERNAME]/.bundle/dms-bind-history-failures-[UNIQUE_NAME]/default/state/metadata.json\"},\"edit_mode\":\"UI_LOCKED\",\"format\":\"MULTI_TASK\",\"max_concurrent_runs\":1,\"name\":\"upload_failed\",\"queue\":{\"enabled\":true}}}", "status": "OPERATION_STATUS_SUCCEEDED"}} | ||
| {"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/4/complete", "body": {"completion_reason": "VERSION_COMPLETE_FAILURE"}} | ||
|
|
||
| === Remote state still contains only the successful bindings | ||
|
|
||
| >>> [CLI] bundle summary -o json | ||
| { | ||
| "complete_failed": "[JOB_ID]", | ||
| "create_failed": "[JOB_ID]", | ||
| "record_failed": "[JOB_ID]", | ||
| "upload_failed": null | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| envsubst < databricks.yml.tmpl > databricks.yml | ||
| job_id=$($CLI jobs create --json '{"name": "bind-target-'$UNIQUE_NAME'"}' | jq -r '.job_id') | ||
| add_repl "$job_id" JOB_ID | ||
|
|
||
| title "A version creation failure does not prevent binding" | ||
| trace fault.py "POST /api/2.0/bundle/deployments/*/versions" 400 0 1 | ||
| trace $CLI bundle deployment bind create_failed "$job_id" --auto-approve | ||
| rm -rf .databricks | ||
|
|
||
| title "An operation recording failure does not prevent binding" | ||
| trace fault.py "PATCH /api/2.0/bundle/**" 400 0 1 | ||
| trace $CLI bundle deployment bind record_failed "$job_id" --auto-approve | ||
| rm -rf .databricks | ||
|
|
||
| title "A version completion failure does not prevent binding" | ||
| trace fault.py "POST /api/2.0/bundle/deployments/*/versions/*/complete" 400 0 1 | ||
| trace $CLI bundle deployment bind complete_failed "$job_id" --auto-approve | ||
|
|
||
| title "All bindings survived the remote state round trips" | ||
| trace jq '.state | map_values(.__id__)' .databricks/bundle/default/resources.json | ||
| trace $CLI jobs get "$job_id" | jq '{job_id, name: .settings.name}' | ||
| trace print_requests.py --dms //versions --oneline | ||
|
|
||
|
|
||
| title "A remote state upload failure fails the bind version" | ||
| trace fault.py "POST /api/2.0/workspace-files/import-file/**resources.json" 400 0 1 | ||
| trace musterr $CLI bundle deployment bind upload_failed "$job_id" --auto-approve | ||
| trace print_requests.py --dms //versions --oneline | ||
|
|
||
| rm -rf .databricks | ||
| title "Remote state still contains only the successful bindings" | ||
| trace $CLI bundle summary -o json | jq '.resources.jobs | map_values(.id // null)' | ||
| rm -f "$OUT_REQUESTS" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Cloud = false # Inject failures into the deployment history service. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| bundle: | ||
| name: dms-bind-immutable-$UNIQUE_NAME | ||
| workspace: | ||
| sync_mode: immutable | ||
| resources: | ||
| jobs: | ||
| foo: | ||
| name: foo |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| Cloud = false | ||
| EnvMatrix.DMS = ["true"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
|
|
||
| >>> [CLI] bundle deployment bind foo [JOB_ID] --auto-approve | ||
| Successfully bound job with an id '[JOB_ID]' | ||
| Run 'bundle deploy' to deploy changes to your workspace | ||
|
|
||
| >>> MSYS_NO_PATHCONV=1 [CLI] bundle-deployments list-resources deployments/[DEPLOYMENT_ID] -o json | ||
| [ | ||
| { | ||
| "last_action_type": "OPERATION_ACTION_TYPE_UPDATE", | ||
| "resource_id": "[JOB_ID]", | ||
| "resource_key": "jobs.foo" | ||
| } | ||
| ] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| envsubst < databricks.yml.tmpl > databricks.yml | ||
| job_id=$($CLI jobs create --json '{"name": "bind-target-'$UNIQUE_NAME'"}' | jq -r '.job_id') | ||
| add_repl "$job_id" JOB_ID | ||
|
|
||
| trace $CLI bundle deployment bind foo "$job_id" --auto-approve | ||
| deployment_id=$(MSYS_NO_PATHCONV=1 $CLI workspace get-status "/Workspace/Users/${CURRENT_USER_NAME}/.bundle/dms-bind-immutable-${UNIQUE_NAME}/default/state/resources.deployment.json" -o json | jq -r '.object_id') | ||
| trace MSYS_NO_PATHCONV=1 $CLI bundle-deployments list-resources "deployments/${deployment_id}" -o json | jq -S 'map({resource_key, resource_id, last_action_type})' | ||
| rm -f "$OUT_REQUESTS" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| # This test ensures that deployment history works with bind and immutable folders. This case is interesting | ||
| # because by default we record the snapshot path in deploy, but that is not available during bind. So it's important | ||
| # to ensure that we don't do that during deployment bind, and recording history continues to work. | ||
| Cloud = false |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,2 +1,2 @@ | ||
| Cloud = false | ||
| Cloud = true | ||
| EnvMatrix.DMS = ["true"] |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Running the DMS=true variants of the three scripts this PR pipes through nostamp (bind/pipelines/update, bind/pipelines/recreate, bind/dashboard/recreation) with the scripts at their base revision passes locally at this HEAD: after a first bind there is no marker, so summary and the state dump never carry
bundle.deploymentIDs. What difference is shape 5 and these three pipes guarding against, a cloud run or a later PR in the stack? If the latter, can they move there?