Skip to content

Commit 8a72f2d

Browse files
Sankalp-MittalSankalp-Mittal
authored
Migrate bundle state filer read off workspace-files to /workspace/export (#6219)
# bundle: read deploy state via /workspace/export instead of deprecated workspace-files API ## Summary Switch the DABs state filer read (`bundle/deploy/filer.go`, `stateFiler.Read`) from `GET /api/2.0/workspace-files/{path}` to the SDK's `Workspace.Download(ctx, absPath)`, which issues `GET /api/2.0/workspace/export?path=<path>&direct_download=true`. This reads the bundle state files (`terraform.tfstate`, `resources.json`) from the workspace. ## Why The `workspace-files` endpoint is deprecated and is mapped to **no API scope**. Under context-based ingress (CBI) with fine-grained scoped tokens it was only reachable via the `all_apis` master scope, so `bundle deploy` returned 403s for customers on least-privilege scoped tokens. `/workspace/export` is properly scoped, so state reads now work under least-privilege auth. ## Implementation Reads go through the SDK's `Workspace.Download` helper, so there is no separate API client, no manual `auth.WorkspaceIDHeaders` routing headers (the SDK sets these), and no intermediate buffer — `Download` returns the streaming body directly. `direct_download=true` is load-bearing: plain `/workspace/export` returns base64 JSON capped at 10 MB, while the streaming variant handles files up to 500 MB, so large state files stream through fine. ## Testing - Regenerated acceptance goldens: request traces now show `GET /api/2.0/workspace/export` with `direct_download=true` (`user_agent`, `state` suites). - Validated against a real workspace (dogfood): streaming export returns the state bytes correctly end-to-end. This pull request and its description were written by Isaac.
1 parent f06cba8 commit 8a72f2d

12 files changed

Lines changed: 86 additions & 54 deletions

‎acceptance/bundle/state/basic/out.state.txt‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,16 @@
11
{
22
"method": "GET",
3-
"path": "/api/2.0/workspace-files/Workspace/Users/[USERNAME]/.bundle/state/default/state/STATE_FILENAME"
3+
"path": "/api/2.0/workspace/export",
4+
"q": {
5+
"direct_download": "true",
6+
"path": "/Workspace/Users/[USERNAME]/.bundle/state/default/state/STATE_FILENAME"
7+
}
48
}
59
{
610
"method": "GET",
7-
"path": "/api/2.0/workspace-files/Workspace/Users/[USERNAME]/.bundle/state/default/state/deployment.json"
11+
"path": "/api/2.0/workspace/export",
12+
"q": {
13+
"direct_download": "true",
14+
"path": "/Workspace/Users/[USERNAME]/.bundle/state/default/state/deployment.json"
15+
}
816
}

‎acceptance/bundle/state/basic/script‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
trace $CLI bundle deploy
22
trace $CLI bundle deploy # We do 2 deploys because only 2nd deploy will pull state from remote after 1st created it
3-
jq 'select(.path | test("/api/2.0/workspace-files/Workspace/Users/.*/.bundle/state/default/state/"))' out.requests.txt > out.state.txt
3+
jq 'select(.path == "/api/2.0/workspace/export" and (.q.path | test("/.bundle/state/default/state/")) and (.q.path | test("deploy.lock") | not))' out.requests.txt > out.state.txt
44
rm out.requests.txt

‎acceptance/bundle/state/force_pull_commands/output.txt‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,11 @@ Resources: 1 created, 0 changed, 0 deleted, 0 unchanged
1515
>>> [CLI] bundle summary --force-pull
1616
{
1717
"method": "GET",
18-
"path": "/api/2.0/workspace-files/Workspace/Users/[USERNAME]/.bundle/force-pull-commands/default/state/STATE_FILENAME"
18+
"path": "/api/2.0/workspace/export",
19+
"q": {
20+
"direct_download": "true",
21+
"path": "/Workspace/Users/[USERNAME]/.bundle/force-pull-commands/default/state/STATE_FILENAME"
22+
}
1923
}
2024

2125
=== bundle open without --force-pull: no remote state read
@@ -29,5 +33,9 @@ Opening browser at [DATABRICKS_URL]/jobs/[NUMID]?w=[NUMID]
2933
Opening browser at [DATABRICKS_URL]/jobs/[NUMID]?w=[NUMID]
3034
{
3135
"method": "GET",
32-
"path": "/api/2.0/workspace-files/Workspace/Users/[USERNAME]/.bundle/force-pull-commands/default/state/STATE_FILENAME"
36+
"path": "/api/2.0/workspace/export",
37+
"q": {
38+
"direct_download": "true",
39+
"path": "/Workspace/Users/[USERNAME]/.bundle/force-pull-commands/default/state/STATE_FILENAME"
40+
}
3341
}

‎acceptance/bundle/state/force_pull_commands/script‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -11,19 +11,19 @@ export BROWSER="echo_browser.py"
1111
title "bundle summary without --force-pull: no remote state read\n"
1212
trace $CLI bundle summary > /dev/null
1313
touch out.requests.txt
14-
print_requests.py --get //workspace-files/
14+
print_requests.py --get //workspace/export
1515

1616
title "bundle summary --force-pull: remote state read\n"
1717
trace $CLI bundle summary --force-pull > /dev/null
1818
touch out.requests.txt
19-
print_requests.py --get //workspace-files/
19+
print_requests.py --get //workspace/export
2020

2121
title "bundle open without --force-pull: no remote state read\n"
2222
trace $CLI bundle open foo > /dev/null
2323
touch out.requests.txt
24-
print_requests.py --get //workspace-files/
24+
print_requests.py --get //workspace/export
2525

2626
title "bundle open --force-pull: remote state read\n"
2727
trace $CLI bundle open foo --force-pull > /dev/null
2828
touch out.requests.txt
29-
print_requests.py --get //workspace-files/
29+
print_requests.py --get //workspace/export

‎acceptance/bundle/user_agent/output.txt‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ MISS deploy.terraform /.well-known/databricks-config 'databricks-tf-provider/[TF
4848
MISS deploy.terraform /.well-known/databricks-config 'databricks-tf-provider/[TF_PROVIDER_VERSION] databricks-sdk-go/[SDK_VERSION] go/1.25.8 os/[OS] cli/[CLI_VERSION] terraform/1.5.5'
4949
MISS deploy.terraform /.well-known/databricks-config 'databricks-tf-provider/[TF_PROVIDER_VERSION] databricks-sdk-go/[SDK_VERSION] go/1.25.8 os/[OS] cli/[CLI_VERSION] terraform/1.5.5'
5050
MISS destroy.direct /api/2.0/preview/scim/v2/Me 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_destroy cmd-exec-id/[UUID] interactive/none auth/pat'
51-
MISS destroy.direct /api/2.0/workspace-files/Workspace/Users/[USERNAME]/.bundle/test-bundle/default/state/resources.json 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_destroy cmd-exec-id/[UUID] interactive/none auth/pat'
51+
MISS destroy.direct /api/2.0/workspace/export 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_destroy cmd-exec-id/[UUID] interactive/none auth/pat'
5252
MISS destroy.direct /api/2.0/workspace/get-status 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_destroy cmd-exec-id/[UUID] interactive/none auth/pat'
5353
MISS destroy.direct /api/2.0/workspace/get-status 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_destroy cmd-exec-id/[UUID] interactive/none auth/pat'
5454
OK destroy.direct /api/2.1/unity-catalog/schemas/mycatalog.myschema engine/direct
@@ -61,7 +61,7 @@ OK destroy.direct /api/2.0/workspace-files/import-file/Workspace/Users/[USERNA
6161
OK destroy.direct /api/2.0/workspace/delete engine/direct
6262
MISS destroy.direct /.well-known/databricks-config 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS]'
6363
MISS destroy.terraform /api/2.0/preview/scim/v2/Me 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_destroy cmd-exec-id/[UUID] interactive/none auth/pat'
64-
MISS destroy.terraform /api/2.0/workspace-files/Workspace/Users/[USERNAME]/.bundle/test-bundle/default/state/terraform.tfstate 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_destroy cmd-exec-id/[UUID] interactive/none auth/pat'
64+
MISS destroy.terraform /api/2.0/workspace/export 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_destroy cmd-exec-id/[UUID] interactive/none auth/pat'
6565
MISS destroy.terraform /api/2.0/workspace/get-status 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_destroy cmd-exec-id/[UUID] interactive/none auth/pat'
6666
MISS destroy.terraform /api/2.0/workspace/get-status 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_destroy cmd-exec-id/[UUID] interactive/none auth/pat'
6767
OK destroy.terraform /api/2.0/workspace/export engine/terraform
@@ -93,30 +93,30 @@ MISS plan.terraform /.well-known/databricks-config 'cli/[CLI_VERSION] databricks
9393
MISS plan.terraform /.well-known/databricks-config 'databricks-tf-provider/[TF_PROVIDER_VERSION] databricks-sdk-go/[SDK_VERSION] go/1.25.8 os/[OS] cli/[CLI_VERSION] terraform/1.5.5'
9494
MISS plan.terraform /.well-known/databricks-config 'databricks-tf-provider/[TF_PROVIDER_VERSION] databricks-sdk-go/[SDK_VERSION] go/1.25.8 os/[OS] cli/[CLI_VERSION] terraform/1.5.5'
9595
MISS plan2.direct /api/2.0/preview/scim/v2/Me 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_plan cmd-exec-id/[UUID] interactive/none auth/pat'
96-
MISS plan2.direct /api/2.0/workspace-files/Workspace/Users/[USERNAME]/.bundle/test-bundle/default/state/resources.json 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_plan cmd-exec-id/[UUID] interactive/none auth/pat'
96+
MISS plan2.direct /api/2.0/workspace/export 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_plan cmd-exec-id/[UUID] interactive/none auth/pat'
9797
MISS plan2.direct /api/2.0/workspace/get-status 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_plan cmd-exec-id/[UUID] interactive/none auth/pat'
9898
MISS plan2.direct /api/2.0/workspace/get-status 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_plan cmd-exec-id/[UUID] interactive/none auth/pat'
99-
OK plan2.direct /api/2.0/workspace-files/Workspace/Users/[USERNAME]/.bundle/test-bundle/default/state/deployment.json engine/direct
99+
OK plan2.direct /api/2.0/workspace/export engine/direct
100100
OK plan2.direct /api/2.0/workspace/get-status engine/direct
101101
OK plan2.direct /api/2.1/unity-catalog/schemas/mycatalog.myschema engine/direct
102102
MISS plan2.direct /.well-known/databricks-config 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS]'
103103
MISS plan2.terraform /api/2.0/preview/scim/v2/Me 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_plan cmd-exec-id/[UUID] interactive/none auth/pat'
104-
MISS plan2.terraform /api/2.0/workspace-files/Workspace/Users/[USERNAME]/.bundle/test-bundle/default/state/terraform.tfstate 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_plan cmd-exec-id/[UUID] interactive/none auth/pat'
104+
MISS plan2.terraform /api/2.0/workspace/export 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_plan cmd-exec-id/[UUID] interactive/none auth/pat'
105105
MISS plan2.terraform /api/2.0/workspace/get-status 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_plan cmd-exec-id/[UUID] interactive/none auth/pat'
106106
MISS plan2.terraform /api/2.0/workspace/get-status 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_plan cmd-exec-id/[UUID] interactive/none auth/pat'
107-
OK plan2.terraform /api/2.0/workspace-files/Workspace/Users/[USERNAME]/.bundle/test-bundle/default/state/deployment.json engine/terraform
107+
OK plan2.terraform /api/2.0/workspace/export engine/terraform
108108
OK plan2.terraform /api/2.0/workspace/get-status engine/terraform
109109
MISS plan2.terraform /.well-known/databricks-config 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS]'
110110
MISS plan2.terraform /api/2.1/unity-catalog/schemas/mycatalog.myschema 'databricks-tf-provider/[TF_PROVIDER_VERSION] databricks-sdk-go/[SDK_VERSION] go/1.25.8 os/[OS] cli/[CLI_VERSION] terraform/1.5.5 sdk/sdkv2 resource/schema auth/pat'
111111
MISS plan2.terraform /.well-known/databricks-config 'databricks-tf-provider/[TF_PROVIDER_VERSION] databricks-sdk-go/[SDK_VERSION] go/1.25.8 os/[OS] cli/[CLI_VERSION] terraform/1.5.5'
112112
MISS plan2.terraform /.well-known/databricks-config 'databricks-tf-provider/[TF_PROVIDER_VERSION] databricks-sdk-go/[SDK_VERSION] go/1.25.8 os/[OS] cli/[CLI_VERSION] terraform/1.5.5'
113113
MISS run.direct /api/2.0/preview/scim/v2/Me 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_run cmd-exec-id/[UUID] interactive/none auth/pat'
114-
MISS run.direct /api/2.0/workspace-files/Workspace/Users/[USERNAME]/.bundle/test-bundle/default/state/resources.json 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_run cmd-exec-id/[UUID] interactive/none auth/pat'
114+
MISS run.direct /api/2.0/workspace/export 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_run cmd-exec-id/[UUID] interactive/none auth/pat'
115115
MISS run.direct /api/2.0/workspace/get-status 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_run cmd-exec-id/[UUID] interactive/none auth/pat'
116116
MISS run.direct /api/2.0/workspace/get-status 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_run cmd-exec-id/[UUID] interactive/none auth/pat'
117117
MISS run.direct /.well-known/databricks-config 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS]'
118118
MISS run.terraform /api/2.0/preview/scim/v2/Me 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_run cmd-exec-id/[UUID] interactive/none auth/pat'
119-
MISS run.terraform /api/2.0/workspace-files/Workspace/Users/[USERNAME]/.bundle/test-bundle/default/state/terraform.tfstate 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_run cmd-exec-id/[UUID] interactive/none auth/pat'
119+
MISS run.terraform /api/2.0/workspace/export 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_run cmd-exec-id/[UUID] interactive/none auth/pat'
120120
MISS run.terraform /api/2.0/workspace/get-status 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_run cmd-exec-id/[UUID] interactive/none auth/pat'
121121
MISS run.terraform /api/2.0/workspace/get-status 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS] cmd/bundle_run cmd-exec-id/[UUID] interactive/none auth/pat'
122122
MISS run.terraform /.well-known/databricks-config 'cli/[CLI_VERSION] databricks-sdk-go/[SDK_VERSION] go/[GO_VERSION] os/[OS]'

‎acceptance/bundle/user_agent/simple/out.requests.destroy.direct.json‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,11 @@
1414
]
1515
},
1616
"method": "GET",
17-
"path": "/api/2.0/workspace-files/Workspace/Users/[USERNAME]/.bundle/test-bundle/default/state/resources.json"
17+
"path": "/api/2.0/workspace/export",
18+
"q": {
19+
"direct_download": "true",
20+
"path": "/Workspace/Users/[USERNAME]/.bundle/test-bundle/default/state/resources.json"
21+
}
1822
}
1923
{
2024
"headers": {

‎acceptance/bundle/user_agent/simple/out.requests.destroy.terraform.json‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,11 @@
1414
]
1515
},
1616
"method": "GET",
17-
"path": "/api/2.0/workspace-files/Workspace/Users/[USERNAME]/.bundle/test-bundle/default/state/terraform.tfstate"
17+
"path": "/api/2.0/workspace/export",
18+
"q": {
19+
"direct_download": "true",
20+
"path": "/Workspace/Users/[USERNAME]/.bundle/test-bundle/default/state/terraform.tfstate"
21+
}
1822
}
1923
{
2024
"headers": {

‎acceptance/bundle/user_agent/simple/out.requests.plan2.direct.json‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,11 @@
1414
]
1515
},
1616
"method": "GET",
17-
"path": "/api/2.0/workspace-files/Workspace/Users/[USERNAME]/.bundle/test-bundle/default/state/resources.json"
17+
"path": "/api/2.0/workspace/export",
18+
"q": {
19+
"direct_download": "true",
20+
"path": "/Workspace/Users/[USERNAME]/.bundle/test-bundle/default/state/resources.json"
21+
}
1822
}
1923
{
2024
"headers": {
@@ -49,7 +53,11 @@
4953
]
5054
},
5155
"method": "GET",
52-
"path": "/api/2.0/workspace-files/Workspace/Users/[USERNAME]/.bundle/test-bundle/default/state/deployment.json"
56+
"path": "/api/2.0/workspace/export",
57+
"q": {
58+
"direct_download": "true",
59+
"path": "/Workspace/Users/[USERNAME]/.bundle/test-bundle/default/state/deployment.json"
60+
}
5361
}
5462
{
5563
"headers": {

‎acceptance/bundle/user_agent/simple/out.requests.plan2.terraform.json‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,11 @@
1414
]
1515
},
1616
"method": "GET",
17-
"path": "/api/2.0/workspace-files/Workspace/Users/[USERNAME]/.bundle/test-bundle/default/state/terraform.tfstate"
17+
"path": "/api/2.0/workspace/export",
18+
"q": {
19+
"direct_download": "true",
20+
"path": "/Workspace/Users/[USERNAME]/.bundle/test-bundle/default/state/terraform.tfstate"
21+
}
1822
}
1923
{
2024
"headers": {
@@ -49,7 +53,11 @@
4953
]
5054
},
5155
"method": "GET",
52-
"path": "/api/2.0/workspace-files/Workspace/Users/[USERNAME]/.bundle/test-bundle/default/state/deployment.json"
56+
"path": "/api/2.0/workspace/export",
57+
"q": {
58+
"direct_download": "true",
59+
"path": "/Workspace/Users/[USERNAME]/.bundle/test-bundle/default/state/deployment.json"
60+
}
5361
}
5462
{
5563
"headers": {

‎acceptance/bundle/user_agent/simple/out.requests.run.direct.json‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,11 @@
1414
]
1515
},
1616
"method": "GET",
17-
"path": "/api/2.0/workspace-files/Workspace/Users/[USERNAME]/.bundle/test-bundle/default/state/resources.json"
17+
"path": "/api/2.0/workspace/export",
18+
"q": {
19+
"direct_download": "true",
20+
"path": "/Workspace/Users/[USERNAME]/.bundle/test-bundle/default/state/resources.json"
21+
}
1822
}
1923
{
2024
"headers": {

0 commit comments

Comments
 (0)