[yolocs]

Changes

databricks auth docker configure no longer fails when several databricks-cli profiles match the selected workspace. It configures Docker and prints a warning:

Warning: multiple Databricks profiles match workspace ID 1234567890123456: dev, prod. Docker credential helper uses the first logged-in profile in this order

databricks auth docker token (the Docker credential helper) tries the matching profiles in profile order ([DEFAULT] first, then file order) and uses the first one that is logged in. It skips a profile only when it has no cached login, no refresh token, or a rejected refresh token. Any other error (network, server, timeout, cancellation) is returned, so the helper never switches identity because of a transient failure. A single matching profile behaves as before, including its error text.

To tell a cache miss apart from other errors, loadToken now keeps storage.ErrNotFound in its error chain via storage.NewNotFoundHint. databricks auth token messages are unchanged.

Why

Docker gives the credential helper only the registry host, which encodes the workspace ID, so the helper maps it back to a profile by workspace_id. auth login now saves workspace_id on every profile, so two logins to the same workspace were enough to block auth docker configure.

This replaces the "remove the duplicate profile" hint that #7363 added in v1.25.0.

Tests

• Unit: TestProfilesForRegistry and TestTokenForRegistry cover first-wins, skipping each logged-out case, stopping on other errors and on cancellation, and the unchanged single-profile path. TestToken_loadToken now asserts that ErrNotFound and ErrMissingRefreshToken stay in the error chain.
• Acceptance: updated cmd/auth/docker/{configure,configure-errors,token,help}. Configure warns and succeeds when profiles share a workspace. Token picks the first logged-in profile (the two profiles have different cached tokens), skips a logged-out profile on the default force-refresh path, and lists every profile when none is logged in.
• go test ./cmd/auth/... ./libs/dockercredentials/... ./libs/auth/... ./libs/databrickscfg/... and go test ./acceptance -run TestAccept/cmd/auth pass. Lint, deadcode, and whitespace checks pass.

This pull request and its description were written by Isaac.

[github-actions]

Commit: 49a2dc9ce | Workflow run

Regression test report

Selected 4 changed tests (limit=20, 0 not selected)

4 tests: regression=3 unreleased=0 coverage=1 failing=0

test	branch	main (a572447)	latest (v1.26.0)
TestAccept/cmd/auth/docker/configure	✅	❌	❌
TestAccept/cmd/auth/docker/configure-errors	✅	✅	✅
TestAccept/cmd/auth/docker/help	✅	❌	❌
TestAccept/cmd/auth/docker/token	✅	❌	❌

TestAccept/cmd/auth/docker/configure: branch ✅ | main (a572447) ❌ | latest (v1.26.0) ❌

main (a572447):

acceptance_test.go:1373: Diff:
        --- cmd/auth/docker/configure/output.txt
        +++ /tmp/TestAcceptcmdauthdockerconfigure1684965037/002/output.txt
        @@ -158,24 +158,6 @@
         === Profiles sharing the workspace
         
         >>> databricks auth docker configure docker-test
        -Warning: multiple Databricks profiles match workspace ID [WORKSPACE_ID]: docker-test, shared. Docker credential helper uses the first logged-in profile in this order
        -Configured Docker credential helper for [WORKSPACE_ID].container.us-west-2.localhost
        -Updated Docker config: [TEST_TMP_DIR]/home/docker/config.json
        -Installed Docker credential helper: [TEST_TMP_DIR]/home/bin/docker-credential-databricks
        +Error: multiple Databricks profiles match workspace ID [WORKSPACE_ID]: docker-test and shared. Docker credential helper requires exactly one profile per workspace. Remove the duplicate profile with: databricks auth logout --profile <profile-name> --delete
         
        ->>> print_requests.py --get //api
        -{
        -  "headers": {
        -    "X-Databricks-Workspace-Id": [
        -      "[WORKSPACE_ID]"
        -    ]
        -  },
        -  "method": "GET",
        -  "path": "/api/2.1/unity-catalog/metastore_summary"
        -}
        -
        ->>> docker-credential-databricks get
        -{
        -  "Secret": "oauth-token",
        -  "Username": "oauthtoken"
        -}
        +Exit code: 1
        
    acceptance_test.go:1151: 
        LOG.configure-again: Configured Docker credential helper for 900800700600.container.us-west-2.localhost
        LOG.configure-again: Updated Docker config: /tmp/TestAcceptcmdauthdockerconfigure1684965037/001/home/docker/config.json
        LOG.configure-again: Installed Docker credential helper: /tmp/TestAcceptcmdauthdockerconfigure1684965037/001/home/bin/docker-credential-databricks
FAIL

latest (v1.26.0): same output as main (a572447)

TestAccept/cmd/auth/docker/help: branch ✅ | main (a572447) ❌ | latest (v1.26.0) ❌

main (a572447):

acceptance_test.go:1373: Diff:
        --- cmd/auth/docker/help/output.txt
        +++ /tmp/TestAcceptcmdauthdockerhelp3680274155/002/output.txt
        @@ -65,13 +65,11 @@
         This command installs docker-credential-databricks and configures Docker to use
         it for the selected workspace's Artifact Registry host. If the selected profile
         does not already include a workspace_id, the command resolves and saves it so
        -the Docker helper can map the registry host back to the profile. If several
        -profiles match the workspace, the helper uses the first logged-in one, checking
        -[DEFAULT] first and then the others in file order. The registry region is
        -inferred from the workspace's metastore. Select the workspace with [PROFILE] or
        ---profile; --host, --account-id, and --workspace-id are not supported. The
        -deprecated --region flag is retained for compatibility; omit it because it will
        -be fully removed in the next release.
        +the Docker helper can map the registry host back to the profile. The registry
        +region is inferred from the workspace's metastore. Select the workspace with
        +[PROFILE] or --profile; --host, --account-id, and --workspace-id are not
        +supported. The deprecated --region flag is retained for compatibility; omit it
        +because it will be fully removed in the next release.
         
         Usage:
           databricks auth docker configure [PROFILE] [flags]

latest (v1.26.0): same output as main (a572447)

TestAccept/cmd/auth/docker/token: branch ✅ | main (a572447) ❌ | latest (v1.26.0) ❌

main (a572447):

acceptance_test.go:1373: Diff:
        --- cmd/auth/docker/token/output.txt
        +++ /tmp/TestAcceptcmdauthdockertoken2516194160/002/output.txt
        @@ -54,22 +54,6 @@
         === Multiple profiles for the workspace use the first logged-in profile
         
         >>> [CLI] auth docker token --no-force-refresh
        -{
        -  "Secret": "docker-test-access-token",
        -  "Username": "oauthtoken"
        -}
        +Error: multiple Databricks profiles match workspace ID [WORKSPACE_ID]: docker-test and duplicate. Docker credential helper requires exactly one profile per workspace. Remove the duplicate profile with: databricks auth logout --profile <profile-name> --delete
         
        -=== Profiles that are not logged in are skipped
        -
        ->>> [CLI] auth docker token
        -{
        -  "Secret": "oauth-token",
        -  "Username": "oauthtoken"
        -}
        -
        -=== No logged-in profile for the workspace
        -
        ->>> musterr [CLI] auth docker token
        -Error: no logged-in Databricks profile for workspace ID 555666777 from registry host 555666777.container.us-west-2.localhost. Run databricks auth login --profile <profile-name> for one of these profiles:
        -profile "logged-out-a": cache: databricks OAuth is not configured for this host. no cached credentials; run `databricks auth login` to sign in
        -profile "logged-out-b": cache: databricks OAuth is not configured for this host. no cached credentials; run `databricks auth login` to sign in
        +Exit code: 1

latest (v1.26.0): same output as main (a572447)

[eng-dev-ecosystem-bot]

✅ Integration test report

Commit: 49a2dc9

Run: 43292409922

	Env	🔄​flaky	💚​RECOVERED	✅​pass	🙈​skip	Time
💚​	aws linux-2core-8gb		1	270	19	10:17
💚​	aws-windows-latest-4core-16gb		1	272	17	8:43
🔄​	azure linux-2core-8gb	1	1	268	19	9:03
🔄​	azure-windows-latest-4core-16gb	1	1	270	17	7:32
💚​	gcp linux-2core-8gb		1	270	19	11:31
🔄​	gcp-windows-latest-4core-16gb	1	1	271	17	7:28

	Test Name	aws linux-2core-8gb	aws-windows-latest-4core-16gb	azure linux-2core-8gb	azure-windows-latest-4core-16gb	gcp linux-2core-8gb	gcp-windows-latest-4core-16gb
💚​	TestAccept	💚​R	💚​R	💚​R	💚​R	💚​R	💚​R
🔄​	TestSyncIncrementalSyncPythonNotebookDelete	✅​p	✅​p	✅​p	✅​p	✅​p	🔄​f
🔄​	TestSyncIncrementalSyncPythonNotebookToFile	✅​p	✅​p	✅​p	🔄​f	✅​p	✅​p
🔄​	TestSyncNestedSpacePlusAndHashAreEscapedSync	✅​p	✅​p	🔄​f	✅​p	✅​p	✅​p

Top 13 slowest tests (at least 2 minutes):

duration	env	testname
3:58	gcp linux-2core-8gb	TestAccept
3:58	azure linux-2core-8gb	TestAccept
3:56	aws linux-2core-8gb	TestAccept
3:51	aws-windows-latest-4core-16gb	TestAccept
3:36	gcp-windows-latest-4core-16gb	TestAccept
3:33	azure-windows-latest-4core-16gb	TestAccept
2:58	gcp linux-2core-8gb	TestFilerWorkspaceFilesExtensionsReadDir
2:39	aws-windows-latest-4core-16gb	TestSyncIncrementalFileSync
2:22	azure-windows-latest-4core-16gb	TestFilerRecursiveDelete/workspace_files
2:21	gcp linux-2core-8gb	TestImportDir
2:17	gcp linux-2core-8gb	TestImportDirDoesNotOverwrite
2:10	aws linux-2core-8gb	TestFilerWorkspaceFilesExtensionsReadDir
2:04	aws-windows-latest-4core-16gb	TestFilerWorkspaceFilesExtensionsReadDir

[pietern]

This can have unintended side effects. Let's say you configure Docker assuming profile B, but profile A is earlier in the .databrickscfg for the same workspace host. What happens? Does a docker push use profile A?

I recommend using the "default profile" switcher (run databricks auth switch to see what happens) to be the tie-breaker, if it exists, and keep the error otherwise.

[yolocs]

This can have unintended side effects. Let's say you configure Docker assuming profile B, but profile A is earlier in the .databrickscfg for the same workspace host. What happens? Does a docker push use profile A?

I recommend using the "default profile" switcher (run databricks auth switch to see what happens) to be the tie-breaker, if it exists, and keep the error otherwise.

@pietern Yes, it will indeed have potential unintended effects. The problem originates from the credential helper protocol which can only use hostname (not profile) to pick the profile to get the auth token. And our user feedback suggested they'd prefer allowing profiles linking to the same workspace.

Your suggest would only work if one of such profiles is the default profile, right?