Skip to content

Commit e09a58e

Browse files
denikIsaac
andauthored
Ignore the cluster policy creator's CAN_USE in config-remote-sync (#7717)
## Changes `bundle config-remote-sync` no longer imports the current user's `CAN_USE` on a cluster policy that declares no permissions. The backend grants the creator `CAN_USE` on a new policy, so this entry is there without any remote edit. It is dropped like the deployer's own `IS_OWNER`/`CAN_MANAGE` on other resources. `withoutCurrentUserManage` is renamed to `withoutCurrentUserDefaults` and now takes the resource type. The testserver now gives the creator `CAN_USE` when a cluster policy is created, matching the cloud, so local runs cover this case. ## Why Since #7536, config-remote-sync reads the ACL of deployed resources that declare no permissions. On cloud, the three `config-remote-sync` cluster policy tests began reporting `Resource: resources.cluster_policies.policy permissions: add` and writing this into databricks.yml: ``` + permissions: + - level: CAN_USE + service_principal_name: [USERNAME] ``` After that, the next deploy created `cluster_policies.policy.permissions`. They fail on AWS, Azure and GCP, for both DMS variants. The testserver gave a new policy no ACL, which is why #7536 passed locally. Decisions: - The current user's `CAN_USE` on a cluster policy is always dropped, also when other principals are imported with it. Only workspace admins create or edit cluster policies, and admins can use every policy regardless of its ACL. So the grant isn't needed in the config, and writing it would apply one deployer's grant to other targets and deployers. As a result, a deploy after such a sync removes the deployer's direct `CAN_USE` entry, and they keep access as an admin. - Only cluster policies are affected. Other resource types keep the existing filter as is. - The current user is taken to be the creator, so the policy's `creator_user_name` is not read. ## Tests - Unit tests for the cluster policy filter, alone and next to other grants. The existing owner-filter test still passes. - New acceptance test `config-remote-sync/cluster_policy_permissions`: the first sync reports no changes. Then a group is granted `CAN_USE` remotely, and the sync imports only the group. The deploy that follows applies that ACL, and the plan is a no-op. - Before the fix, the testserver change makes the three tests fail locally the same way they fail on cloud. With the fix, the whole local acceptance suite passes. - Ran the three tests and the new one on AWS with the fix, both DMS variants: all pass. The commit message asks CI to run them on every cloud. ## Dashboard Failing since the 2026-11-17 nightly (the dashboard has not ingested that run yet; I will add the error links once it does): - [TestAccept/bundle/config-remote-sync/cluster_policy_cross_target](http://go/cli/dashboard/test/acceptance.TestAccept/bundle/config-remote-sync/cluster_policy_cross_target/DMS=true?main=1&pr=1&end=2026-11-18T00:00:00Z&last=2d) - [TestAccept/bundle/config-remote-sync/cluster_policy_remote_addition](http://go/cli/dashboard/test/acceptance.TestAccept/bundle/config-remote-sync/cluster_policy_remote_addition/DMS=true?main=1&pr=1&end=2026-11-18T00:00:00Z&last=2d) - [TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields](http://go/cli/dashboard/test/acceptance.TestAccept/bundle/config-remote-sync/policy_injected_cluster_fields/DMS=true?main=1&pr=1&end=2026-11-18T00:00:00Z&last=2d) ``` +Created cluster_policies.policy.permissions -Resources: 0 created, 1 changed, 0 deleted, 1 unchanged +Resources: 1 created, 1 changed, 0 deleted, 1 unchanged ``` This pull request and its description were written by Isaac. --------- Co-authored-by: Isaac <no-reply@databricks.com>
1 parent c15c8c6 commit e09a58e

8 files changed

Lines changed: 155 additions & 9 deletions

File tree

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
bundle:
2+
name: test-bundle-$UNIQUE_NAME
3+
4+
workspace:
5+
root_path: ~/.bundle/$UNIQUE_NAME
6+
7+
resources:
8+
cluster_policies:
9+
policy:
10+
name: test-policy-$UNIQUE_NAME
11+
definition:
12+
custom_tags.PolicyTag:
13+
type: fixed
14+
value: from-policy
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
Cloud = true
2+
GOOS.windows = false
3+
EnvMatrix.DMS = ["", "true"]
Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
2+
>>> [CLI] bundle deploy
3+
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/[UNIQUE_NAME]/files...
4+
Created cluster_policies.policy
5+
Files: 2 uploaded, 0 deleted
6+
Resources: 1 created, 0 changed, 0 deleted, 0 unchanged
7+
8+
=== The creator's CAN_USE is not a remote change
9+
10+
>>> [CLI] bundle config-remote-sync
11+
No changes detected.
12+
13+
14+
=== Grant a group CAN_USE in the UI
15+
16+
>>> [CLI] bundle config-remote-sync --save
17+
Detected changes in 1 resource(s):
18+
19+
Resource: resources.cluster_policies.policy
20+
permissions: add
21+
22+
23+
24+
>>> [CLI] bundle validate -o json
25+
[{"principal":"users","level":"CAN_USE"}]
26+
27+
=== Deploy applies the synced ACL
28+
29+
>>> [CLI] bundle deploy
30+
Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/[UNIQUE_NAME]/files...
31+
Created cluster_policies.policy.permissions
32+
Files: 1 uploaded, 0 deleted
33+
Resources: 1 created, 0 changed, 0 deleted, 1 unchanged
34+
35+
>>> [CLI] permissions get cluster-policies [POLICY_ID]
36+
[{"principal":"users","level":"CAN_USE"}]
37+
38+
>>> [CLI] bundle plan
39+
Plan: 0 to add, 0 to change, 0 to delete, 2 unchanged
40+
41+
>>> [CLI] bundle destroy --auto-approve
42+
The following resources will be deleted:
43+
delete resources.cluster_policies.policy
44+
45+
All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/[UNIQUE_NAME]
46+
47+
Destroy: 1 deleted
Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
envsubst < databricks.yml.tmpl > databricks.yml
2+
3+
cleanup() {
4+
trace $CLI bundle destroy --auto-approve
5+
}
6+
trap cleanup EXIT
7+
8+
trace $CLI bundle deploy
9+
policy_id=$(read_id.py policy)
10+
11+
title "The creator's CAN_USE is not a remote change"
12+
trace $CLI bundle config-remote-sync
13+
14+
title "Grant a group CAN_USE in the UI"
15+
# The UI keeps the existing direct entries, including the creator's.
16+
$CLI permissions get cluster-policies "$policy_id" | jq '{access_control_list: (
17+
[.access_control_list[] | select(.all_permissions[0].inherited | not)
18+
| {user_name, service_principal_name, group_name, permission_level: .all_permissions[0].permission_level}
19+
| with_entries(select(.value != null))]
20+
+ [{group_name: "users", permission_level: "CAN_USE"}])}' > acl.json
21+
$CLI permissions set cluster-policies "$policy_id" --json @acl.json > /dev/null
22+
rm acl.json
23+
24+
trace $CLI bundle config-remote-sync --save
25+
# The deployer is a user locally and a service principal on cloud, so print principals only.
26+
trace $CLI bundle validate -o json | jq -c '[.resources.cluster_policies.policy.permissions[] | {principal: (.user_name // .service_principal_name // .group_name), level}] | sort_by(.principal)'
27+
28+
title "Deploy applies the synced ACL"
29+
trace $CLI bundle deploy
30+
trace $CLI permissions get cluster-policies "$policy_id" | jq -c '[.access_control_list[] | select(.all_permissions[0].inherited | not) | {principal: (.user_name // .service_principal_name // .group_name), level: .all_permissions[0].permission_level}] | sort_by(.principal)'
31+
trace $CLI bundle plan
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
Cloud = true
2+
RecordRequests = false
3+
Ignore = [".databricks", "databricks.yml"]

‎bundle/configsync/diff_test.go‎

Lines changed: 20 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -371,7 +371,7 @@ resources:
371371
}, changes)
372372
}
373373

374-
func TestWithoutCurrentUserManage(t *testing.T) {
374+
func TestWithoutCurrentUserDefaults(t *testing.T) {
375375
permissions := []dresources.StatePermission{
376376
{Level: iam.PermissionLevelIsOwner, UserName: "me@example.com"},
377377
{Level: iam.PermissionLevelCanManage, ServicePrincipalName: "me@example.com"},
@@ -384,7 +384,25 @@ func TestWithoutCurrentUserManage(t *testing.T) {
384384
{Level: iam.PermissionLevelIsOwner, UserName: "owner@example.com"},
385385
{Level: iam.PermissionLevelCanManage, UserName: "other@example.com"},
386386
{Level: iam.PermissionLevelCanView, GroupName: "users"},
387-
}, withoutCurrentUserManage(permissions, "me@example.com"))
387+
}, withoutCurrentUserDefaults(permissions, "me@example.com", "jobs"))
388+
}
389+
390+
func TestWithoutCurrentUserDefaultsClusterPolicy(t *testing.T) {
391+
permissions := []dresources.StatePermission{
392+
{Level: iam.PermissionLevelCanUse, ServicePrincipalName: "me@example.com"},
393+
{Level: iam.PermissionLevelCanUse, UserName: "other@example.com"},
394+
{Level: iam.PermissionLevelCanUse, GroupName: "users"},
395+
}
396+
397+
assert.Equal(t, []dresources.StatePermission{
398+
{Level: iam.PermissionLevelCanUse, UserName: "other@example.com"},
399+
{Level: iam.PermissionLevelCanUse, GroupName: "users"},
400+
}, withoutCurrentUserDefaults(permissions, "me@example.com", "cluster_policies"))
401+
402+
creatorOnly := []dresources.StatePermission{
403+
{Level: iam.PermissionLevelCanUse, UserName: "me@example.com"},
404+
}
405+
assert.Empty(t, withoutCurrentUserDefaults(creatorOnly, "me@example.com", "cluster_policies"))
388406
}
389407

390408
func TestPermissionsReadSkip(t *testing.T) {

‎bundle/configsync/permissions.go‎

Lines changed: 18 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -144,7 +144,7 @@ func readUnmanagedPermissions(ctx context.Context, b *bundle.Bundle, plan *deplo
144144
skipped = append(skipped, permissionsReadSkip(resourceKey, err))
145145
return nil
146146
}
147-
permissions = withoutCurrentUserManage(permissions, currentUser)
147+
permissions = withoutCurrentUserDefaults(permissions, currentUser, config.GetResourceTypeFromKey(resourceKey))
148148
if len(permissions) > 0 {
149149
result[resourceKey] = permissions
150150
}
@@ -234,16 +234,27 @@ func lookupDeployedField(b *bundle.Bundle, resourceKey, field string) (any, erro
234234
return structaccess.Get(remote, path)
235235
}
236236

237-
// withoutCurrentUserManage drops the current user's IS_OWNER and CAN_MANAGE, one of which
238-
// FixPermissions adds on deploy. Another principal's IS_OWNER is kept, so a transferred owner
239-
// stays the owner after the next deploy.
240-
func withoutCurrentUserManage(permissions []dresources.StatePermission, currentUser string) []dresources.StatePermission {
237+
// withoutCurrentUserDefaults drops the current user's grants that exist without being declared.
238+
// For most resources these are IS_OWNER and CAN_MANAGE, one of which FixPermissions adds on
239+
// deploy; another principal's IS_OWNER is kept, so a transferred owner stays the owner after the
240+
// next deploy. For a cluster policy it is the CAN_USE the backend grants its creator: only admins
241+
// create policies and admins can use every policy, so the grant is never needed in the config.
242+
func withoutCurrentUserDefaults(permissions []dresources.StatePermission, currentUser, resourceType string) []dresources.StatePermission {
241243
return slices.DeleteFunc(permissions, func(p dresources.StatePermission) bool {
242-
isCurrentUser := currentUser != "" && (p.UserName == currentUser || p.ServicePrincipalName == currentUser)
243-
return isCurrentUser && (p.Level == iam.PermissionLevelIsOwner || p.Level == iam.PermissionLevelCanManage)
244+
if !isCurrentUserPermission(p, currentUser) {
245+
return false
246+
}
247+
if resourceType == "cluster_policies" {
248+
return p.Level == iam.PermissionLevelCanUse
249+
}
250+
return p.Level == iam.PermissionLevelIsOwner || p.Level == iam.PermissionLevelCanManage
244251
})
245252
}
246253

254+
func isCurrentUserPermission(p dresources.StatePermission, currentUser string) bool {
255+
return currentUser != "" && (p.UserName == currentUser || p.ServicePrincipalName == currentUser)
256+
}
257+
247258
func currentUserName(b *bundle.Bundle) string {
248259
if b.Config.Workspace.CurrentUser == nil {
249260
return ""

‎libs/testserver/cluster_policies.go‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ import (
99

1010
"github.com/databricks/cli/libs/clusterpolicy"
1111
"github.com/databricks/databricks-sdk-go/service/compute"
12+
"github.com/databricks/databricks-sdk-go/service/iam"
1213
"github.com/databricks/databricks-sdk-go/service/jobs"
1314
)
1415

@@ -50,6 +51,24 @@ func (s *FakeWorkspace) ClusterPoliciesCreate(req Request) any {
5051
policy.PolicyId = id
5152
s.ClusterPolicies[id] = policy
5253

54+
// The backend grants the creator CAN_USE on a new policy.
55+
creatorName := userForToken(req.Token).UserName
56+
creator := iam.AccessControlResponse{
57+
DisplayName: creatorName,
58+
AllPermissions: []iam.Permission{{PermissionLevel: "CAN_USE", ForceSendFields: []string{"Inherited"}}},
59+
}
60+
if creatorName == TestUserSP.UserName {
61+
creator.ServicePrincipalName = creatorName
62+
} else {
63+
creator.UserName = creatorName
64+
}
65+
objectID := "/cluster-policies/" + id
66+
s.Permissions[objectID] = iam.ObjectPermissions{
67+
ObjectId: objectID,
68+
ObjectType: "cluster-policy",
69+
AccessControlList: []iam.AccessControlResponse{creator},
70+
}
71+
5372
return Response{Body: compute.CreatePolicyResponse{PolicyId: id}}
5473
}
5574

0 commit comments

Comments
 (0)