Skip to content

Include variable locations in --include-locations output - #7351

Open
petrapostelnicu-db wants to merge 3 commits into
mainfrom
variable-locations
Open

petrapostelnicu-db wants to merge 3 commits into
mainfrom
variable-locations

Conversation

@petrapostelnicu-db

@petrapostelnicu-db petrapostelnicu-db commented Nov 4, 2026 •

Copy link
Copy Markdown

The location map that bundle validate and bundle summary emit with --include-locations now covers variables:

  • variables.: where the variable is declared.
  • variables..value: where the resolved value comes from, such as a target override or .databricks/bundle//variable-overrides.json. It is absent when the value has no file, as with --var, BUNDLE_VAR_*, and lookups.

The workspace UI uses this to link each variable to its file in the deployments modal.

The location map that `bundle validate` and `bundle summary` emit with
--include-locations now covers variables:

- variables.<name>: where the variable is declared.
- variables.<name>.value: where the resolved value comes from, such as a
  target override or .databricks/bundle/<target>/variable-overrides.json.
  It is absent when the value has no file, as with --var, BUNDLE_VAR_*,
  and lookups.

The workspace UI uses this to link each variable to its file.

Co-authored-by: Isaac <no-reply@databricks.com>
@petrapostelnicu-db
petrapostelnicu-db requested review from a team as code owners November 4, 2026 15:37
@github-actions github-actions Bot added the DABs DABs related issues label Nov 4, 2026
@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Commit: f969f68e6 | Workflow run

Regression test report

Selected 1 changed tests (limit=20, 0 not selected)

2 tests: regression=2 unreleased=0 coverage=0 failing=0
test branch main (663e1f7) latest (v1.24.0)
TestAccept/bundle/variables/locations/DMS= ✅ ❌ ❌
TestAccept/bundle/variables/locations/DMS=true ✅ ❌ ❌
TestAccept/bundle/variables/locations/DMS=: branch ✅ | main (663e1f7) ❌ | latest (v1.24.0) ❌

main (663e1f7):

acceptance_test.go:1345: Diff:
        --- bundle/variables/locations/output.txt
        +++ /tmp/TestAcceptbundlevariableslocationsDMS=2855894031/002/output.txt
        @@ -24,72 +24,5 @@
           "variables": [
             "databricks.yml:9:3",
             "variables/shared.yml:2:3"
        -  ],
        -  "variables.complex_var": [
        -    "databricks.yml:18:5"
        -  ],
        -  "variables.complex_var.value": [
        -    "databricks.yml:43:9"
        -  ],
        -  "variables.dash-name": [
        -    "databricks.yml:30:5"
        -  ],
        -  "variables.dash-name.value": [
        -    "databricks.yml:30:14"
        -  ],
        -  "variables.from_cli": [
        -    "databricks.yml:12:5"
        -  ],
        -  "variables.from_env": [
        -    "databricks.yml:14:5"
        -  ],
        -  "variables.from_file": [
        -    "databricks.yml:16:5"
        -  ],
        -  "variables.from_file.value": [
        -    ".databricks/bundle/dev/variable-overrides.json:2:28"
        -  ],
        -  "variables.included": [
        -    "variables/shared.yml:3:5"
        -  ],
        -  "variables.included.value": [
        -    "targets.yml:4:17"
        -  ],
        -  "variables.lookup_overridden": [
        -    "databricks.yml:32:5"
        -  ],
        -  "variables.lookup_overridden.value": [
        -    "databricks.yml:44:26"
        -  ],
        -  "variables.override_full": [
        -    "databricks.yml:26:5"
        -  ],
        -  "variables.override_full.value": [
        -    "databricks.yml:41:18"
        -  ],
        -  "variables.override_short": [
        -    "databricks.yml:24:5"
        -  ],
        -  "variables.override_short.value": [
        -    "databricks.yml:39:23"
        -  ],
        -  "variables.simple": [
        -    "databricks.yml:10:5"
        -  ],
        -  "variables.simple.value": [
        -    "databricks.yml:10:14"
        -  ],
        -  "variables.split": [
        -    "databricks.yml:28:5",
        -    "variables/shared.yml:5:5"
        -  ],
        -  "variables.split.value": [
        -    "variables/shared.yml:5:14"
        -  ],
        -  "variables.var_in_var": [
        -    "databricks.yml:22:5"
        -  ],
        -  "variables.var_in_var.value": [
        -    "databricks.yml:22:14"
           ]
         }
        
FAIL

latest (v1.24.0): same output as main (663e1f7)

TestAccept/bundle/variables/locations/DMS=true: branch ✅ | main (663e1f7) ❌ | latest (v1.24.0) ❌

main (663e1f7):

acceptance_test.go:1345: Diff:
        --- bundle/variables/locations/output.txt
        +++ /tmp/TestAcceptbundlevariableslocationsDMS=true3555543991/002/output.txt
        @@ -24,72 +24,5 @@
           "variables": [
             "databricks.yml:9:3",
             "variables/shared.yml:2:3"
        -  ],
        -  "variables.complex_var": [
        -    "databricks.yml:18:5"
        -  ],
        -  "variables.complex_var.value": [
        -    "databricks.yml:43:9"
        -  ],
        -  "variables.dash-name": [
        -    "databricks.yml:30:5"
        -  ],
        -  "variables.dash-name.value": [
        -    "databricks.yml:30:14"
        -  ],
        -  "variables.from_cli": [
        -    "databricks.yml:12:5"
        -  ],
        -  "variables.from_env": [
        -    "databricks.yml:14:5"
        -  ],
        -  "variables.from_file": [
        -    "databricks.yml:16:5"
        -  ],
        -  "variables.from_file.value": [
        -    ".databricks/bundle/dev/variable-overrides.json:2:28"
        -  ],
        -  "variables.included": [
        -    "variables/shared.yml:3:5"
        -  ],
        -  "variables.included.value": [
        -    "targets.yml:4:17"
        -  ],
        -  "variables.lookup_overridden": [
        -    "databricks.yml:32:5"
        -  ],
        -  "variables.lookup_overridden.value": [
        -    "databricks.yml:44:26"
        -  ],
        -  "variables.override_full": [
        -    "databricks.yml:26:5"
        -  ],
        -  "variables.override_full.value": [
        -    "databricks.yml:41:18"
        -  ],
        -  "variables.override_short": [
        -    "databricks.yml:24:5"
        -  ],
        -  "variables.override_short.value": [
        -    "databricks.yml:39:23"
        -  ],
        -  "variables.simple": [
        -    "databricks.yml:10:5"
        -  ],
        -  "variables.simple.value": [
        -    "databricks.yml:10:14"
        -  ],
        -  "variables.split": [
        -    "databricks.yml:28:5",
        -    "variables/shared.yml:5:5"
        -  ],
        -  "variables.split.value": [
        -    "variables/shared.yml:5:14"
        -  ],
        -  "variables.var_in_var": [
        -    "databricks.yml:22:5"
        -  ],
        -  "variables.var_in_var.value": [
        -    "databricks.yml:22:14"
           ]
         }

latest (v1.24.0): same output as main (663e1f7)

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: f969f68

Run: 42218836155

Env ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 271 16 5:19
✅​ aws-windows-latest-4core-16gb 273 14 3:27
✅​ azure linux-2core-8gb 270 16 4:22
✅​ azure-windows-latest-4core-16gb 272 14 3:40
✅​ gcp linux-2core-8gb 271 16 4:43
✅​ gcp-windows-latest-4core-16gb 273 14 3:10
Top 3 slowest tests (at least 2 minutes):
duration env testname
3:11 aws-windows-latest-4core-16gb TestAccept
3:05 gcp-windows-latest-4core-16gb TestAccept
3:03 azure-windows-latest-4core-16gb TestAccept

dyn.NewPattern(dyn.Key("resources"), dyn.Key("jobs"), dyn.AnyKey(), dyn.Key("tasks")), // Job tasks ("resources.jobs.my_job.tasks")
dyn.NewPattern(dyn.Key("resources"), dyn.Key("jobs"), dyn.AnyKey(), dyn.Key("tasks"), dyn.AnyIndex()), // Job task items ("resources.jobs.my_job.tasks[2]")
dyn.NewPattern(dyn.Key("variables"), dyn.AnyKey()), // Variable definitions ("variables.my_var")
dyn.NewPattern(dyn.Key("variables"), dyn.AnyKey(), dyn.Key("value")), // Resolved variable values ("variables.my_var.value")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this get a .nextchanges/bundles/ entry? The last --include-locations change (#7185) has one in the changelog, and the UI side will want to know which CLI version ships the variable keys.

@@ -0,0 +1,10 @@
[[Server]]
Pattern = "GET /api/2.0/preview/scim/v2/ServicePrincipals"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the same stub as in variables/lookup/test.toml, and issue_3039_lookup_with_ref carries a third variant. Maybe time to add a ServicePrincipals handler to libs/testserver instead, like .agents/rules/testing.md asks?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you're right. i removed the variable and the stub instead, I think adding service principals handler should be a separate pr. in my case, it wasn't needed, the variable with no value location test case is already covered by from_cli

"databricks.yml:18:5"
],
"variables.complex_var.value": [
"databricks.yml:46:9"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it expected that the complex value points at the first key inside the override (46:9) rather than the override itself (complex_var: at 45:7)? Probably fine if the UI only needs a line to link to, just checking it's intentional.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, it is intentional (kept consistent with resources ) and the ui only needs to link the file

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DABs DABs related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants