Reject unprefixed volume schema references during bundle validation by lennartkats-db · Pull Request #7285 · databricks/cli

lennartkats-db · 2026-11-01T21:31:11Z

Reject unprefixed schema references in volume fields during bundle validation.

Problem

Fixes Issue #3566, which showed ${schemas.test.name} and ${schemas.test.catalog_name} in volume schema_name and catalog_name and reports: "Validation is successful, only failing on deploy." The base fixture at 3aa5c9607bff73d3fb18a4c6e63b9f03235f3de7 also exited successfully while leaving both references literal.

Change

Reject those missing-prefix references during bundle validate and suggest the matching ${resources.schemas.*} path.
Identify the volume and field in each diagnostic, with the configuration path and YAML location when available.
Add a CLI acceptance case for both invalid fields. The existing valid prefixed-reference case remains covered.

Verification

go test -p 4 ./bundle/config/mutator -count=1 passed locally at 95a9959d951318eb1cfb6d64b879f790d76b8ce1. The regression test covers repeated references across two volumes, both fields, source locations, and valid prefixed references.
go test -p 4 ./acceptance -run '^TestAccept$/bundle/variables/reference-typo' -count=1 -v passed locally in both DMS variants. The new fixture captures two bundle validate errors with field paths, YAML locations, and suggestions.
go test -p 4 ./acceptance -run '^TestAccept$/bundle/resource_deps/resources_var_presets_implicit_deps' -count=1 -v passed locally in both DMS variants. That existing fixture validates correctly prefixed schema references in volume fields.

The acceptance tests ran locally against the test server. A live workspace deployment was not exercised.

eng-dev-ecosystem-bot · 2026-11-01T21:57:12Z

Integration test report

Commit: 95a9959

Run: 41837081214

	Env	✅pass	🙈skip	Time
✅	aws linux-2core-8gb	271	16	6:17
✅	aws-windows-latest-4core-16gb	273	14	4:08
✅	azure linux-2core-8gb	270	16	4:54
✅	azure-windows-latest-4core-16gb	272	14	4:29
✅	gcp linux-2core-8gb	271	16	5:58
✅	gcp-windows-latest-4core-16gb	273	14	4:17

Top 3 slowest tests (at least 2 minutes):

duration	env	testname
3:45	gcp-windows-latest-4core-16gb	TestAccept
3:23	aws-windows-latest-4core-16gb	TestAccept
3:18	azure-windows-latest-4core-16gb	TestAccept

github-actions · 2026-11-04T13:25:13Z

Commit: 95a9959d9 | Workflow run

Regression test report

Selected 1 changed tests (limit=20, 0 not selected)

2 tests: regression=2 unreleased=0 coverage=0 failing=0

test	branch	main (`cd5c64d`)	latest (v1.23.0)
TestAccept/bundle/variables/reference-typo/DMS=	✅	❌	❌
TestAccept/bundle/variables/reference-typo/DMS=true	✅	❌	❌

TestAccept/bundle/variables/reference-typo/DMS=: branch ✅ | main (cd5c64d) ❌ | latest (v1.23.0) ❌

main (cd5c64d):

acceptance_test.go:1342: Diff:
        --- bundle/variables/reference-typo/output.txt
        +++ /tmp/TestAcceptbundlevariablesreference-typoDMS=1763537916/002/output.txt
        @@ -101,24 +101,14 @@
         === Volume schema and catalog references missing resources
         
         >>> [CLI] bundle validate
        -Error: volume "ingest" field "schema_name": reference does not exist: ${schemas.test.name}
        -  at resources.volumes.ingest.schema_name
        -  in databricks.yml:12:20
        -
        -did you mean:
        -  ${resources.schemas.test.name}
        -
        -Error: volume "ingest" field "catalog_name": reference does not exist: ${schemas.test.catalog_name}
        -  at resources.volumes.ingest.catalog_name
        -  in databricks.yml:13:21
        -
        -did you mean:
        -  ${resources.schemas.test.catalog_name}
        -
         Name: reference-typo-volume-missing-resources
         Target: default
         Workspace:
           User: [USERNAME]
           Path: /Workspace/Users/[USERNAME]/.bundle/reference-typo-volume-missing-resources/default
         
        -Found 2 errors
        +Validation OK!
        +
        +Unexpected success
        +
        +Exit code: 1
        
    acceptance_test.go:1132: Test produced unexpected files:
        volume-missing-resources/.databricks/.gitignore

latest (v1.23.0):

acceptance_test.go:1342: Diff:
        --- bundle/variables/reference-typo/output.txt
        +++ /tmp/TestAcceptbundlevariablesreference-typoDMS=4217705201/002/output.txt
        @@ -101,24 +101,14 @@
         === Volume schema and catalog references missing resources
         
         >>> [CLI] bundle validate
        -Error: volume "ingest" field "schema_name": reference does not exist: ${schemas.test.name}
        -  at resources.volumes.ingest.schema_name
        -  in databricks.yml:12:20
        -
        -did you mean:
        -  ${resources.schemas.test.name}
        -
        -Error: volume "ingest" field "catalog_name": reference does not exist: ${schemas.test.catalog_name}
        -  at resources.volumes.ingest.catalog_name
        -  in databricks.yml:13:21
        -
        -did you mean:
        -  ${resources.schemas.test.catalog_name}
        -
         Name: reference-typo-volume-missing-resources
         Target: default
         Workspace:
           User: [USERNAME]
           Path: /Workspace/Users/[USERNAME]/.bundle/reference-typo-volume-missing-resources/default
         
        -Found 2 errors
        +Validation OK!
        +
        +Unexpected success
        +
        +Exit code: 1
        
    acceptance_test.go:1132: Test produced unexpected files:
        volume-missing-resources/.databricks/.gitignore
FAIL

TestAccept/bundle/variables/reference-typo/DMS=true: branch ✅ | main (cd5c64d) ❌ | latest (v1.23.0) ❌

main (cd5c64d):

acceptance_test.go:1342: Diff:
        --- bundle/variables/reference-typo/output.txt
        +++ /tmp/TestAcceptbundlevariablesreference-typoDMS=true3537407100/002/output.txt
        @@ -101,24 +101,14 @@
         === Volume schema and catalog references missing resources
         
         >>> [CLI] bundle validate
        -Error: volume "ingest" field "schema_name": reference does not exist: ${schemas.test.name}
        -  at resources.volumes.ingest.schema_name
        -  in databricks.yml:12:20
        -
        -did you mean:
        -  ${resources.schemas.test.name}
        -
        -Error: volume "ingest" field "catalog_name": reference does not exist: ${schemas.test.catalog_name}
        -  at resources.volumes.ingest.catalog_name
        -  in databricks.yml:13:21
        -
        -did you mean:
        -  ${resources.schemas.test.catalog_name}
        -
         Name: reference-typo-volume-missing-resources
         Target: default
         Workspace:
           User: [USERNAME]
           Path: /Workspace/Users/[USERNAME]/.bundle/reference-typo-volume-missing-resources/default
         
        -Found 2 errors
        +Validation OK!
        +
        +Unexpected success
        +
        +Exit code: 1
        
    acceptance_test.go:1132: Test produced unexpected files:
        volume-missing-resources/.databricks/.gitignore
FAIL

latest (v1.23.0):

acceptance_test.go:1342: Diff:
        --- bundle/variables/reference-typo/output.txt
        +++ /tmp/TestAcceptbundlevariablesreference-typoDMS=true74235680/002/output.txt
        @@ -101,24 +101,14 @@
         === Volume schema and catalog references missing resources
         
         >>> [CLI] bundle validate
        -Error: volume "ingest" field "schema_name": reference does not exist: ${schemas.test.name}
        -  at resources.volumes.ingest.schema_name
        -  in databricks.yml:12:20
        -
        -did you mean:
        -  ${resources.schemas.test.name}
        -
        -Error: volume "ingest" field "catalog_name": reference does not exist: ${schemas.test.catalog_name}
        -  at resources.volumes.ingest.catalog_name
        -  in databricks.yml:13:21
        -
        -did you mean:
        -  ${resources.schemas.test.catalog_name}
        -
         Name: reference-typo-volume-missing-resources
         Target: default
         Workspace:
           User: [USERNAME]
           Path: /Workspace/Users/[USERNAME]/.bundle/reference-typo-volume-missing-resources/default
         
        -Found 2 errors
        +Validation OK!
        +
        +Unexpected success
        +
        +Exit code: 1
        
    acceptance_test.go:1132: Test produced unexpected files:
        volume-missing-resources/.databricks/.gitignore

[factory] dabs UI candidate 4001dd47a437

600c78c

lennartkats-db requested review from a team as code owners November 1, 2026 21:31

lennartkats-db requested review from rclarey and shreyas-goenka November 1, 2026 21:31

lennartkats-db added DABs DABs related issues software-factory PR produced by the software factories task labels Nov 1, 2026

lennartkats-db changed the title ~~[draft] Fix bundle/config/mutator/resolve_variable_references.go~~ Fix bundle/config/mutator/resolve_variable_references.go Nov 1, 2026

lennartkats-db assigned shreyas-goenka Nov 1, 2026

[factory] Repair attached DABs CI

b4db510

shreyas-goenka approved these changes Nov 2, 2026

View reviewed changes

Comment thread bundle/config/mutator/resolve_variable_references_test.go Resolved

lennartkats-db removed the software-factory PR produced by the software factories task label Nov 3, 2026

[factory] Address DABs PR feedback

abb2431

lennartkats-db added the software-factory PR produced by the software factories task label Nov 3, 2026

Report volume and field for invalid schema references

95a9959

lennartkats-db changed the title ~~Fix bundle/config/mutator/resolve_variable_references.go~~ Reject unprefixed volume schema references during bundle validation Nov 4, 2026

lennartkats-db changed the title ~~Reject unprefixed volume schema references during bundle validation~~ [draft] Reject unprefixed volume schema references during bundle validation Nov 4, 2026

lennartkats-db changed the title ~~[draft] Reject unprefixed volume schema references during bundle validation~~ Reject unprefixed volume schema references during bundle validation Nov 6, 2026

lennartkats-db added this pull request to the merge queue Nov 6, 2026

Merged via the queue into main with commit dd0811b Nov 6, 2026
27 checks passed

lennartkats-db deleted the factory/dabs/4001dd47a437 branch November 6, 2026 16:28

Navigation Menu

Repository navigation

Reject unprefixed volume schema references during bundle validation - #7285

Reject unprefixed volume schema references during bundle validation#7285
lennartkats-db merged 4 commits into
mainfrom
factory/dabs/4001dd47a437

lennartkats-db commented Nov 1, 2026 •

edited

Loading

Uh oh!

eng-dev-ecosystem-bot commented Nov 1, 2026 •

edited

Loading

Uh oh!

Uh oh!

github-actions Bot commented Nov 4, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

lennartkats-db commented Nov 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Change

Verification

Uh oh!

eng-dev-ecosystem-bot commented Nov 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Integration test report

Uh oh!

Uh oh!

github-actions Bot commented Nov 4, 2026

Regression test report

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

lennartkats-db commented Nov 1, 2026 •

edited

Loading

eng-dev-ecosystem-bot commented Nov 1, 2026 •

edited

Loading