Repository navigation
Conversation
Contributor
|
Commit: Regression test reportSelected 20 changed tests (limit=20, 21 not selected) 37 tests: regression=33 unreleased=0 coverage=4 failing=0
TestAccept/bundle/migrate/auto/schema-rename/DMS=: branch ✅ | main (77950b6) ❌ | latest (v1.25.0) ❌main (77950b6): latest (v1.25.0): same output as main (77950b6) TestAccept/bundle/resources/catalogs/rename/DMS=: branch ✅ | main (77950b6) ❌ | latest (v1.25.0) ❌main (77950b6): latest (v1.25.0): TestAccept/bundle/resources/catalogs/rename/DMS=true: branch ✅ | main (77950b6) ❌ | latest (v1.25.0) ❌main (77950b6): latest (v1.25.0): TestAccept/bundle/resources/catalogs/rename_all_out_of_band/DMS=: branch ✅ | main (77950b6) ❌ | latest (v1.25.0) ❌main (77950b6): latest (v1.25.0): 29 more failure details omitted, see the full report in the run. |
denik
force-pushed
the
denik/schemas-remove
branch
from
November 4, 2026 13:15
c643054 to
3aabe1b
Compare
Collaborator
✅ Integration test reportCommit: 37e8eb3
Top 4 slowest tests (at least 2 minutes):
|
denik
force-pushed
the
denik/schemas-remove
branch
4 times, most recently
from
November 4, 2026 15:00
5f6e047 to
0d42d75
Compare
denik
force-pushed
the
denik/schemas-remove
branch
2 times, most recently
from
November 4, 2026 16:43
3678e96 to
891fceb
Compare
denik
force-pushed
the
denik/schemas-remove
branch
from
November 4, 2026 16:50
891fceb to
fe7fb69
Compare
denik
marked this pull request as ready for review
November 5, 2026 12:59
denik
force-pushed
the
denik/schemas-remove
branch
from
November 5, 2026 13:09
a4a9475 to
6845544
Compare
andrewnester
approved these changes
Nov 5, 2026
denik
force-pushed
the
denik/schemas-remove
branch
from
November 5, 2026 13:57
4f7fd4e to
27ceb82
Compare
denik
changed the title
pietern
reviewed
Nov 5, 2026
| return fullName, nil, nil | ||
| } | ||
|
|
||
| response, err := r.client.Schemas.Update(ctx, makeUpdateSchema(fullName, newName, config)) |
Contributor
There was a problem hiding this comment.
IIRC including other fields during a rename caused issues.
Also; what happens if we rename to something that UC considers equivalent?
E.g. case change (via EqualFold)?
schema-rename now renames in place, so it no longer covers the schema recreate. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
…ename UC moves schemas and volumes with their renamed catalog or schema, so the re-key needs no update. Read the child instead to record its full name as UC spells it. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
…ename Co-authored-by: Isaac <no-reply@databricks.com>
Azure and GCP return their own volume storage_location and predictive optimization metastore, and UC may touch a schema's updated_at after create. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Like the other DoUpdateWithID implementations, take the new ID from config, so a child moved by its parent's rename needs no request at all. Co-authored-by: Isaac <no-reply@databricks.com>
…lied A deploy can stop after renaming a schema but before re-keying its volume. The next plan then saw no rename in progress, read the volume at its old path, got 404, and planned a create, which fails on UC with RESOURCE_ALREADY_EXISTS. An update_id now records the old values of the fields it changes (renamed_from) in the same state write as the new ID. A child whose parent is not being renamed in the plan follows that recorded rename instead, and is not planned as a create just because its read at the old ID found nothing. Co-authored-by: Isaac <no-reply@databricks.com>
…amed_from When the read at the recorded ID finds nothing, the plan reads at the ID the config gives the resource and takes what it finds only if its stable ID (schema_id, volume_id, now saved in the state entry and the DMS envelope) matches. This covers a deploy that stopped after renaming the parent and a rename done out of band, and never adopts a different object of the same name. It replaces renamed_from, which missed repeated renames and could not tell a rename from a recreate. The test server now deletes children with a forced catalog/schema delete and rejects a catalog create under a taken name, like UC. The migrate fixtures dropped the out-of-band catalog create that the scripts already run. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
…of stable_id The internal ID now lives in the volume and schema state (VolumeState, SchemaState), set from the API responses and copied by RemapState; the framework reads it back through a new optional InternalID method. This replaces the stable_id field on the state entry and in the DMS envelope. A new drop_changes lifecycle rule leaves the always-different volume_id and schema_id (config never sets them) out of the plan. Co-authored-by: Isaac <no-reply@databricks.com>
With volume_id and schema_id in the state, the field-overlap match picks VolumeInfo and SchemaInfo, as registered models already map to their Info type; volume_id is marked output-only from the spec. Co-authored-by: Isaac <no-reply@databricks.com>
Use ignore_local_changes (config never sets them) and ignore_remote_changes (generated from the spec for volume_id, hand-written for schema_id, which the spec does not mark output-only) instead of a new drop_changes rule. The skipped change keeps the recorded and remote IDs visible in the plan. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
…talog The captured requests already create the destination catalog out of band, so the scripts no longer create it again. This restores terraform-requests.json as captured instead of editing it. Co-authored-by: Isaac <no-reply@databricks.com>
The parent-rename rule now runs where provided ID fields are classified, so the change is planned as update_id the first time instead of being planned as a recreate and rewritten by a separate pass. The plan output is unchanged. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
…nal IDs there An update_id whose changes only follow a parent's rename (or a move the plan found) is marked state_only. For a resource with IDFromState, apply records it under the new ID without calling DoUpdateWithID; it is left out of the textual plan and the applied lines, and counted as unchanged. The internal ID is declared in config as internal_id_field. The framework records it from the remote state on every save, skips its changes in the plan and reads it in findMoved, replacing the InternalID method, the YAML ignore rules and the VolumeId/SchemaId assignments in the adapters. Co-authored-by: Isaac <no-reply@databricks.com>
A schema or catalog renamed with its comment changed sends both in one PATCH and both land. A case-only rename of a schema or volume is accepted by UC, which keeps the lowercase name, so the deploy succeeds and the next plan is a no-op. The test server now lowercases a renamed volume, like UC. Co-authored-by: Isaac <no-reply@databricks.com>
…hain Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
denik
force-pushed
the
denik/schemas-remove
branch
from
November 9, 2026 15:11
fc6bb67 to
c23c64f
Compare
…leReason Co-authored-by: Isaac <no-reply@databricks.com>
…ting on main) Co-authored-by: Isaac <no-reply@databricks.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Changing a schema's
namenow renames it in place (PATCH new_name) instead of deleting and recreating it, so the schema and its contents survive.UC moves the children of a renamed catalog or schema along with it. A volume in a renamed schema, or a schema in a renamed catalog (
schema_name/catalog_name: ${resources.<parent>.name}), used to be planned as a recreate. Its create then failed withRESOURCE_ALREADY_EXISTSon every deploy, because the recreate had already dropped it from state. The planner now re-keys such children (update_id, reasonparent_renamed) under their new parents. Catalog rename was already supported and had the same latent bug for child schemas.A child whose only change is following its parent is planned
state_only: the framework saves it under the ID the adapter's new optionalIDFromStategives it, with no request. Such entries are left out of the text plan and counted as unchanged. A child with its own changes too goes throughDoUpdateWithID, which addresses it at its new path.A resource can also be missing from its recorded ID with no rename in the plan: a deploy stopped after renaming the parent, or the resource or its parent was renamed out of band. When the read at the recorded ID finds nothing, the plan reads at the
IDFromStateID. It takes what it finds only if its internal ID matches the one in the state. The newinternal_id_fieldlifecycle setting names it (schema_id,volume_id); the framework records it from the remote state on every save, and the plan skips it (reasoninternal_id). A different object under that name is planned as a create, which UC rejects, so it is never adopted. The plan entry'sidfield (declared, but never set until now) records where the resource was found, and apply addresses it there. Catalogs have no internal ID and are not looked for yet. With the new state fields,apitypes.generated.ymlnow maps schemas and volumes toSchemaInfo/VolumeInfo, as registered models already map to their Info type.Stacked on #7408, which makes the test server move, delete and reject UC objects the way UC does. With #7345 and #7361 (merged), which re-key grants and permissions when their parent's ID changes, a renamed schema keeps those in sync too. Registered models and UC secrets follow parent renames in #7379 and #7381.
Tests
Cloud = true, passing locally and on AWS, GCP and Azure:schemas/rename,schemas/rename_with_volume,catalogs/rename(catalog → schema → volume).schemas/rename_then_change: after a parent rename, a volume update, rename and recreate all reach its new path, also after a second rename and a rename back.volumes/rename_catalog_and_schema: the catalog and the schema are renamed in one deploy, so the volume follows two parent renames at once. It gets no request and keeps itsvolume_id.schemas/rename_with_volume_ref: a job reading the volume'sfull_name,idandvolume_idgets the right values after the rename.schemas/rename_with_comment,catalogs/rename_with_comment(rename and comment change go in one PATCH) andschemas/rename_case_only,volumes/rename_case_only(UC accepts the update and keeps the lowercase name; the redeploy is a no-op).schemas/rename_interrupted(local only; uses the test server's kill hook): the CLI is killed after the schema's rename is saved and before the volume's re-key. The next deploy finds the volume at its new path byvolume_id. Without the fix it planned a create.schemas/recreate_interrupted(local only): the same kill during a recreate of the schema under a new name. The volume went with the old schema, so the next deploy creates a new one.Cloud = true, config renamed to match:volumes/rename_out_of_bandandschemas/rename_out_of_band(found byvolume_id/schema_id, deploy sends no requests);catalogs/rename_out_of_bandandcatalogs/rename_all_out_of_band(schema and volume found; the catalog's create fails withCATALOG_ALREADY_EXISTS).Cloud = true:volumes/recreate_out_of_bandandschemas/recreate_out_of_band. An object is found under the new name but its id differs, so the plan creates instead of adopting it.grants/schemas/change_name(from Re-key grants when their parent's ID changes #7345) now shows the in-place rename;migrate/auto/schema-renamenow renames;lifecycle/prevent-destroyswitched its recreate lever from name tocatalog_name.followsParentRename,Changes.HasOnlySingleReasonand the skipped internal-ID changes.This pull request and its description were written by Isaac.