Skip to content

Support segment and recursive wildcards in fault patterns - #7241

Merged
shreyas-goenka merged 2 commits into
mainfrom
shreyas-goenka_data/shreyas/test-fault-glob-main
Oct 28, 2026
Merged

Support segment and recursive wildcards in fault patterns#7241
shreyas-goenka merged 2 commits into
mainfrom
shreyas-goenka_data/shreyas/test-fault-glob-main

Conversation

@shreyas-goenka

@shreyas-goenka shreyas-goenka commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Changes

Support * within a path segment and ** across path segments in fault.py patterns. Both match zero or more characters; all other characters are literal. Compile each rule once using short, commented steps.

Update the recursive DMS operation fault to use ** and target the dashboard publish fault with */published. Unit tests cover segment boundaries, recursive matching, empty matches, endpoint suffixes, and literal regex characters.

Why

Allow precise rules such as POST .../versions/*/complete without enumerating version numbers. A rule ending in versions/* must not also match nested operation endpoints.

This PR targets main and provides the matcher for #7231. It has no dependency on the generator-fix branch. The cloud recording proxy is unchanged.

Tests

  • Full libs/testserver unit suite.
  • Existing DMS operation and dashboard publishing fault acceptance cases using a CLI built from main.
  • Formatting, lint, and whitespace checks.

Validation uses a temporary modfile with approved x/net v0.61.0 because the dependency proxy blocks v0.62.0. Repository dependencies are unchanged.

This PR was written by Codex.

@github-actions github-actions Bot added the DABs DABs related issues label Oct 28, 2026
@shreyas-goenka shreyas-goenka mentioned this pull request Oct 28, 2026
Open
@github-actions

github-actions Bot commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Commit: 2afd98495 | Workflow run

Regression test report

Selected 2 changed tests (limit=20, 0 not selected)

3 tests: regression=0 unreleased=1 coverage=2 failing=0

test branch main (8b4a341) latest (v1.22.1)
TestAccept/bundle/dms/operation-upload-fails-once/DMS=true ✅ ✅ ❌
TestAccept/bundle/resources/dashboards/publish-failure-stale-content/DMS= ✅ ✅ ✅
TestAccept/bundle/resources/dashboards/publish-failure-stale-content/DMS=true ✅ ✅ ✅
TestAccept/bundle/dms/operation-upload-fails-once/DMS=true: branch ✅ | main (8b4a341) ✅ | latest (v1.22.1) ❌

latest (v1.22.1):

acceptance_test.go:1341: Diff:
        --- bundle/dms/operation-upload-fails-once/output.txt
        +++ /tmp/TestAcceptbundledmsoperation-upload-fails-onceDMS=true4135539997/002/output.txt
        @@ -14,7 +14,7 @@
           "lineage": "[UUID]",
           "serial": 1,
           "features": {
        -    "deployment_history_shadow": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -39,18 +39,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_shadow": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.schemas.foo": {
        -   "__id__": "main.dms_upload_fails_once_schema_[UNIQUE_NAME]",
        -   "state": {
        -    "catalog_name": "main",
        -    "comment": "v1",
        -    "name": "dms_upload_fails_once_schema_[UNIQUE_NAME]"
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === A recreate records twice, and the first one fails: the second must still carry the sequence id staging left, because a rejected update earned no new one
        @@ -64,113 +54,13 @@
         This action will result in the deletion or recreation of the following UC schemas. Any underlying data may be lost:
           recreate resources.schemas.foo
         Recreated schemas.foo
        -Warn: Failed to record deployment history: recording operation for resources.schemas.foo: Fault injected by test.
        -Files: 2 uploaded, 0 deleted
        -Resources: 1 created, 0 changed, 1 deleted, 0 unchanged
        +Error: recording operation for resources.schemas.foo: Fault injected by test. (500 INJECTED)
         
        -=== The service lists the schema with the failure
        ->>> MSYS_NO_PATHCONV=1 [CLI] bundle-deployments list-resources deployments/[DEPLOYMENT_ID] -o json
        -[
        -  {
        -    "last_action_type": "OPERATION_ACTION_TYPE_RECREATE",
        -    "last_version_id": "2",
        -    "resource_key": "schemas.foo"
        -  }
        -]
        +Endpoint: PATCH [DATABRICKS_URL]/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/2/operations/schemas.foo?update_mask=state%2Cerror_message%2Cstatus
        +HTTP Status: 500 Internal Server Error
        +API error_code: INJECTED
        +API message: Fault injected by test.
         
        -=== The raw recorded state after the failure
        ->>> print_state.py --no-dms
        -{
        - "state_version": 3,
        - "cli_version": "[CLI_VERSION]",
        - "lineage": "[UUID]",
        - "serial": 2,
        - "features": {
        -  "deployment_history_shadow": {}
        - },
        - "state": {
        -  "resources.schemas.foo": {
        -   "__id__": "dms_other_[UNIQUE_NAME].dms_upload_fails_once_schema_[UNIQUE_NAME]",
        -   "state": {
        -    "catalog_name": "dms_other_[UNIQUE_NAME]",
        -    "comment": "v1",
        -    "name": "dms_upload_fails_once_schema_[UNIQUE_NAME]"
        -   }
        -  }
        - }
        -}
        +Files: 2 uploaded, 0 deleted
         
        ->>> print_requests.py --dms //api/2.0/bundle
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions",
        -  "q": {
        -    "version_id": "2"
        -  },
        -  "body": {
        -    "cli_version": "[CLI_VERSION]",
        -    "operations": [
        -      {
        -        "action_type": "OPERATION_ACTION_TYPE_RECREATE",
        -        "resource_key": "schemas.foo"
        -      }
        -    ],
        -    "previous_version_id": "1",
        -    "version_type": "VERSION_TYPE_DEPLOY"
        -  }
        -}
        -{
        -  "method": "PATCH",
        -  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/2/operations/schemas.foo",
        -  "q": {
        -    "update_mask": "state,error_message,status"
        -  },
        -  "body": {
        -    "error_message": "",
        -    "sequence_id": 0,
        -    "status": "OPERATION_STATUS_PENDING"
        -  }
        -}
        -{
        -  "method": "PATCH",
        -  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/2/operations/schemas.foo",
        -  "q": {
        -    "update_mask": "state,error_message,resource_id,status"
        -  },
        -  "body": {
        -    "error_message": "",
        -    "resource_id": "dms_other_[UNIQUE_NAME].dms_upload_fails_once_schema_[UNIQUE_NAME]",
        -    "sequence_id": 0,
        -    "state": "{\"state\":{\"catalog_name\":\"dms_other_[UNIQUE_NAME]\",\"comment\":\"v1\",\"name\":\"dms_upload_fails_once_schema_[UNIQUE_NAME]\"}}",
        -    "status": "OPERATION_STATUS_SUCCEEDED"
        -  }
        -}
        -{
        -  "method": "POST",
        -  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/2/complete",
        -  "body": {
        -    "completion_reason": "VERSION_COMPLETE_FAILURE"
        -  }
        -}
        -
        -=== A completion failure does not block destroy or remote file cleanup
        ->>> fault.py POST /api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/3/complete 400 0 1
        -
        ->>> [CLI] bundle destroy --auto-approve
        -The following resources will be deleted:
        -  delete resources.schemas.foo
        -
        -This action will result in the deletion of the following UC schemas. Any underlying data may be lost:
        -  delete resources.schemas.foo
        -
        -All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/dms-upload-fails-once-[UNIQUE_NAME]/default
        -
        -Warn: Failed to complete deployment history version: Fault injected by test.
        -Destroy: 1 deleted
        -
        ->>> print_requests.py --dms //versions/3 --oneline
        -{"method": "PATCH", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/3/operations/schemas.foo", "q": {"update_mask": "state,error_message,status"}, "body": {"error_message": "", "sequence_id": 0, "status": "OPERATION_STATUS_SUCCEEDED"}}
        -{"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/3/complete", "body": {"completion_reason": "VERSION_COMPLETE_SUCCESS"}}
        -
        ->>> musterr env MSYS_NO_PATHCONV=1 [CLI] workspace get-status /Workspace/Users/[USERNAME]/.bundle/dms-upload-fails-once-[UNIQUE_NAME]/default/state/resources.json
        -Error: Path (/Workspace/Users/[USERNAME]/.bundle/dms-upload-fails-once-[UNIQUE_NAME]/default/state/resources.json) doesn't exist.
        +Exit code: 1
        
    acceptance_test.go:1119: 
        LOG.catalog: {
        LOG.catalog:   "catalog_type": "MANAGED_CATALOG",
        LOG.catalog:   "created_at": 1793206941551,
        LOG.catalog:   "created_by": "tester@databricks.com",
        LOG.catalog:   "full_name": "dms_other_k4cwih2lp5d6nnfmm5dtzcjm5u",
        LOG.catalog:   "metastore_id": "7a07bf1f-fbb2-faca-7a07-bf1ffbb2fbb0",
        LOG.catalog:   "name": "dms_other_k4cwih2lp5d6nnfmm5dtzcjm5u",
        LOG.catalog:   "owner": "tester@databricks.com",
        LOG.catalog:   "updated_at": 1793206941551,
        LOG.catalog:   "updated_by": "tester@databricks.com"
        LOG.catalog: }

@shreyas-goenka
shreyas-goenka marked this pull request as ready for review October 28, 2026 17:06
@shreyas-goenka
shreyas-goenka requested review from a team as code owners October 28, 2026 17:06
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: 2afd984

Run: 40730791311

Env ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 281 17 5:16
✅​ aws-windows-latest-4core-16gb 283 15 3:28
✅​ azure linux-2core-8gb 280 17 5:23
✅​ azure-windows-latest-4core-16gb 282 15 4:09
✅​ gcp linux-2core-8gb 281 17 5:33
✅​ gcp-windows-latest-4core-16gb 283 15 3:55
Top 6 slowest tests (at least 2 minutes):
duration env testname
4:04 azure-windows-latest-4core-16gb TestAccept
3:40 aws linux-2core-8gb TestAccept
3:38 azure linux-2core-8gb TestAccept
3:29 gcp linux-2core-8gb TestAccept
3:24 aws-windows-latest-4core-16gb TestAccept
3:21 gcp-windows-latest-4core-16gb TestAccept

@shreyas-goenka
shreyas-goenka added this pull request to the merge queue Oct 28, 2026
Merged via the queue into main with commit da7a8be Oct 28, 2026
28 checks passed
@shreyas-goenka
shreyas-goenka deleted the shreyas-goenka_data/shreyas/test-fault-glob-main branch October 28, 2026 17:34
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: da7a8be

Run: 40737204778

Env 🔄​flaky ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 1499 1227 154:55
✅​ aws-windows-latest-4core-16gb 1424 1252 168:43
🔄​ azure linux-2core-8gb 3 1343 1279 111:25
✅​ azure-windows-latest-4core-16gb 1271 1304 130:51
✅​ gcp linux-2core-8gb 1333 1284 128:59
✅​ gcp-windows-latest-4core-16gb 1258 1309 135:35
Test Name azure linux-2core-8gb
🔄​ TestAccept 🔄​f
🔄​ TestAccept/bundle/invariant/no_drift 🔄​f
🔄​ TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=genie_space.yml.tmpl/READPLAN= 🔄​f
Top 50 slowest tests (at least 2 minutes):
duration env testname
17:08 aws linux-2core-8gb TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=
16:22 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=
14:31 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
14:20 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
12:10 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
10:46 aws linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
10:45 aws linux-2core-8gb TestAccept/bundle/resources/clusters/resize-terminated-fallback/DMS=
10:29 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
9:34 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
9:23 gcp linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
8:45 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
8:42 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
8:37 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
8:13 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=true
8:01 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
7:37 aws linux-2core-8gb TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
7:33 aws-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
7:25 aws-windows-latest-4core-16gb TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
7:19 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=true
7:19 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/data_security_mode/DMS=true
7:17 aws linux-2core-8gb TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=1
7:13 aws-windows-latest-4core-16gb TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
7:13 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
7:11 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
7:10 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=
7:10 aws-windows-latest-4core-16gb TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
7:09 gcp-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true
7:09 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
7:09 azure-windows-latest-4core-16gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
7:07 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/update-after-create/DMS=true
7:07 aws-windows-latest-4core-16gb TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=1
7:06 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DMS=
7:06 aws linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
7:05 gcp linux-2core-8gb TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
7:04 aws-windows-latest-4core-16gb TestAccept
7:03 aws-windows-latest-4core-16gb TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
7:03 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/data_security_mode/DMS=
7:01 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/simple/DMS=
7:00 aws linux-2core-8gb TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster.yml.tmpl/READPLAN=
6:59 aws linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=
6:58 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DMS=true
6:57 aws-windows-latest-4core-16gb TestAccept/bundle/invariant/destroy_idempotent/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
6:56 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/update-after-create/DMS=
6:54 azure linux-2core-8gb TestAccept/bundle/resources/apps/lifecycle-started/DMS=true
6:53 aws-windows-latest-4core-16gb TestAccept/bundle/resources/clusters/deploy/simple/DMS=true
6:48 aws linux-2core-8gb TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
6:47 aws linux-2core-8gb TestAccept/bundle/invariant/no_drift/DMS=true/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
6:42 aws linux-2core-8gb TestAccept/bundle/invariant/no_drift/DMS=/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
6:32 aws-windows-latest-4core-16gb TestAccept/bundle/invariant/delete_idempotent/DMS=/INPUT_CONFIG=cluster_libraries.yml.tmpl/READPLAN=1
6:31 gcp linux-2core-8gb TestAccept/bundle/resources/clusters/lifecycle-started/DMS=true

Sankalp-Mittal added a commit that referenced this pull request Oct 28, 2026
@Sankalp-Mittal Isaac
acceptance: use ** for import-file fault patterns
aa0b9d6 
Since #7241, * in a fault pattern matches within one path segment and **
across segments. Upload paths span many segments, so the faults stopped
firing.

Co-authored-by: Isaac <no-reply@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@denik denik denik approved these changes
@anton-107 anton-107 Awaiting requested review from anton-107 anton-107 is a code owner automatically assigned from databricks/eng-deco-cli

Assignees

No one assigned

Labels

DABs DABs related issues

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@shreyas-goenka @eng-dev-ecosystem-bot @denik