Skip to content

test(transaction-scope): read Sch-M at a held load instead of sampling it - #888

Merged
axellpadilla merged 1 commit into
dbt-msft:masterfrom
axellpadilla:test/sch-m-gated-load
Oct 3, 2026
Merged

axellpadilla merged 1 commit into
dbt-msft:masterfrom
axellpadilla:test/sch-m-gated-load

Conversation

@axellpadilla

Copy link
Copy Markdown
Collaborator

The pre_hook_transaction_scope lock tests sampled the building session every 0.1 s during a 1.5M-row load and failed when a sample landed on the load's last instant, as in #884's pyodbc / SQL2025 job. Two momentary Sch-M locks show up there: the load batch's trailing DROP VIEW, and one taken as the minimally logged INSERT commits (seen under PREEMPTIVE_OS_FLUSHFILEBUFFERS). Neither is held across the load, but a sampler can't tell them apart.

The tests now hold the load instead of racing it:

  • The test holds an X lock on one row of a two-row source in its own transaction. The empty create reads no rows and passes; the INSERT, the only statement that reads rows from the source, waits mid-load.
  • Once that INSERT is waiting on the lock (LCK_M_%, TABLOCK in its current statement), the building session's Sch-M is read once, and the gate is rolled back.
  • READCOMMITTEDLOCK on the model's source keeps the wait under READ_COMMITTED_SNAPSHOT.
  • The gate rolls back explicitly before closing: with mssql-python's pooling, close() alone left the transaction open and the load waiting.

Same five cases and assertions. The build-scope case checks Sch-M at one point mid-load rather than in every sample; a transaction's locks are held to commit, so it can't have been released earlier.

Verification

  • test_pre_hook_transaction_scope.py with -n 4: 18 passed in each of 6 runs on mssql-python 1.14.0 and 6 on pyodbc (ODBC Driver 18), SQL Server 2022 CU27. The sampling version failed about 1 run in 6 on pyodbc locally.
  • Giving TestBuildScopeWithoutAnInTxHookHoldsNothing an in-transaction pre-hook makes it fail with Sch-M held during the load.
  • The five lock tests: 57 s to 4 s serially; the file: about 28 s to 6.5 s at -n 4.
  • tests/functional, mssql-python: 424 passed, 40 skipped, 2 xfailed.

…g it

The lock tests sampled the building session every 0.1s during a 1.5M-row
load and failed on a sample that landed at the very end: the batch's
trailing DROP VIEW, or a Sch-M taken for a moment as the minimally logged
INSERT commits (seen under PREEMPTIVE_OS_FLUSHFILEBUFFERS). Neither is held
across the load, but a sampler can't tell them apart.

The test now holds an X lock on one source row, so the INSERT, the only
statement that reads rows from the source, waits mid-load. The locks are
read once at that point, then the gate is rolled back. READCOMMITTEDLOCK
keeps the wait under READ_COMMITTED_SNAPSHOT. The five lock tests drop from
57s to 4s serially.
@axellpadilla
axellpadilla merged commit 53416cd into dbt-msft:master Oct 3, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant