Skip to content

FIX: Deflake arrow reader cross-thread cancel test - #857

Merged
ttk (Theekshna) merged 1 commit into
mainfrom
theekshna-fix-flaky-cancel-and-gil-perf-tests
Oct 22, 2026
Merged

ttk (Theekshna) merged 1 commit into
mainfrom
theekshna-fix-flaky-cancel-and-gil-perf-tests

Conversation

@Theekshna

@Theekshna ttk (Theekshna) commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Work Item / Issue Reference

GitHub Issue: #856


Summary

This fixes the intermittent failure of tests/test_004_cursor_arrow.py::test_arrow_reader_cancel_from_other_thread (#856). The failure was seen in microsoft/mssql-rs CI, which runs the stress tests against the Rust mssql-odbc driver. The test passed on other runs at the same commit. Only the test file changes.

  • When SQLCancel from the closer thread lands while the driver is inside DDBCSQLFetchArrowBatch, the fetch returns SQLSTATE HY008. mssql-python raises that as OperationalError("Operation canceled"), which escaped the test. ODBC specifies HY008 for this case, so it's a legitimate outcome of the race the test sets up.
  • The test now also accepts OperationalError when driver_error == "Operation canceled". HY008 is the only SQLSTATE mapped to that message in exceptions.py, because exceptions don't expose the SQLSTATE itself. Any other OperationalError is re-raised.
  • All post-conditions are unchanged: the closer thread finished and raised nothing, reader.closed is true, and the parent cursor is reusable.

Validation

  • python -m pre_commit run black-check --all-files --hook-stage pre-push: passed.
  • I did not run the test locally because no SQL Server or built ddbc_bindings was available in this environment. The test is marked stress, so CI excludes it by default.

Copilot AI balanced review requested due to automatic review settings October 22, 2026 05:34
@github-actions github-actions Bot added the pr-size: small Minimal code update label Oct 22, 2026
@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

PR Performance Report

✅ No regression detected

No consistent slowdowns detected across all 2 environments.

0 IMPROVEMENTS 0 SLOWDOWNS 2/2 ENVIRONMENTS

Coverage: 2 of 2 environments completed. Advisory result; does not block merging.

Performance diagnostics

Phase times are inclusive diagnostics and must not be added together. They identify where measured time changed, not why it changed.

No affected phases or call-count changes were recorded.

All database tasks and timings

Unix / SQL Server 2022

Database task Before After Paired change Result
Connection opening 10.104 ms 10.133 ms +0.4% no signal
SELECT queries 1.049 ms 1.038 ms -1.4% no signal
Row insertion 31.424 ms 31.578 ms +0.0% no signal
Executemany inserts 155.676 ms 156.199 ms +0.3% no signal
Fetch-all queries 118.418 ms 118.799 ms +0.3% no signal
Row-by-row fetching 14.993 ms 14.716 ms -1.9% no signal
Batched row fetching 115.114 ms 115.316 ms +0.6% no signal
Transaction commit and rollback 102.195 ms 102.864 ms +1.1% no signal
Arrow row fetching 88.802 ms 89.366 ms +0.5% no signal
100,000-row insertion 425.210 ms 425.658 ms +0.9% no signal
Row fetching in batches of 100 119.426 ms 119.329 ms -0.2% no signal
Row fetching in batches of 10,000 132.870 ms 122.071 ms -6.0% no signal
Repeated positional queries 31.029 ms 30.910 ms -3.5% no signal
Repeated named-parameter queries 33.339 ms 33.263 ms -1.0% no signal
Legacy 100,000-row insertion 339.845 ms 338.524 ms -0.1% no signal
Insertion with explicit input sizes 469.060 ms 477.863 ms +3.0% no signal
Joined aggregation queries 175.628 ms 175.747 ms +0.1% no signal
Large joined-result fetching 179.254 ms 180.111 ms +0.4% no signal
1.2-million-row fetching 3422.605 ms 3408.529 ms -0.1% no signal
Common table expression queries 5.351 ms 5.412 ms +2.5% no signal
256 KiB VARCHAR(MAX) / fetchall() 1.536 ms 1.366 ms -0.6% no signal
10,000 scalar values / fetchval() (debug disabled) 105.812 ms 117.697 ms +6.9% no signal

Unix / SQL Server 2025

Database task Before After Paired change Result
Connection opening 97.711 ms 97.963 ms +0.6% no signal
SELECT queries 1.097 ms 1.124 ms +2.4% no signal
Row insertion 34.583 ms 34.590 ms +0.7% no signal
Executemany inserts 155.193 ms 153.795 ms +0.1% no signal
Fetch-all queries 121.700 ms 121.597 ms -0.9% no signal
Row-by-row fetching 14.628 ms 14.547 ms -0.6% no signal
Batched row fetching 116.449 ms 116.649 ms +0.3% no signal
Transaction commit and rollback 114.294 ms 114.611 ms -0.1% no signal
Arrow row fetching 92.783 ms 93.500 ms +1.0% no signal
100,000-row insertion 437.846 ms 455.182 ms +2.3% no signal
Row fetching in batches of 100 119.581 ms 122.646 ms +1.3% no signal
Row fetching in batches of 10,000 136.503 ms 142.058 ms +4.5% no signal
Repeated positional queries 33.675 ms 33.575 ms -0.5% no signal
Repeated named-parameter queries 36.627 ms 36.083 ms -1.8% no signal
Legacy 100,000-row insertion 352.841 ms 389.678 ms +7.7% no signal
Insertion with explicit input sizes 485.744 ms 482.986 ms -1.0% no signal
Joined aggregation queries 162.064 ms 161.617 ms -0.7% no signal
Large joined-result fetching 198.062 ms 180.211 ms -6.1% no signal
1.2-million-row fetching 3481.895 ms 3487.846 ms +0.9% no signal
Common table expression queries 5.100 ms 5.098 ms -0.8% no signal
256 KiB VARCHAR(MAX) / fetchall() 1.462 ms 1.486 ms +0.9% no signal
10,000 scalar values / fetchval() (debug disabled) 108.644 ms 108.184 ms +0.3% no signal
Build and measurement details

ADO build 184879

PR head: 73facf1a7376030b830562efcbd0aaf712c4394f
Base: 9424705251baf8ded41a8a4968216b1fd9b00929
Measured merge: 08dc131435ec9607a93299380f2ef52cf84615d8

  • Unix / SQL Server 2022: Python 3.12.3, x86_64, SQL 16.0.4305.5; 5 paired comparisons and 1 warmup.
  • Unix / SQL Server 2025: Python 3.12.3, x86_64, SQL 17.0.5015.4; 5 paired comparisons and 1 warmup.

A consistent change requires more than 20% median paired movement, at least 1 ms between the median runtimes, and at least 80% of pairs exceeding the relative threshold in the same direction. A slowdown without enough pair agreement is reported as inconsistent.

The displayed change is the median of paired before-and-after ratios. It is not recalculated from the two displayed median runtimes.

Both revisions use profiling-enabled builds on the same agent and database, with alternating order and discarded warmups. Results are diagnostic and do not represent production-wheel latency.

Raw samples and logs are attached to the ADO run as profiler-* artifacts.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The narrow test-only changes match #856 and preserve unrelated-error detection, with no identified blocking issues.

No open issues 🎉

What changed in this PR

This test-only PR addresses the intermittent stress-test failures reported in #856 without changing driver behavior.

Changes:

  • Accepts the cancellation-specific OperationalError during cross-thread Arrow reader shutdown while re-raising unrelated errors and preserving postconditions.
  • Lowers the connection-concurrency speedup threshold from >2x to >1.5x and aligns the documentation and failure message.

The affected stress tests were not run locally; runtime validation remains outstanding.

File Description
tests/​test_021_concurrent_connection_perf.py Introduces MIN_SPEEDUP = 1.5 to tolerate shared-CI contention.
tests/​test_004_cursor_arrow.py Accepts the expected cancellation error during concurrent reader closure.

Review effort: Balanced

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

📊 Code Coverage Report

🔥 Diff Coverage

100%


🎯 Overall Coverage

85%


📈 Total Lines Covered: 9443 out of 11092
📁 Project: mssql-python


Diff Coverage

Diff: main...HEAD, staged and unstaged changes

No lines with coverage information in this diff.


📋 Files Needing Attention

📉 Files with overall lowest coverage (click to expand)
mssql_python.pybind.performance_counter.hpp: 0.7%
mssql_python.pybind.logger_bridge.cpp: 57.9%
mssql_python.pybind.ddbc_bindings.h: 62.6%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.pybind.connection.connection_pool.cpp: 82.3%
mssql_python.pybind.connection.connection.cpp: 83.1%
mssql_python.logging.py: 86.2%
mssql_python.pooling.py: 90.1%
mssql_python.pybind.fetch_temporal.hpp: 92.1%
mssql_python.cursor.py: 92.5%

🔗 Quick Links

⚙️ Build Summary 📋 Coverage Details

View Azure DevOps Build

Browse Full Coverage Report

test_arrow_reader_cancel_from_other_thread: accept OperationalError for
HY008 (Operation canceled) when SQLCancel lands mid-fetch; re-raise others.

GitHub Issue: #856

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: fe8e84e6-37b7-49d5-b0ca-09c5d1adb60c
@Theekshna
ttk (Theekshna) force-pushed the theekshna-fix-flaky-cancel-and-gil-perf-tests branch from 6711bae to 73facf1 Compare October 22, 2026 06:01
Copilot AI balanced review requested due to automatic review settings October 22, 2026 06:01
@Theekshna ttk (Theekshna) changed the title FIX: Deflake arrow reader cross-thread cancel and GIL-release speedup tests FIX: Deflake arrow reader cross-thread cancel test Oct 22, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The narrow exception filter handles the expected cancellation outcome without weakening unrelated-error handling or existing post-conditions.

No open issues 🎉

Review effort: Balanced

ttk (Theekshna) pushed a commit to microsoft/mssql-rs that referenced this pull request Oct 22, 2026
The arrow cross-thread cancel test is fixed upstream in microsoft/mssql-python#857.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 1ffa58e9-92dc-4a59-bfc3-e3ffd820fbc5

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Approving

@Theekshna
ttk (Theekshna) enabled auto-merge (squash) October 22, 2026 07:02
@Theekshna
ttk (Theekshna) merged commit bd0aaf8 into main Oct 22, 2026
37 checks passed
@Theekshna
ttk (Theekshna) deleted the theekshna-fix-flaky-cancel-and-gil-perf-tests branch October 22, 2026 08:44
ttk (Theekshna) pushed a commit to microsoft/mssql-rs that referenced this pull request Oct 22, 2026
bd0aaf89 is 94247052 (validated by main CI builds 184870/184884) plus microsoft/mssql-python#857.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 1ffa58e9-92dc-4a59-bfc3-e3ffd820fbc5
ttk (Theekshna) pushed a commit to microsoft/mssql-rs that referenced this pull request Oct 22, 2026
The arrow cross-thread cancel test is fixed upstream in microsoft/mssql-python#857.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 1ffa58e9-92dc-4a59-bfc3-e3ffd820fbc5
ttk (Theekshna) pushed a commit to microsoft/mssql-rs that referenced this pull request Oct 22, 2026
bd0aaf89 is 94247052 (validated by main CI builds 184870/184884) plus microsoft/mssql-python#857.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 1ffa58e9-92dc-4a59-bfc3-e3ffd820fbc5
ttk (Theekshna) added a commit to microsoft/mssql-rs that referenced this pull request Oct 22, 2026
…ation (#763)

* Deselect timing-flaky mssql-python stress tests in mssql-odbc PR validation

Fixes #762

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 1ffa58e9-92dc-4a59-bfc3-e3ffd820fbc5

* Deselect only the GIL-release speedup test

The arrow cross-thread cancel test is fixed upstream in microsoft/mssql-python#857.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 1ffa58e9-92dc-4a59-bfc3-e3ffd820fbc5

* Bump mssql-python pin to include the arrow cancel deflake

bd0aaf89 is 94247052 (validated by main CI builds 184870/184884) plus microsoft/mssql-python#857.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 1ffa58e9-92dc-4a59-bfc3-e3ffd820fbc5

* Split PYTEST_DESELECT on newlines too

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 1ffa58e9-92dc-4a59-bfc3-e3ffd820fbc5

---------

Co-authored-by: Theekshna Kotian <tkotian@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 1ffa58e9-92dc-4a59-bfc3-e3ffd820fbc5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-size: small Minimal code update

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants