Repository navigation
FIX: Deflake arrow reader cross-thread cancel test - #857
Conversation
PR Performance Report✅ No regression detectedNo consistent slowdowns detected across all 2 environments. 0 IMPROVEMENTS 0 SLOWDOWNS 2/2 ENVIRONMENTS Coverage: 2 of 2 environments completed. Advisory result; does not block merging. Performance diagnosticsPhase times are inclusive diagnostics and must not be added together. They identify where measured time changed, not why it changed. No affected phases or call-count changes were recorded. All database tasks and timingsUnix / SQL Server 2022
Unix / SQL Server 2025
Build and measurement detailsPR head:
A consistent change requires more than 20% median paired movement, at least 1 ms between the median runtimes, and at least 80% of pairs exceeding the relative threshold in the same direction. A slowdown without enough pair agreement is reported as inconsistent. The displayed change is the median of paired before-and-after ratios. It is not recalculated from the two displayed median runtimes. Both revisions use profiling-enabled builds on the same agent and database, with alternating order and discarded warmups. Results are diagnostic and do not represent production-wheel latency. Raw samples and logs are attached to the ADO run as |
There was a problem hiding this comment.
🟢 Approval recommended
The narrow test-only changes match #856 and preserve unrelated-error detection, with no identified blocking issues.
No open issues 🎉
What changed in this PR
This test-only PR addresses the intermittent stress-test failures reported in #856 without changing driver behavior.
Changes:
- Accepts the cancellation-specific
OperationalErrorduring cross-thread Arrow reader shutdown while re-raising unrelated errors and preserving postconditions. - Lowers the connection-concurrency speedup threshold from >2x to >1.5x and aligns the documentation and failure message.
The affected stress tests were not run locally; runtime validation remains outstanding.
| File | Description |
|---|---|
tests/test_021_concurrent_connection_perf.py |
Introduces MIN_SPEEDUP = 1.5 to tolerate shared-CI contention. |
tests/test_004_cursor_arrow.py |
Accepts the expected cancellation error during concurrent reader closure. |
Review effort: Balanced
📊 Code Coverage Report
Diff CoverageDiff: main...HEAD, staged and unstaged changesNo lines with coverage information in this diff. 📋 Files Needing Attention📉 Files with overall lowest coverage (click to expand)mssql_python.pybind.performance_counter.hpp: 0.7%
mssql_python.pybind.logger_bridge.cpp: 57.9%
mssql_python.pybind.ddbc_bindings.h: 62.6%
mssql_python.pybind.logger_bridge.hpp: 70.8%
mssql_python.pybind.connection.connection_pool.cpp: 82.3%
mssql_python.pybind.connection.connection.cpp: 83.1%
mssql_python.logging.py: 86.2%
mssql_python.pooling.py: 90.1%
mssql_python.pybind.fetch_temporal.hpp: 92.1%
mssql_python.cursor.py: 92.5%🔗 Quick Links
|
test_arrow_reader_cancel_from_other_thread: accept OperationalError for HY008 (Operation canceled) when SQLCancel lands mid-fetch; re-raise others. GitHub Issue: #856 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: fe8e84e6-37b7-49d5-b0ca-09c5d1adb60c
6711bae to
73facf1
Compare
The arrow cross-thread cancel test is fixed upstream in microsoft/mssql-python#857. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1ffa58e9-92dc-4a59-bfc3-e3ffd820fbc5
Jahnvi Thakkar (jahnvi480)
left a comment
There was a problem hiding this comment.
LGTM! Approving
bd0aaf89 is 94247052 (validated by main CI builds 184870/184884) plus microsoft/mssql-python#857. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1ffa58e9-92dc-4a59-bfc3-e3ffd820fbc5
The arrow cross-thread cancel test is fixed upstream in microsoft/mssql-python#857. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1ffa58e9-92dc-4a59-bfc3-e3ffd820fbc5
bd0aaf89 is 94247052 (validated by main CI builds 184870/184884) plus microsoft/mssql-python#857. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1ffa58e9-92dc-4a59-bfc3-e3ffd820fbc5
…ation (#763) * Deselect timing-flaky mssql-python stress tests in mssql-odbc PR validation Fixes #762 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1ffa58e9-92dc-4a59-bfc3-e3ffd820fbc5 * Deselect only the GIL-release speedup test The arrow cross-thread cancel test is fixed upstream in microsoft/mssql-python#857. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1ffa58e9-92dc-4a59-bfc3-e3ffd820fbc5 * Bump mssql-python pin to include the arrow cancel deflake bd0aaf89 is 94247052 (validated by main CI builds 184870/184884) plus microsoft/mssql-python#857. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1ffa58e9-92dc-4a59-bfc3-e3ffd820fbc5 * Split PYTEST_DESELECT on newlines too Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1ffa58e9-92dc-4a59-bfc3-e3ffd820fbc5 --------- Co-authored-by: Theekshna Kotian <tkotian@microsoft.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1ffa58e9-92dc-4a59-bfc3-e3ffd820fbc5
Work Item / Issue Reference
Summary
This fixes the intermittent failure of
tests/test_004_cursor_arrow.py::test_arrow_reader_cancel_from_other_thread(#856). The failure was seen in microsoft/mssql-rs CI, which runs the stress tests against the Rustmssql-odbcdriver. The test passed on other runs at the same commit. Only the test file changes.SQLCancelfrom the closer thread lands while the driver is insideDDBCSQLFetchArrowBatch, the fetch returns SQLSTATE HY008. mssql-python raises that asOperationalError("Operation canceled"), which escaped the test. ODBC specifies HY008 for this case, so it's a legitimate outcome of the race the test sets up.OperationalErrorwhendriver_error == "Operation canceled". HY008 is the only SQLSTATE mapped to that message inexceptions.py, because exceptions don't expose the SQLSTATE itself. Any otherOperationalErroris re-raised.reader.closedis true, and the parent cursor is reusable.Validation
python -m pre_commit run black-check --all-files --hook-stage pre-push: passed.ddbc_bindingswas available in this environment. The test is markedstress, so CI excludes it by default.