Skip to content

[ci] Omit empty filters when running sliced tests - #13685

Merged
jonathanpeppers merged 1 commit into
mainfrom
simonrozsival-sliced-test-filter-fix
Nov 20, 2026
Merged

jonathanpeppers merged 1 commit into
mainfrom
simonrozsival-sliced-test-filter-fix

Conversation

@simonrozsival

Copy link
Copy Markdown
Member

Context: #13563

Six Windows shards in scheduled build 1691566 failed test discovery with Unknown option '--workers'. The emitted command was --list-tests json --filter --workers --no-banner. The slicer parses options in pairs, so when the empty filter argument is lost, it consumes --workers as the filter value and the remaining options shift. This failure reproduces on macOS with PowerShell Legacy argument passing; the specific cause of argument loss on the affected CI agents remains unknown.

Pass --filter to the slicer only when TEST_FILTER is nonempty, and likewise omit it from MTP test discovery when no filter is supplied. This avoids depending on empty native arguments at either process boundary.

Validation:

  • dotnet build /Users/simon/.copilot/session-state/cf93eb8a-f33b-49cb-ad3c-b43327c25f9d/files/sliced-filter-validation/Fixture.csproj --nologo -v quiet — passed.
  • python3 /Users/simon/.copilot/session-state/cf93eb8a-f33b-49cb-ad3c-b43327c25f9d/files/sliced-filter-validation/validate.py — passed 17 cases against NUnit3TestAdapter 6.3.1, including empty/unset filters, category and name filters, spaces/operators, quoted parameterized names, slicing, no-match behavior, and both PowerShell Standard and Legacy argument passing.
  • YAML parse and git diff --check — passed.

Windows CI agents were not reproduced locally; the specific source of the empty-argument loss remains unconfirmed.

Context: #13563

Six Windows shards in scheduled build 1691566 failed discovery with
`Unknown option '--workers'`. Losing the empty filter argument shifts
the slicer option pairs and makes --workers the filter value. PowerShell
Legacy argument passing reproduces the exact failure; the source of the
argument loss on the affected agents remains unconfirmed.

Only pass --filter to the slicer when TEST_FILTER is nonempty, and omit it
from MTP discovery when no filter is requested. This avoids depending on
empty native arguments at either process boundary.

Validation: the rendered pipeline PowerShell step passed 16 cases against
NUnit3TestAdapter 6.3.1 on macOS, covering unset/empty filters, category and
name filters, spaces/operators, slicing, and no-match failures under both
Standard and Legacy argument passing. An explicit empty filter also passed
when invoking the slicer directly. YAML parsing and git diff --check pass.
Windows agents and CodeQL were not reproduced locally.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 20, 2026 16:50

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues block the changes to omit empty filter arguments at both process boundaries.

No open issues 🎉

What changed in this PR

This PR hardens the CI sliced-test runner introduced in #13563 by omitting empty filter arguments at both process boundaries, avoiding argument shifts when empty arguments are lost.

Changes:

  • Passes --filter from PowerShell only when TEST_FILTER is nonempty.
  • Omits --filter from Microsoft.Testing.Platform test discovery when no filter is supplied.
File Description
build-tools/​automation/​yaml-templates/​run-sliced-nunit-tests.yaml Conditionally supplies filter arguments to the slicer using PowerShell array splatting.
build-tools/​automation/​scripts/​RunSlicedTests.cs Builds discovery arguments with a filter only when its value is nonempty.

Review effort: Balanced

@simonrozsival simonrozsival added the flaky-ci Intermittent CI failures and work to improve CI reliability label Nov 20, 2026
@jonathanpeppers

Copy link
Copy Markdown
Member

CI triage for build 1692412: the build is red (31 jobs passed, 5 failed), but the failures do not appear related to this PR's test-slicing change.

All six failed test results are from AndroidDependenciesTest.InstallAndroidDependenciesTest on Windows, Linux, and macOS. The Android CLI could not install platform-tools@37.0.1. The same failure occurred on the PR's base commit. This matches #13644; PR #13666, which updates the pin to 37.0.3, is still open.

@jonathanpeppers
jonathanpeppers merged commit 9098fd4 into main Nov 20, 2026
33 of 39 checks passed
@jonathanpeppers
jonathanpeppers deleted the simonrozsival-sliced-test-filter-fix branch November 20, 2026 20:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

flaky-ci Intermittent CI failures and work to improve CI reliability

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants