Skip to content

Respect TestRunRequest.exclude when running tests - #26273

Open
Rahmat Eka Satria (rhmatzeka) wants to merge 1 commit into
microsoft:mainfrom
rhmatzeka:fix/test-explorer-exclude
Open

Rahmat Eka Satria (rhmatzeka) wants to merge 1 commit into
microsoft:mainfrom
rhmatzeka:fix/test-explorer-exclude

Conversation

@rhmatzeka

Copy link
Copy Markdown

Fixes #25742. This continues #25743 by Andrew Lavin (@andravin), who said they no longer have time for it; their commits are credited as co-author.

Tests hidden by a Test Explorer filter (anything in TestRunRequest.exclude) were still started and run. This builds the excluded set once, including descendants, and skips those items when expanding the request into test cases, in both the workspace path (WorkspaceTestAdapter.executeTests) and the project-based path (executeTestsForProjects).

The review comments on #25743 are covered:

  • Rebased on main. It's a fresh branch off current main, which now has the project-based execution path.
  • Test-ID deduplication is preserved. Both paths still collect run IDs through the existing sets; exclusion only filters nodes before that. There's a test for an overlapping selection ([TestClass, test1] with test2 excluded) sending test1 once.
  • pytest is not invoked when exclusions leave no tests. The workspace path now returns before calling the execution adapter when an exclude set was given and no IDs are left, since an empty ID list makes pytest run everything. The project path already returned early on an empty list; there's a test that a project whose tests are all excluded is skipped while the other project still runs.

One unrelated test change: the existing "the right tests should be sent to be executed" test stubbed getTestCaseNodes with sinon.stub instead of the suite's sandbox, so the stub was never restored and leaked into later tests in the suite. It now uses sandbox.stub.

Testing: npm run test:unittests passes for everything under src/test/testing/testController (276 tests); 12 of the new tests fail against the old code. The only failures in the full suite are environment-related on my machine (no python on PATH, no native locator binary) and are in files this PR doesn't touch. eslint and prettier --check pass on the changed files.

Tests hidden by a Test Explorer filter (or otherwise excluded from the
run request) were still started and run. Build the excluded set once,
including descendants, and skip those items when expanding the request
into test cases, for both the workspace and the project-based paths.

If nothing is left after exclusions, don't start the adapter at all:
an empty ID list makes pytest run every test.

Test-ID deduplication is unchanged.

Fixes microsoft#25742

Co-authored-by: Andrew Lavin <alavin@acm.org>
@rhmatzeka

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Test runs ignore TestRunRequest.exclude (filtered tests still run)

1 participant