Repository navigation
ci: select pytest worker count automatically - #5132
Merged
KSchlobohm merged 1 commit intoOct 26, 2026
Merged
Conversation
Use pytest-xdist auto selection within the existing OS/Python matrix. Assisted-by: GitHub Copilot (model: unknown, Auto mode, autonomous) Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
Full six-job validation of the rebased patch remains unverified, particularly macOS/Python 3.13 with automatic workers.
No open issues 🎉
What changed in this PR
Updates Spec Kit’s existing parallel test setup from #5051 so pytest-xdist selects the worker count automatically for each CI runner.
Changes:
- Replace
pytest -n 2withpytest -n auto. - Keep the full suite, six-job OS/Python matrix, dependencies, fixtures, and separate Ruff job unchanged.
| File | Description |
|---|---|
.github/workflows/test.yml |
Enable automatic pytest worker selection. |
Review effort: Max
1 of 5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Use
uv run pytest -n autoinstead of-n 2in the existing six-job OS/Python matrix. Let pytest-xdist select the worker count for each runner. Keep the full suite, matrix, dependencies, fixture behavior, and separate Ruff job unchanged.Builds on #5051, which already added pytest-xdist and faster test-server shutdown.
In a three-attempt fork experiment,
autoselected four workers on Ubuntu and Windows and five on macOS. Ubuntu test steps were approximately 23% faster than two workers. Windows differences were within runner noise; macOS coverage was insufficient for a speed claim. This is observed evidence, not a guaranteed speedup.Testing
uv run specify --helpuv sync && uv run pytestHosted experiment at source commit
380663f9474cd1257a440813dc9d63229cea9c57:uv run pytest -n auto: Ubuntu and Windows passed on Python 3.13 and 3.14 in all three attempts. macOS/Python 3.14 passed once; the other five macOS auto jobs were cancelled without running tests because hosted runners could not be acquired.2,4,auto), 42 test jobs succeeded and 12 were cancelled. No completed test job failed. The overall workflow was not green because of the cancellations.Local Windows/Python 3.14.8 checks on this patch:
Results: dependency setup, CLI help, and whitespace checks passed; 80 focused tests passed with eight auto-selected workers, including valid and invalid adapter metadata cases.
The broader command
.venv\Scripts\python -m pytest tests\test_security_workflow.py tests\specify_cli\integrations\test_installed_adapters.py -n auto -qwas interrupted after failures and stalled progress. A repeat with-x --tb=shortended with 302 passed, 2 skipped, and 3 failed. All three captured failures wereWinError 1314(missing symlink privilege). Repeating those three cases with upstream's-n 2reproduced all three failures:No local full-suite pass is claimed. Sample-project testing is not applicable to this CI-only change.
AI Disclosure
AI disclosure: Prepared with GitHub Copilot in Auto mode; the underlying model and reasoning effort were not exposed. The agent autonomously prepared the one-line workflow change, ran local checks, inspected hosted results, and drafted this description within a human-approved scope. Earlier experimental work was AI-assisted as disclosed in the linked fork experiment.