Skip to content

[Hackathon] fix(ci): generated workflows for different projects cancel each other #583

Open

Description

@Ololadestephen

Verified against (commit SHA or released version)

d61e6df — public main, package version 0.15.0. Built and reproduced locally on macOS, Node v25.7.0, October 20, 2026.

Discord Username / User ID

ololadestephen

What does this improvement do?

Prevent separate TestSprite CI workflows in the same repository from cancelling each other.

The workflow generator assigns every generated job the same concurrency group for a given pull request or deployment commit:

group: testsprite-${{ github.event.pull_request.number || github.event.deployment.sha }}
cancel-in-progress: true

Neither the workflow path nor the TestSprite project/environment identifies the group.

For example, generate testsprite-web.yml for a web project and testsprite-api.yml for an API project. On PR #42, both jobs use testsprite-42. Because GitHub concurrency groups are shared across workflows within a repository, one project's job can cancel the other project's validation.

The deployment-status configuration also collides when different project deployments share the same commit SHA.

This can prevent an independent validation from finishing. If tests were already dispatched, the remote TestSprite work may continue after the Actions job is cancelled. This is not a claim that cancelled checks count as passing.

Details / implementation notes

Affected source

jobs:
testsprite-tests:
# B (pull_request): a new push to the pull request cancels the previous job.
# A (deployment_status): one group per COMMIT — Vercel sets deployment.ref to
# the commit sha, so a new push does not cancel the previous commit's job.
# Job-level, so events the if: below drops (pending, inactive, another
# project's deployment of the same commit) never cancel a running gate.
# Cancelling only stops this job's CLI: the TestSprite run it started keeps
# executing and reports in the Portal; the new push starts its own run.
concurrency:
group: testsprite-\${{ github.event.pull_request.number || github.event.deployment.sha }}
cancel-in-progress: true

Offline reproduction

From a built checkout:

node --input-type=module <<'JS'
import assert from 'node:assert/strict';
import { buildGithubWorkflow } from './dist/commands/ci.js';

const web = buildGithubWorkflow({
  projectId: 'project_web',
  environment: 'staging',
  workflowPath: '.github/workflows/testsprite-web.yml',
});

const api = buildGithubWorkflow({
  projectId: 'project_api',
  environment: 'qa',
  workflowPath: '.github/workflows/testsprite-api.yml',
});

const group = yaml =>
  yaml.split('\n').find(line => line.trimStart().startsWith('group:')).trim();

assert.notEqual(web, api);
assert.equal(group(web), group(api));
assert.ok(web.includes('cancel-in-progress: true'));
assert.ok(api.includes('cancel-in-progress: true'));

console.log({ webGroup: group(web), apiGroup: group(api) });
JS

Both generated workflows have identical concurrency groups despite targeting different projects and using different workflow paths.

Local verification also parsed both generated workflows as YAML and confirmed the differing project commands and identical cancellation configuration. No hosted Actions runs, TestSprite API calls or testing credits were used.

The cancellation consequence follows from GitHub's documented cross-workflow concurrency behavior:
https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#example-only-cancel-in-progress-jobs-or-runs-for-the-current-workflow

Proposed approach

Include a stable workflow-path identifier, such as a short digest, in the generated concurrency group. Preserve the PR-number/deployment-SHA component so successive runs of the same workflow still cancel previous runs.

github.workflow alone would not distinguish the current generated files: they all have the display name TestSprite. Project ID alone would not distinguish separate environment gates for the same project.

Regression coverage should verify:

  • Different workflow paths produce different groups.
  • Separate workflows targeting the same project remain independent.
  • Regenerating the same workflow produces the same group.
  • PR and deployment-status configurations retain cancellation within the same workflow.

Confirmations

  • I have searched existing issues and this is not a duplicate.
  • I have provided my Discord identity above for reward coordination.

Activity

  1. fyZhang66 commented on Oct 22, 2026

    @fyZhang66
    Contributor

    Confirmed on v0.15.0, thanks @Ololadestephen. The generated job's concurrency group is testsprite-<PR number or deployment sha> with no workflow in it (ci.ts:328-330), and concurrency groups are repository-wide. So a repo that gates two TestSprite projects (two sites, or a frontend and a backend project) with two workflows from ci init github --path has them cancel each other on every PR.

    Accepted, scope:

    • Put the workflow file into the group at generation time, as a readable slug of --path (e.g. testsprite-testsprite-api-${{ github.event.pull_request.number || github.event.deployment.sha }}). No digest needed: the generator already knows the path, and regenerating the same file must give the same group.
    • Keep the PR-number / deployment-sha part, so a new push still cancels the same workflow's previous job.
    • Update the snapshot and the comment above the group.
    • Update the two example workflows in DOCUMENTATION.md (around lines 1161 and 1210) to match.
    • Tests (each must fail before the fix): two different --path values give different groups; the same path regenerated gives the same group; both the pull_request and deployment_status shapes still cancel within one workflow.

    No new dependency. Comment /assign to claim it.

  2. added
    acceptedTriaged and accepted — a PR for this will be reviewed
    and removed
    needs-triageNew issue/PR awaiting first triage
    on Oct 22, 2026
  3. Herbicide7665 commented on Oct 22, 2026

    @Herbicide7665

    /assign

  4. testsprite-hob commented on Oct 22, 2026

    @testsprite-hob

    Thanks for the interest, @Herbicide7665! @Ololadestephen opened this issue, so it's reserved for them until 2026-10-24T01:25:53Z (48 hours after it was accepted). If they haven't claimed it by then, comment /assign again.

  5. Ololadestephen commented on Oct 22, 2026

    @Ololadestephen
    Author

    /assign

  6. testsprite-hob commented on Oct 22, 2026

    @testsprite-hob

    Assigned to @Ololadestephen. Thanks for taking this on. (1 more slot available.)

    Your open assigned issues (2/3):

  7. testsprite-hob commented on Oct 23, 2026

    @testsprite-hob

    @Ololadestephen, your claim on this issue expires in about 24 hours unless you open a PR that links it (for example, Closes #583 or Part of #583). You can re-claim later with /assign if the issue is still free.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

acceptedTriaged and accepted — a PR for this will be reviewedhackathonCLI hackathon submissionsin-progressAssigned and actively being worked on

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions