[jasonkuhrt]

Motivation

When a task fails, vp run cancels the whole run. Tasks that are still running are killed with exit code 137, no new task starts, and none of them gets a cache entry. In a recursive lint or test over many packages, the first failure hides every other package's result, and the killed tasks run again from nothing next time. #822 asks for an option to keep going.

This PR is the first step of #822. Tasks that depend on a failed task don't run and aren't listed, the same way fast-fail already treats the tasks it never started. Listing them as skipped, as #822 proposes, needs decisions about the reporter and the summary format, so it's left for a follow-up.

Part of #822.

flowchart TB
  subgraph today["Today: a failure cancels the run"]
    direction TB
    g1["db#generate<br/>failed"]:::failed --> a1["api#lint<br/>not run"]:::notrun
    g1 --> w1["web#lint<br/>not run"]:::notrun
    u1["utils#lint<br/>killed, exit 137, not cached"]:::failed
  end
  subgraph cont["With --continue"]
    direction TB
    g2["db#generate<br/>failed"]:::failed --> a2["api#lint<br/>not run"]:::notrun
    g2 --> w2["web#lint<br/>not run"]:::notrun
    u2["utils#lint<br/>passed, cached"]:::passed
  end
  classDef failed fill:#FCEBEB,stroke:#A32D2D,color:#791F1F
  classDef passed fill:#EAF3DE,stroke:#3B6D11,color:#27500A
  classDef notrun fill:#F1EFE8,stroke:#5F5E5A,color:#444441

Loading

Both runs exit with a non-zero status. Today utils#lint is reported as failed (137) and runs again next time. With --continue it is cached, and api#lint and web#lint aren't listed, as with fast-fail today.

Change

vp run gets a --continue flag (RunFlags::continue_on_failure). Without it, nothing changes.

With it, a failed task doesn't cancel fast_fail_token. Running tasks aren't killed, new tasks keep being scheduled, and tasks that succeed are cached through the existing path. Ctrl-C still cancels everything.

In the scheduler, execute_leaf, execute_node and execute_expanded_graph now return whether they failed. A failed node doesn't release its dependents, so their dependency counts never reach zero and they never start, directly or through other tasks. A node stops at its first failed && item. A task whose script runs vp run fails if any task in that nested graph fails, so its own dependents don't start either.

The exit status needs no change: once every task that can run has finished, the reporter already exits with the failed task's exit code for one failure, and 1 for several.

New exit_codes snapshot cases cover a task that finishes and is cached beside a failure and replays from the cache on the next run, dependents of a failure and of a failure in a nested vp run, a && chain that stops at its failure, and the exit code with two failures. docs/cancellation.md describes the flag in its Fast-fail section.

[limzykenneth]

I'm not sure why this behavior is guarded behind a --continue flag rather than being the default? Could it not have been that in the case where a task in the graph failed, nothing gets cached but succeeded tasks keep their succeeded status while tasks not depending on failed tasks continue to completion?

If the concern is that even when a task failed, its output may be the one contributing to other failures, simply not caching the entire graph when there is a failure, while reporting accurate exit codes for individual task seems to be a sensible default that provides the maximum amount of information.