gha-tools

This provides a tool gha to help maintain an allowist of GitHub Actions and reusable workflows for a GitHub organisation. It is used from a per-organisation repo, which holds the allowlist files and pins a version of this package.

Basic usage for updating the allowlist is documented in that repo's README.md - see Setting up an organisation repo.

This README covers:

• How to use gha for other tasks
• How gha works and how to develop it.

Contents

• Overview
• Command summary
• Command details
• Setting up an organisation repo
• Actions file format
• How it works
• Development
• TODO

Overview

The basic concept is that we want to define an allowlist in the format required by the GitHub org-level Actions settings UI, so it can be copy/pasted in by an org admin, avoiding having to provide the admin-level permissions needed to automate this. This file is $ORG.final.txt and it contains only the 3rd-party actions/workflows to allow, as it is assumed that those from GitHub (actions/, github/) and the org itself are always allowed in the org-level Actions settings.

The "user interface" is provided by a file $ORG.actions.txt which contains a list of all actions directly called by the org's workflows, including those from the org itself. The format of this is more permissive, to allow comments to be added to show why an action/workflow has been added or the version for a SHA etc.

The $ORG.final.txt file is generated by running gha update $ORG which reads the $ORG.actions.txt, and recursively reads action/workflow files to find all transitive dependencies which also need to be allowlisted. It then scans (by default) any changes to actions, looking for any which are known-vulnerable (via Zizmor and the GitHub Advisories database). It will warn if any actions/workflows are specified without a SHA, as this is discouraged to avoid mutability. If the scan passes it (re-)generates $ORG.final.txt as well as $ORG.final.json which contains information on tags (versions) matching the specified SHA as well as the call-chain leading to this action to aid debugging.

Other subcommands are provided to help maintain the organisations workflows and allowlist, see below.

Command summary

• gha update ORG: Find all dependencies from actions/workflows in ORG.actions.txt, check for known-vulerable actions/workflows and generate a final allowlist ORG.final.txt.
• gha review ORG: Review all allowlist entries.
• gha review ACTION@REF Review a specific action.
• gha fix ORG REPO ACTION[@REF]: Clone a repo and rewrite its uses: of an action to the allowlisted ref.
• gha scan ORG: Find every action used by the active workflows in an org's repos.
• gha blocked ORG: Report workflows from the last scan which are not allowed by ORG.final.txt.

All commands read and write ORG.* files in the current directory. LOG_LEVEL sets the log level (default INFO).

Command details

This section provides more details on commands, to help organisation allowlist maintainers and gha developers.

Update subcommand

Regenerate the final allowlist ORG.final.txt by finding all dependencies from the actions/workflows listed in ORG.actions.txt, and review changes to the allowlist for known-vulnerable actions/workflows.

Usage:

usage: gha update [--no-review] ORG

See Overview for a functional description. Additional details:

• See Actions file format for the full description of ORG.actions.txt.

• Warnings are raised for any # vX.Y.Z which do not match tags found for a SHA

• The ORG.final.txt and ORG.final.json files are only written if the review succeeds (or --no-review was given).

• The changes to the allowlist to review are determined by comparing what will be written to the new ORG.final.txt with the merge-base of HEAD and origin/main, so only changes made on the current branch are reviewed. This avoids a PR having to deal with failures due to newly-identified vulnerabilities in actions/workflows not changed in the current branch (they will be caught by CI reviews of the entire allowlist instead).

• An error is raised if the review finds any action/workflow with a known vulerability.

• Errors are raised for unresolved dependencies (after retries for transient API errors), to avoid an incomplete allowlist leading to failing workflow runs.

• The entries in ORG.final.txt use exactly the ref given in the uses: calls (i.e. there is no translation from versions to SHAs etc), as this is what the GitHub allowlist uses. Entries are deduplicated sorted to provide stable diffs.

• ORG.final.json records provenance for each entry:

  • Its kind
  • The resolved sha
  • The version from the most-specific tag on that SHA, i.e. if a SHA is tagged both v2 and v2.3 the latter is used
  • The ref_source (is_sha, tag or branch)
  • Whether it is a root (i.e. listed directly in actions.txt)
  • Every call chain leading to it. Chains stop at the nearest root, and cycles are skipped.

  The version information is used by gha fix.

A GitHub token is needed, as both the walk and the review use the API.

Scan subcommand

Find every action and reusable workflow used by the active workflows in an org's repositories, including transitive dependencies.

Usage:

usage: gha scan [--repo REPO] [--resume] ORG

If --repo is passed the scan is limited to the specified repository. Otherwise, every non-archived repository is scanned.

The --resume option allows restarting a scan without reprocessing in the event of a network outage or API failure.

Workflows are skipped if:

• Workflows are disabled on the repo (which needs org admin or repo owner permissions to determine)
• The workflow is "dynamic" (created by GitHub, e.g. Dependabot or Pages)
• The workflow is not "active" (a per-workflow setting)

While the GitHub API endpoint /repos/ORG/REPO/actions/workflows returns the paths for workflows shown in the web UI, a workflow may exist on multiple branches, potentially with different contents. Searching every branch is prohibitively slow, and what what matters is which workflows will be run: if foo/bar@v2 is allowlisted and a default branch uses foo/bar@v1, that branch needs updating, and an old feature branch is then fixed by rebasing or merging on/from the default branch. So when finding workflows first the default branch is checked for the path; if this exists the workflow source is extracted from this branch. Otherwise branches are searched shortest name first (then alphabetically) and the workflow source taken from the first branch on which the path exists.

Finding all actions requires a recursive approach:

• Workflows may call actions or reusable workflows
• Composite actions may call other actions

This forms a DAG but one where nodes may occur multiple times. E.g. multiple workflows may call foo/bar@v2; the same action is used, but the call chain is different.

Scanning involves many API calls, so an attempt at efficiency is required and the chance of a transient API failure is high, so a way to restart without repeating work is required. Hence the walker uses a local database to track "done" nodes in the graph - see below for details.

Review subcommand

Review actions/reusable workflows.

Important

The review cannot determine whether the relevant actions are safe, but only whether an action is known-vulnerable and whether it has mutability which makes that determination weak.

Usage:

gha review [--base BASE] TARGET

where TARGET may be either:

• The org name to review actions from ORG.final.txt
• action@ref to review a specific action

If target is an ORG, --base may be used to restrict reviews to only changes to ORG.final.txt since the given git ref. This is used internally by gha update - see comments there. Otherwise all actions in ORG.final.txt are reviewed.

In either case the review includes transitive dependencies of directly-referenced actions/workflows.

Specifically, it currently checks for and raises:

• An error if an action is known-vulerable (via the GitHub Advisories database)
• A warning if the action reference is weak, i.e. not SHA
• A warning if no version is found for a SHA
• A warning if no release is found for a version
• A warning if a release is mutable
• For docker actions, a warning if the image is not pinned to a SHA
• For docker actions, a warning if the base image in the Dockerfile is not pinned to a SHA
• For node actions, a warning if the entrypoint is not bundled (as this implies mutability)

It also prints information on the type of action found.

Zizmor is used for the review in two ways:

• A "fake composite action" is constructed which uses: the relevant actions, and scanned with Zizmor to determine issues with the uses: themselves, i.e. known-vulnerable actions, unpinned refs etc.
• Each action's "metadata file" (the actual action.y*ml file) is scanned with Zizmore to detect similar issues in the action itself. Other checks are from manual parsing of the metadata file.

Fix subcommand

Clone a repository and rewrite its uses: of an action to the allowlisted ref.

Usage:

gha fix [-h] [--path PATH] ORG REPO ACTION

where:

• ORG: the organisation name
• REPO: the repository name
• ACTION: Either action or action@ref. In the former case if ORG.actions.txt only specifies a single ref for this action that ref is used.

This clones the repo (by default to ~, unless --path is given) and rewrites uses: ACTION@<anything> in .github/workflows/* and any action.y{a,}ml to uses: ACTION@REF # VERSION. The VERSION comes from ORG.final.json.

Note this does not commit changes. They should be checked and a normal PR process followed manually when happy.

Setting up an organisation repo

1. Copy the org-template file to a new repo, replacing @ORG@ with the organisation name.
2. Run gha scan ORG and copy the suggested actions.txt output shown into a file ORG.actions.txt. It may be necessary to manually trim the list of actions or e.g. disable workflows on repositories and rerun the scan, if this is too broad.
3. Run gha update ORG.
4. Paste ORG.final.txt into the org's Actions settings.

Scheduled review workflow

This repo provides a reusable workflow, .github/workflows/review.yml, which an organisation repo can call to run gha review ORG over every entry in ORG.final.txt. As allowlisted actions can be found to be vulnerable after they were added, running it on a schedule catches that without anyone touching the allowlist. The job fails if any entry is known-vulnerable.

An example caller (which runs nightly, on merge to main and on demand) is in the comments at the top of the workflow. The inputs are:

• org (required): the organisation name, i.e. the prefix of ORG.final.txt.
• requirements (default requirements.txt): the file in the calling repo which pins the version of gha-tools to install, so the review runs the same version as local use.
• Secret slack_webhook_url (optional): a Slack incoming webhook. If given, a message is posted when the job fails, for any reason including install errors. Nothing is posted on success.

Unlike gha update this reviews all entries rather than only those added on a branch.

Actions file format

One owner/repo[/path]@ref per line, e.g.:

# Needed for building docs
ConorMacBride/install-package@3e7ad059e07782ee54fa35f827df52aae0626f30, # v1.1.0
anchore/sbom-action@e22c389904149dbc22b58101806040fa8d37a610
stackhpc/some-workflows/.github/workflows/multinode.yml@main

• Comments and blank lines are allowed; use them to say why an action is needed.
• A trailing comma is optional, so lines can be copied from the web UI format.
• Refs for external actions should be SHAs; gha update warns about tags or branches.
• An optional # vX.Y.Z comment documents the version; gha update warns if it does not match the tags on that SHA.
• GitHub-owned actions (actions/*, github/*) should nto be be listed, as they are allowed by the "Allow actions created by GitHub" setting.
• Although internal, ORG-owned actions/workflows are always allowed, they should also be listed so that their external dependencies are found - see below.

Dependencies of internal actions

To explain the last point above; gha update only walks from what is listed in ORG.actions.txt and does not scan the org's own repos. So for an org MYORG, if an internal action MYORG/zob has uses: foo/bar@sha, one of these must be true for foo/bar@sha to end up in MYORG.final.txt:

1. MYORG/zob@ref is listed in MYORG.actions.txt, so gha update walks it and finds foo/bar@sha as a transitive dependency. This is the recommended approach.
2. foo/bar@sha is listed in MYORG.actions.txt directly.

If neither is listed then any workflow run which tries to use MYORG/zob will fail, with the error showing why.

How it works

Modules

Under src/gha_tools/:

• cli.py: argument parsing and dispatch to commands/.
• github.py: the single GitHub API client (REST and GraphQL). Errors are split into NotFound (any 4xx, permanent) and Transient (rate limiting, network, 5xx, worth retrying), which the walker relies on.
• identity.py: parsing uses: strings into a node identity (owner, repo, path, ref) and kind (action, reusable workflow, docker, local, unresolved), plus ref helpers.
• walker.py: the dependency graph walker - see below.
• allowlist.py: reading ORG.actions.txt, building and writing ORG.final.{txt,json}, and reading ORG.final.txt at a git ref.
• discovery.py: finding an org's repos and the workflows in them which are in use (gha scan only).
• commands/*.py: one module per subcommand.
• zizmor.yml: the Zizmor config used by gha review.

The walker

Walker builds the call graph of actions and reusable workflows, starting from roots: workflow files (add_root, used by scan) or uses: strings (add_action_root, used by update). For each node it fetches the workflow, action.yml or action.yaml, parses out the uses: of its steps/jobs, and adds those as child nodes. Workflows can call actions and reusable workflows; composite actions can call actions. The result is a DAG in which the same node can be reached by many call chains.

State is kept in SQLite (a file for scan, :memory: for update):

• nodes: keyed by node identity, i.e. exactly what appears after uses:. Each node is fetched and parsed at most once, however many callers it has.
• edges: (caller, callee, call_site) rows. Meeting an already-processed node just adds an edge.
• ref_resolution: caches ref -> commit SHA lookups per (owner, repo, ref), with the ref source (is_sha, tag, branch) and the most-specific tag on that SHA (the "version").

Local uses: ./path refs are rewritten to owner/repo/path@ref using the caller's identity. The exception is a local ref in a composite action from outside the org being walked: GitHub resolves this against the calling workflow's workspace, not the action's repo, so it can't be known (e.g. pypa/gh-action-pypi-publish generates the action it calls at runtime). As local actions are always allowed, these become local leaves, with a warning. docker:// refs are leaves. Anything which can't be parsed or fetched (a NotFound) becomes an unresolved leaf, rather than stopping the walk. Transient errors leave the node in error, to be retried. Nodes are claimed in_progress while being processed, so a killed scan can be resumed with --resume.

Development

python3 -m venv .venv
. .venv/bin/activate
pip install -e '.[test]'
pytest

The tests in tests/ are offline. tests/cassette.py can also record the GitHub API traffic of a real gha run and replay it later, so changes can be checked against a full org run without repeating it. Recordings may contain private repository contents, so keep them under tests/recordings/ (which is gitignored):

mkdir -p tests/recordings/baseline && cd tests/recordings/baseline
# record (live, needs GITHUB_TOKEN):
python ../../cassette.py record stackhpc.jsonl -- scan stackhpc
# replay (offline; any request not in the recording is a hard failure):
python ../../cassette.py replay stackhpc.jsonl -- scan stackhpc
# replay, fetching live only what is missing from the recording:
python ../../cassette.py fill stackhpc.jsonl -- scan stackhpc

pytest also replays any recordings in tests/recordings/ against the outputs saved next to them - see tests/test_regression.py.