Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/fork-ci-tests.txt
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ fork-ci/actions/ci-demand/ci_demand.test.py ~13s
fork-ci/actions/docs-scope/docs-scope.test.py ~16s
fork-ci/actions/fleet-route/fleet_route.test.py
fork-ci/actions/fleet-watch/fleet_watch.test.py
fork-ci/actions/git-clean/git_clean.test.py
fork-ci/actions/gradle-cache/gradle_cache.test.py
fork-ci/actions/head-check/head_check.test.py ~12s
fork-ci/actions/head-check/head_gate.test.py
Expand Down
1 change: 1 addition & 0 deletions fork-ci/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ Each action's README has a full example, every input and output, and what happen
| [gradle-cache](actions/gradle-cache/README.md) | stages the Maven files a lock lists, sha256-checked, from release assets | `lock`, `out`, `token` | `maven-repo-local` | the lock file |
| [fleet-route](actions/fleet-route/README.md) | which runner each job of a run gets: the project's self-hosted pools or hosted | `config`, `vars`, `mode`, `gates` | `runners`, `mode`, `attempt`, `fallback`, `watch` | the fleet config (e.g. `.github/fleet.json`) |
| [fleet-watch](actions/fleet-watch/README.md) | re-runs, once, a run that a dispatcher pool's runner cut off | `config`, `verb`, `run-id`, `attempt` | `rerun`, `pending` | the fleet config (its `watch` section) |
| [git-clean](actions/git-clean/README.md) | empties a reused self-hosted workspace, refuses a .git config it does not know | `path` | none | none |
| [head-check](actions/head-check/README.md) | fails a pull request head that is not the change its review read | `config`, `required-lenses`, `token` | `result` | the pr-queue config |
| [pr-queue](actions/pr-queue/README.md) | a merge queue for an up-to-date-required repository (also a reusable workflow) | `config`, `state-file`, `cache`, `token` | none | the pr-queue config |
| [weakened-tests](actions/weakened-tests/README.md) | reports (never fails) the existing tests a pull request deletes, skips or weakens | `config`, `token` | `status`, `count`, `missing_why` | `.github/weakened-tests.json` (optional: added patterns) |
Expand Down
4 changes: 4 additions & 0 deletions fork-ci/actions/docs-scope/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,10 @@ outputs:
runs:
using: composite
steps:
# Before any git call: a .git config that actions/checkout does not write (one an earlier job left) is refused.
- name: Refuse a git config actions/checkout does not write (fork-ci git-clean)
shell: bash
run: exec python3 -I "$GITHUB_ACTION_PATH/../git-clean/git_clean.py" check --path . --require
# Only code from this action's folder (the caller's pinned SHA) runs; nothing from the checkout.
# The script also reads the runner's GITHUB_SHA and GITHUB_EVENT_PATH (HEAD must be the run's commit).
- id: scope
Expand Down
7 changes: 4 additions & 3 deletions fork-ci/actions/docs-scope/docs-scope.test.py
Original file line number Diff line number Diff line change
Expand Up @@ -889,14 +889,15 @@ class Action(CleanEnv):

def test_runs_only_its_own_script(self):
runs = re.findall(r'^\s*run: (.*)$', self.text, re.M)
self.assertEqual(runs, ['python3 -I "$GITHUB_ACTION_PATH/docs_scope.py" --config "$CONFIG" '
self.assertEqual(runs, ['exec python3 -I "$GITHUB_ACTION_PATH/../git-clean/git_clean.py" check --path . --require', # the git-clean check of this pinned SHA
'python3 -I "$GITHUB_ACTION_PATH/docs_scope.py" --config "$CONFIG" '
'--event "$EVENT" --base "$BASE"'])
self.assertNotIn('uses:', self.text)
self.assertIn('using: composite', self.text)

def test_inputs_reach_the_script_by_env_only(self): # never ${{ }} inside run: (script injection)
run = re.search(r'^\s*run: (.*)$', self.text, re.M)[1]
self.assertNotIn('${{', run)
for run in re.findall(r'^\s*run: (.*)$', self.text, re.M):
self.assertNotIn('${{', run)

def test_declares_every_output_the_script_writes(self):
for name in ('docs_only', 'jobs', 'reason'):
Expand Down
10 changes: 9 additions & 1 deletion fork-ci/actions/docs-scope/docs_scope.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,9 +47,17 @@ class Unclear(Exception):
"""Anything that makes the answer unknown: the run is full and this is the reason."""


# Every git call: no global or system config (on a self-hosted runner an earlier job may have written ~/.gitconfig), and
# no hooks, fsmonitor command, commit-graph or replace refs from the repository. The same as
# ../git-clean/git_clean.py's (git_clean.test.py pins them); the action's first step refused any other .git config.
GIT_ENV = {'GIT_CONFIG_GLOBAL': os.devnull, 'GIT_CONFIG_NOSYSTEM': '1'}
GIT_SAFE = ('-c', 'core.hooksPath=/dev/null', '-c', 'core.fsmonitor=false', '-c', 'core.commitGraph=false',
'--no-replace-objects')


def git(*args):
"""stdout, or None when git fails. Bytes that are not UTF-8 survive as lone surrogates (never a crash)."""
r = subprocess.run(['git', *args], capture_output=True)
r = subprocess.run(['git', *GIT_SAFE, *args], capture_output=True, env={**os.environ, **GIT_ENV})
return r.stdout.decode('utf-8', 'surrogateescape') if r.returncode == 0 else None


Expand Down
66 changes: 66 additions & 0 deletions fork-ci/actions/git-clean/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# git-clean

A composite action for the **first step of a job, before `actions/checkout`**. A self-hosted runner reuses its
workspace, and an earlier job may have run pull request code there. That code can write `.git/config` keys that
`actions/checkout` does not reset: `core.fsmonitor` (a command git runs during the checkout's own git calls),
`core.hooksPath`, `attr.tree` with a merge driver, an include of another file. The next job would run them before any
of its own steps runs pull request code.

- **Self-hosted runner:** it empties the workspace (`GITHUB_WORKSPACE`), so the checkout starts from nothing, as on a
GitHub-hosted runner. Symlinks are removed as links, never followed: nothing outside the workspace is touched.
- **GitHub-hosted runner:** each job starts on a fresh disk, so nothing is removed.
- **Both:** it then refuses a `.git` at the checkout path whose config `actions/checkout` does not write (below).

The fork-ci actions that read a checkout with git (docs-scope, job-changed, test-select, head-check, pr-queue's
tested-base) run the same check as their first step, and every git call of theirs runs with no global or system
config (`GIT_CONFIG_GLOBAL=/dev/null`, `GIT_CONFIG_NOSYSTEM=1`) and with hooks, `core.fsmonitor`, the commit-graph and
replace refs off. tested-base writes no note instead of failing.

## Usage

Pin a release commit by its full SHA, never a commit of main ([why](../../README.md#pinning)):

```yaml
jobs:
build:
runs-on: [self-hosted, build]
steps:
- uses: scuttletoby/t3code/fork-ci/actions/git-clean@<release sha> # ci/vN
- uses: actions/checkout@<sha>
with: { fetch-depth: 0 }
```

Put it in every job that runs on a self-hosted runner and checks out code, before the first checkout and before any
other step that writes the workspace. With several checkouts, list their paths (`path: | ...`): all of them are
checked.

| Input | Default | Meaning |
| ------ | ------- | -------------------------------------------------------------------- |
| `path` | `.` | the checkout path or paths (actions/checkout's `path`), one per line |

No outputs. A refusal fails the step, so the checkout and every later step do not run.

## What is refused

- `GITHUB_WORKSPACE` that is not an absolute folder, is a symlink, is not inside `RUNNER_WORKSPACE`, or holds `HOME`,
`RUNNER_TEMP`, `RUNNER_TOOL_CACHE` or the action itself: nothing is removed.
- A delete that fails (a file of another user, for example from a container): the step fails with the path. What is
left is removed by the next run once the file can be deleted.
- A checkout path that is absolute or leads outside the workspace.
- `<path>/.git` that is a file or a symlink (a worktree or submodule: git would read another repository), or that holds
`commondir`.
- A key in `.git/config` or `.git/config.worktree` that `actions/checkout` (v4 to v7) and `git init` do not write, or
one of those keys with another kind of value. Allowed: the core settings of `git init`, sparse checkout, the object
format, `gc.auto 0`, `remote.origin` url/fetch/partial clone, `branch.*.remote`/`merge`, the token header, the LFS
filter exactly as `git lfs install` writes it, submodule url/active, `user.name`/`user.email`. The include
checkout writes for its token (`includeIf.gitdir:...path`) only to a `git-credentials-<uuid>.config` in
`RUNNER_TEMP` that holds nothing but the token header. Everything else is refused by name, among it
`core.fsmonitor`, `core.hooksPath`, `core.sshCommand` (checkout's `ssh-key` with `persist-credentials`: use
`persist-credentials: false`), `core.pager`, `attr.tree`, merge/diff/filter drivers, `include.path`, other
includes, `url.*.insteadOf` and `alias.*`.

## Not covered

The runner user's own files outside the workspace (`~/.gitconfig`, the system git config) are not removed: the fork-ci
git calls ignore them, `actions/checkout` does not. A runner that takes pull requests from people who are not trusted
also needs a job home that does not outlive the job.
19 changes: 19 additions & 0 deletions fork-ci/actions/git-clean/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
name: git-clean
description: >-
The first step of a job, before actions/checkout. On a self-hosted runner it empties the reused workspace, so a git
config, hook or file an earlier job (pull request code) left there cannot run during this job's checkout. On a
GitHub-hosted runner (a fresh disk) it removes nothing. Then it refuses a .git config at the checkout path that
actions/checkout does not write. Needs git and python3.
inputs:
path:
description: The checkout path or paths (actions/checkout's path), one per line, relative to the workspace.
required: false
default: "."
runs:
using: composite
steps:
- name: Empty a reused workspace; refuse a git config actions/checkout does not write
shell: bash
env:
CHECKOUT_PATH: ${{ inputs.path }}
run: exec python3 -I "$GITHUB_ACTION_PATH/git_clean.py" prepare --path "$CHECKOUT_PATH"
Loading
Loading