Repository navigation
Implement ability for run-task to fetch secrets #1110
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,98 @@ | ||
| Use Secrets | ||
| =========== | ||
|
|
||
| Tasks often need secrets such as API tokens or private keys. Taskgraph can | ||
| have ``run-task`` fetch these automatically. | ||
|
|
||
| Using Secrets | ||
| ------------- | ||
|
|
||
| Secrets are implemented as part of the :mod:`~taskgraph.transforms.run` | ||
| transforms, and are fetched by ``run-task``. So make sure your task uses those | ||
| transforms and a ``run-task`` based ``run.using``: | ||
|
|
||
| .. code-block:: yaml | ||
|
|
||
| loader: taskgraph.loader.transform:loader | ||
|
|
||
| transforms: | ||
| - taskgraph.transforms.run:transforms | ||
| - taskgraph.transforms.task:transforms | ||
|
|
||
| Then list the secrets in the task's ``secrets`` key: | ||
|
|
||
| .. code-block:: yaml | ||
|
|
||
| tasks: | ||
| deploy: | ||
| run: | ||
| using: run-task | ||
| command: ./deploy.sh | ||
| secrets: | ||
| - type: github-repo-token | ||
| app: my-github-app | ||
| env: GITHUB_TOKEN | ||
| - type: secret | ||
| name: project/myproject/level-3/deploy | ||
| key: privkey | ||
| file: ~/.ssh/id_rsa | ||
|
|
||
| Here ``deploy.sh`` will find a GitHub token in the ``GITHUB_TOKEN`` env and a | ||
| ``~/.ssh/id_rsa`` file containing a private ssh key. | ||
|
|
||
| The transforms add the scopes each secret needs and enable the | ||
| ``taskclusterProxy`` feature, then pass the list to ``run-task`` in the | ||
| ``RUN_TASK_SECRETS`` environment variable. No two secrets can be exported to | ||
| the same variable or file, and a secret can't be exported to a variable that is | ||
| also set in ``worker.env``. | ||
|
|
||
| Substituting Values | ||
| ------------------- | ||
|
|
||
| Secret names often depend on the level or project. Use the | ||
| :ref:`task_context <task_context transforms>` transforms, before the run | ||
| transforms, to substitute parameters into them: | ||
|
|
||
| .. code-block:: yaml | ||
|
|
||
| transforms: | ||
| - taskgraph.transforms.task_context | ||
| - taskgraph.transforms.run:transforms | ||
| - taskgraph.transforms.task:transforms | ||
|
|
||
| tasks: | ||
| deploy: | ||
| task-context: | ||
| from-parameters: | ||
| level: level | ||
| project: project | ||
| substitution-fields: [secrets] | ||
| run: | ||
| using: run-task | ||
| command: ./deploy.sh | ||
| secrets: | ||
| - type: secret | ||
| name: project/{project}/level-{level}/deploy | ||
| env: DEPLOY_TOKEN | ||
|
|
||
| Every string in ``secrets`` is formatted, so write a literal ``{`` or ``}`` as | ||
| ``{{`` or ``}}``. ``secrets`` can also be keyed by a value as a whole, but not | ||
| entry by entry. | ||
|
|
||
| Secret Types | ||
| ------------ | ||
|
|
||
| ``github-repo-token`` | ||
| An installation token of the GitHub App ``app``. ``repo`` defaults to the | ||
| ``base_repository`` parameter, ``permissions`` to ``{"contents": "read"}`` | ||
| and ``env`` to ``GITHUB_TOKEN``. Adds an | ||
| ``auth:github-repo-token:<app>/<owner>/<repo>:<permission>:<level>`` scope | ||
| for each permission. | ||
|
|
||
| ``secret`` | ||
| A secret from Taskcluster's secrets service, exported to ``env`` or written | ||
| to ``file``. ``key`` exports a single key of the secret instead, using dots | ||
| for nested keys such as ``foo.bar``. Adds a ``secrets:get:<name>`` scope. | ||
|
|
||
| See :ref:`run-task-secrets` for the details of each type, including what | ||
| happens when a secret can't be fetched. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -24,10 +24,24 @@ the task can both use them. List them as JSON in ``RUN_TASK_SECRETS``: | |
| - type: github-repo-token | ||
| app: read | ||
| repo: https://github.com/myorg/myrepo | ||
| - type: secret | ||
| name: project/myproject/level-3/deploy | ||
| key: token | ||
| env: DEPLOY_TOKEN | ||
|
|
||
| The only type so far is ``github-repo-token``. It asks Taskcluster's auth service | ||
| for an installation token of the GitHub App ``app``, scoped to ``repo``. Its | ||
| keys are: | ||
| Each entry has a ``type``, which decides its other keys. An unknown ``type`` or | ||
| key, or invalid JSON, fails the task. A variable that is already set is never | ||
| replaced. | ||
|
|
||
| Rather than setting ``RUN_TASK_SECRETS`` by hand, tasks using the | ||
| :mod:`~taskgraph.transforms.run` transforms can list secrets in their | ||
| ``secrets`` key. See :doc:`/howto/use-secrets`. | ||
|
|
||
| github-repo-token | ||
| ~~~~~~~~~~~~~~~~~ | ||
|
|
||
| Asks Taskcluster's auth service for an installation token of the GitHub App | ||
| ``app``, scoped to ``repo``. Its keys are: | ||
|
|
||
| ``app`` (required) | ||
| The name of the GitHub App registered with Taskcluster. | ||
|
|
@@ -37,17 +51,41 @@ keys are: | |
| The token's permissions. Defaults to ``{"contents": "read"}``. | ||
| ``env`` | ||
| The environment variable to export the token in. Defaults to | ||
| ``GITHUB_TOKEN``. A variable that is already set is never replaced. | ||
| ``GITHUB_TOKEN``. | ||
|
|
||
| ``run-task`` also keeps the token for its own GitHub requests. The task needs | ||
| the ``taskclusterProxy`` feature and an | ||
| ``auth:github-repo-token:<app>/<owner>/<repo>:<permission>:<level>`` scope for | ||
| each permission. If a request fails, for example because a pull request from a | ||
| fork lacks the scope, ``run-task`` logs a warning and carries on without the | ||
| token. The same happens when the auth service rejects the request, such as for | ||
| an unknown ``app`` or invalid ``permissions``. Invalid JSON, an unknown ``type`` | ||
| or key, or a ``repo`` that isn't on GitHub fails the task. Installation tokens | ||
| expire after an hour. | ||
| an unknown ``app`` or invalid ``permissions``. A ``repo`` that isn't on GitHub | ||
| fails the task. Installation tokens expire after an hour. | ||
|
|
||
| secret | ||
| ~~~~~~ | ||
|
|
||
| Fetches a secret from Taskcluster's secrets service. Its keys are: | ||
|
|
||
| ``name`` (required) | ||
| The name of the secret. | ||
| ``env`` | ||
| The environment variable to export the secret in. | ||
| ``file`` | ||
| The file to write the secret to. Relative paths are relative to the | ||
| task's working directory. Missing parent directories are created, and on | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is there a potential conflict here?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Checking for this adds a moderate amount of complexity, I don't think it's worth it tbh.. Just.. don't do that :p |
||
| POSIX systems only the task's user can read the file. An existing file is | ||
| never replaced. | ||
| ``key`` | ||
| A key of the secret to export, rather than the whole secret. Dots select | ||
| keys of nested objects, so ``foo.bar`` exports ``secret["foo"]["bar"]``. | ||
|
|
||
| Exactly one of ``env`` or ``file`` is required. A string value is written as | ||
| is, and any other value as JSON. If the variable is already set or the file | ||
| already exists, the secret isn't fetched. The task needs | ||
| the ``taskclusterProxy`` feature and a ``secrets:get:<name>`` scope. Unlike | ||
| ``github-repo-token``, a failure to fetch the secret, or a missing ``key``, | ||
| fails the task. | ||
|
|
||
| .. _run-task-base-revisions: | ||
|
|
||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.