Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ Most compositions carry their own `README.md` with parameter and profile detail
- `.github/workflows/release.yml` runs the same script before publishing a tag.
- Where a composition ships a smoke fixture under `compositions/<name>/tests/`, it is excerpted or
adapted from a real consumer repository so the contract is documented with a real repo shape
instead of a synthetic placeholder. Five of the twelve have one today; see
instead of a synthetic placeholder. Six of the twelve have one today; see
[docs/roadmap.md](docs/roadmap.md).

## Docs
Expand Down
52 changes: 44 additions & 8 deletions compositions/db-migrate/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@
## Purpose

Run database migrations from a pnpm + Turborepo monorepo: install the
workspace, build the `@saas/db` package, assume the environment's GitHub-OIDC
plan role, then invoke the migration runner (`dist/runner/cli.js`) in either
`plan` or `apply` mode. The Orun environment name is passed through as
`MIGRATION_ENV` and as `--env`, so plan and apply always target the environment
the run belongs to.
workspace, build the db package (`@saas/db` unless `buildFilter` names another),
then invoke its migration runner (`dist/runner/cli.js`) in either `plan` or
`apply` mode. The Orun environment name is passed through as `MIGRATION_ENV`
and as `--env`, so plan and apply always target the environment the run belongs
to.

## Contract

Expand All @@ -20,9 +20,38 @@ the run belongs to.
- **Job:** `migrate` (default) — `jobs/db-migrate-run.yaml`
- **Default profile:** `apply`
- **Required parameters:** `nodeVersion`, `pnpmVersion`
- **Optional parameters:** `owner`, `repo`
- **Optional parameters:** `buildFilter`, `owner`, `repo`

either profile in CI needs them set.
## Parameters

| Parameter | Default | Meaning |
| ------------- | ---------- | ------- |
| `nodeVersion` | (required) | Node.js version for `actions/setup-node`. |
| `pnpmVersion` | (required) | pnpm version for `pnpm/action-setup`. |
| `buildFilter` | `@saas/db` | Turbo filter for the package whose `dist/runner/cli.js` runs the migrations. The build step runs `pnpm exec turbo run build --filter=<buildFilter>`. |
| `owner` | (none) | Repository owner, accepted for callers that still pass it. |
| `repo` | (none) | Repository name, accepted for callers that still pass it. |

The plan and apply steps run from the component's directory and call
`node ../../dist/runner/cli.js`. Point the component's `path` at the migrations
directory two levels below the db package (for example
`packages/db/src/migrations`), and set `buildFilter` to that package's name.

`buildFilter` fails quietly when it is wrong. The build step builds whatever
package the filter names and goes green; the plan or apply step then fails with
`MODULE_NOT_FOUND` for a `dist` that was never built. If a run's build step
reports `Packages in scope: @saas/db` and you set another filter, the parameter
is not reaching the template.

```yaml
spec:
type: db-migrate
path: packages/db/src/migrations
parameters:
nodeVersion: "20"
pnpmVersion: "10.12.1"
buildFilter: "@example/db"
```

## Profiles

Expand All @@ -41,4 +70,11 @@ and persist the pnpm store cache, keyed on the workspace lockfile.
`./scripts/verify.sh` is the catalog's only gate. It asserts that this
directory's `composition.yaml` has a `metadata.name` and `spec.type` matching
the directory, that `schema.yaml` and `jobs/` exist beside it, then packs the
catalog and confirms `db-migrate` exports from the packed artifact.
catalog and confirms `db-migrate` exports from the packed artifact. It then
plans two components through the packed artifact, one setting `buildFilter`
and one leaving it out, and checks the rendered build step of each: the first
must build the declared package, and the second must render exactly the 0.21.0
command, `pnpm exec turbo run build --filter=@saas/db`.

`tests/smoke/component.yaml` is a fixture adapted from a real consumer: a db
package that is not `@saas/db`, with its migrations two levels below it.
11 changes: 10 additions & 1 deletion compositions/db-migrate/jobs/db-migrate-run.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,16 @@ spec:
- id: build-db-package
name: build-db-package
capability: db-migrate.build
run: pnpm exec turbo run build --filter=@saas/db
# buildFilter names the package whose dist/runner/cli.js the plan and
# apply steps run. Absent, this renders exactly as 0.21.0 did. orun does
# not copy the schema's default into .parameters, so the else branch is
# what supplies @saas/db.
#
# The guard and the value must read the same key, .parameters.buildFilter.
# A guard on any other key is always false: the step builds @saas/db, goes
# green, and the mismatch only shows two steps later as MODULE_NOT_FOUND.
# verify.sh renders this step with and without the parameter.
run: pnpm exec turbo run build --filter={{if .parameters.buildFilter}}{{.parameters.buildFilter}}{{else}}@saas/db{{end}}
onFailure: stop

- id: migrate-plan
Expand Down
9 changes: 9 additions & 0 deletions compositions/db-migrate/schema.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,15 @@ spec:
type: string
pnpmVersion:
type: string
buildFilter:
type: string
minLength: 1
default: "@saas/db"
description: >-
Turbo filter for the package whose dist/runner/cli.js runs the
migrations. The job reaches it as ../../dist/runner/cli.js, so it
must be the package that holds this component's migrations
directory. Defaults to @saas/db.
owner:
type: string
repo:
Expand Down
5 changes: 5 additions & 0 deletions compositions/db-migrate/tests/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Tests for db-migrate

These files provide contract or smoke fixtures that CI can inspect for `db-migrate` without requiring a full consumer repository checkout.

- `tests/smoke` — a product whose db package is not `@saas/db`, adapted from a real consumer. `scripts/verify.sh` plans it through the packed artifact and checks that the build step builds the package `buildFilter` names.
53 changes: 53 additions & 0 deletions compositions/db-migrate/tests/smoke/component.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
apiVersion: sourceplane.io/v1
kind: Component

metadata:
name: example-db-migrate
description: Applies the example product's migration ladder to its stage and prod Supabase databases

spec:
type: db-migrate
lifecycle: production
owner: platform
domain: infra

dependsOn:
- component: example-db
- component: example-supabase

# The job runs `node ../../dist/runner/cli.js` from here, so the migrations
# sit two levels below the package buildFilter builds.
path: packages/db/src/migrations

parameters:
nodeVersion: "20"
pnpmVersion: "10.12.1"
# The db package is not @saas/db, so name it. Left out, the build step
# builds @saas/db and the apply step fails with MODULE_NOT_FOUND.
buildFilter: "@example/db"

secretEnv:
SUPABASE_ACCESS_TOKEN: "secret://example/example/{{ .environment }}/SUPABASE_ACCESS_TOKEN"
SUPABASE_PROJECT_REF: "secret://example/example/{{ .environment }}/SUPABASE_PROJECT_REF"
SUPABASE_DB_PASSWORD: "secret://example/example/{{ .environment }}/SUPABASE_DB_PASSWORD"
SUPABASE_DB_URL: "secret://example/example/{{ .environment }}/SUPABASE_DB_URL"

subscribe:
environments:
- name: stage
profile: plan
profileRules:
- profile: apply
when:
triggerRef: github-push-main
- name: prod
profile: plan
profileRules:
- profile: apply
when:
triggerRef: github-push-main

labels:
team: platform
layer: data
scope: database
6 changes: 3 additions & 3 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# Roadmap

- Add a smoke fixture under `tests/smoke/` for the seven compositions that still lack one:
`cloudflare-worker`, `cloudflare-worker-turbo`, `cloudflare-workers-assets-turbo`, `db-migrate`,
- Add a smoke fixture under `tests/smoke/` for the six compositions that still lack one:
`cloudflare-worker`, `cloudflare-worker-turbo`, `cloudflare-workers-assets-turbo`,
`publish-stack`, `terraform`, and `turbo-package`.
- Extend `scripts/verify.sh` to validate smoke fixtures against each composition's `schema.yaml`,
not just the structural presence of the contract files.
- Give every composition a README documenting its parameters and profiles; `db-migrate` has none yet.
- Give every composition a README documenting its parameters and profiles.
- Add runnable composition-level examples beyond starter intents.
- Add scheduled verification so packaging regressions surface without waiting for a pull request.
- Add download and adoption metrics once the consumer footprint grows.
15 changes: 14 additions & 1 deletion docs/verification.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ The catalog has one gate, `scripts/verify.sh`, and two workflows that run it.

## `scripts/verify.sh`

The script runs from the repository root and executes seven stages in order. Any failure stops the run.
The script runs from the repository root and executes eight stages in order. Any failure stops the run.

### 1. Structural validation of every composition contract

Expand Down Expand Up @@ -85,6 +85,19 @@ must pass `--x-provision=false` to `wrangler deploy`, the second's must not, and
same `wrangler deploy --config ... --env ...` command. A consumer that has not opted in therefore
deploys exactly as before.

### 8. Render `db-migrate`'s build step with and without `buildFilter`

The script plans two migration components through the packed artifact, on stage and prod: the
composition's own smoke fixture, which sets `buildFilter: "@example/db"`, and one that leaves it out.
Every build step of the first must render `pnpm exec turbo run build --filter=@example/db`. Every
build step of the second must render `pnpm exec turbo run build --filter=@saas/db`, the exact 0.21.0
command, so a consumer that does not set the parameter plans as before.

orun does not copy a schema's `default` into `.parameters`, so the default comes from the template's
`else` branch. A guard and a value that read different keys still renders a green build of `@saas/db`,
and the mismatch only shows when the apply step fails with `MODULE_NOT_FOUND`. This stage fails on it
instead.

## Verify workflow

`.github/workflows/verify.yml` runs on every pull request and every push to `main`. It is a single
Expand Down
68 changes: 68 additions & 0 deletions scripts/verify.sh
Original file line number Diff line number Diff line change
Expand Up @@ -235,3 +235,71 @@ for run in "$opted_in" "$default_run"; do
esac
done
echo "verify: cloudflare-worker-turbo passes --x-provision=false only when a component opts in"

# db-migrate builds the package a component's buildFilter names, and @saas/db
# when it names none, byte for byte the 0.21.0 command. orun does not copy
# schema defaults into .parameters, so the template's else branch is the
# default. A guard and value on different keys renders @saas/db for everyone:
# the build goes green and only apply fails, on MODULE_NOT_FOUND. The component
# that sets buildFilter is the composition's own smoke fixture, so this also
# plans that fixture against the schema.
migrations="$work_dir/db-migrate"
mkdir -p "$migrations/infra/declared" "$migrations/infra/default"
cp "$repo_root/compositions/db-migrate/tests/smoke/component.yaml" "$migrations/infra/declared/component.yaml"
cat > "$migrations/infra/default/component.yaml" <<EOT
apiVersion: sourceplane.io/v1
kind: Component
metadata:
name: saas-db-migrate
spec:
type: db-migrate
path: packages/db/src/migrations
parameters:
nodeVersion: "20"
pnpmVersion: "10.12.1"
subscribe:
environments:
- name: stage
profile: plan
- name: prod
profile: apply
EOT
cat > "$migrations/intent.yaml" <<EOT
apiVersion: sourceplane.io/v1
kind: Intent
metadata:
name: stack-tectonic-render-db-migrate
compositions:
sources:
- name: stack-tectonic
kind: archive
path: $work_dir/stack.tgz
discovery:
roots:
- infra/
automation:
triggerBindings:
github-push-main:
on:
provider: github
event: push
branches:
- main
environments:
stage: {}
prod: {}
EOT
(cd "$migrations" && orun plan --no-catalog-refresh --format json --output plan.json >/dev/null)
expect_build() { # <component> <the exact command its stage and prod build steps must render>
builds="$(jq -c --arg c "$1" \
'[.jobs[] | select(.id | startswith($c + ".")) | .steps[] | select(.id == "build-db-package") | .run]' \
"$migrations/plan.json")"
want="$(jq -cn --arg r "$2" '[$r, $r]')"
if [ "$builds" != "$want" ]; then
echo "verify: db-migrate rendered $1's build steps as $builds, expected $want (stage and prod)" >&2
exit 1
fi
}
expect_build example-db-migrate "pnpm exec turbo run build --filter=@example/db"
expect_build saas-db-migrate "pnpm exec turbo run build --filter=@saas/db"
echo "verify: db-migrate builds the declared buildFilter and falls back to @saas/db"
2 changes: 1 addition & 1 deletion stack.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ kind: Stack
metadata:
name: stack-tectonic
title: Stack Tectonic
version: 0.21.0
version: 0.22.0
description: Versioned OCI-hosted catalog of Cloudflare, Terraform and monorepo compositions for Sourceplane repositories.
owner: sourceplane
tags:
Expand Down
Loading