Skip to content

bench: read-only CI database role, commits.branch, develop-only reads - #149

Merged
joseph-isaacs merged 3 commits into
mainfrom
ji/wonderful-dijkstra-fe04i3
Oct 30, 2026
Merged

joseph-isaacs merged 3 commits into
mainfrom
ji/wonderful-dijkstra-fe04i3

Conversation

@joseph-isaacs

Copy link
Copy Markdown
Contributor

Warning

Do not merge until migrations 011 and 012 are applied to production. The web change queries commits.branch, which only exists after 012, and merging deploys the site.

What

Lets vortex-data/vortex CI read the benchmark database to diff its numbers against the published ones, and records which branch each commit's numbers came from.

1. Read-only CI role

  • migrations/011_ci_read_role.sql (requires-superuser) creates the Postgres user bench_ci_read:
    • IAM-auth only, through rds_iam, with no password.
    • SELECT on the six data tables, plus default privileges so future migrator tables are granted too. No access to _applied_migrations.
    • Limits: default_transaction_read_only = on, statement_timeout = 60s, idle_in_transaction_session_timeout = 60s, CONNECTION LIMIT 16.
    • It's separate from bench_read because that user has to keep password auth for Vercel, and on RDS an rds_iam member can't use a password.
  • infra/provision.sh adds GitHubBenchmarkReadRole:
    • rds-db:connect on bench_ci_read only.
    • Trusts repo:vortex-data/vortex:ref:refs/heads/* and repo:vortex-data/vortex:pull_request, so PR benchmark runs can use it. Fork PRs get no OIDC token.
    • --print-trust read prints the policy without touching AWS.

2. commits.branch

  • migrations/012_commit_branch.sql (applied by migrator) adds branch TEXT NOT NULL DEFAULT 'develop' and an index on (branch, timestamp DESC, commit_sha DESC).
  • Every existing row was ingested from develop, the only branch the ingest role trusts, so the default is correct for them.
  • The current writer (scripts/post-ingest.py) doesn't send a branch, so it keeps writing develop rows unchanged.

3. Website reads only develop

  • web/lib/branch.ts sets READ_BRANCH = 'develop'.
  • Every read of commits in queries.ts and summary.ts is filtered by it: commit windows, chart seeds, group rows and summaries.
  • Query-measurement paths that skip the commits join check the row's commit is on develop.
  • The summary fallback keeps its LEFT JOIN, so orphan rows still show (the documented fail-visible behaviour).

Deploy order

  1. As the RDS master (IAM token), from this branch: uv run --no-project scripts/migrate-schema.py apply. This applies 011 and 012.
  2. Merge this PR.
  3. Run infra/provision.sh to create GitHubBenchmarkReadRole.
  4. gh variable set GH_BENCH_READ_ROLE_ARN --repo vortex-data/vortex --body <arn>.

Checks

Check Result
bun run --cwd bench/web test against Postgres 16 746 passed, 6 skipped; DB suites ran
New test: other branches excluded from chart, group and summary reads 4 passed; all 4 fail against the previous queries
tsc --noEmit, bun run biome ci bench clean
infra/tests (trust policies, including the new read role) 4 passed
scripts/tests/test_migrate_schema.py 3 passed
Manual apply in production order on local Postgres 16 migrator refused 011, master applied it, migrator applied 012. bench_ci_read could read; inserts and ledger access were refused

Not verified:

  • The migration test ran against a local Postgres 16 through a small wrapper instead of its Docker container, because Docker wasn't available.
  • Real RDS IAM login and AWS trust.
  • Query plans at production data size.

Generated by Claude Code

joseph-isaacs and others added 2 commits October 30, 2026 11:41
Migration 011 creates bench_ci_read, an IAM-auth Postgres user with SELECT
on the six data tables, read-only transactions, a 60 s statement timeout
and a 16-connection cap. provision.sh adds GitHubBenchmarkReadRole, which
scopes rds-db:connect to that user and trusts vortex-data/vortex on any
branch and on its own pull requests, so benchmark CI can diff its numbers
against the published ones.

Migration 012 adds commits.branch, NOT NULL DEFAULT 'develop'. Every
existing row was ingested from develop, which is the only branch the
ingest role trusts, so the default backfills them correctly, and the
current writer keeps working without sending the column.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UeniBAnpRTzETB7g2Hnzj8
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Every read of commits now goes through a develop-only filter
(web/lib/branch.ts): commit windows, chart seeds, group rows and
summaries. Query-measurement paths that skip the commits join check the
row's commit is on develop. Numbers ingested from another branch can no
longer reach the site.

Requires migration 012 to be applied first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UeniBAnpRTzETB7g2Hnzj8
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@vercel

vercel Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
vortex Ready Ready Preview Oct 30, 2026 12:28pm UTC

Request Review

@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Bench preview for 4424225874b39cb6fcce5d467b917a9f0884e2fb: https://benchmarks-website-neuqs927u-vortex-data.vercel.app

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UeniBAnpRTzETB7g2Hnzj8
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
@github-actions

Copy link
Copy Markdown

🔦 Lighthouse audit

URL Perf A11y Best practices SEO Report
/ 🟢 100 🟢 100 🟢 96 🟢 100 report
/blog 🟢 100 🟢 100 🟢 96 🟢 100 report
/blog/btrblocks-compressor 🟢 100 🟢 96 🟢 96 🟢 100 report

Run 41055600744 · 2026-10-30 12:31 UTC

@joseph-isaacs
joseph-isaacs merged commit 23c2f47 into main Oct 30, 2026
22 checks passed
@joseph-isaacs
joseph-isaacs deleted the ji/wonderful-dijkstra-fe04i3 branch October 30, 2026 12:41

This branch was successfully deployed

2 active deployments
bench-preview — 44242258 Deployed Oct 30, 2026 by joseph-isaacs via Bench: deploy #148
Preview — 44242258 Deployed Oct 30, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants