Repository navigation
fix(db): isolate billing and auth connection pools, release the redeploy-check transaction early, cache integration tool schemas - #9347
Conversation
postgres.js pipelines new queries onto busy connections when none is idle, so slow reads saturating the process pool made short usage-ledger transactions wait seconds for BEGIN until Postgres killed them. - Add `billing` (4) and `auth` (3) dbFor roles, with DATABASE_URL_BILLING and DATABASE_URL_AUTH overrides falling back to the process URL. - Billing pool: recordCumulativeUsage, recordUsage without a caller tx, the execution logger's usage reconciliation, both threshold settlement transactions, and recordJobRunSpend. Each moved transaction only queries through its tx handle; readers stay on the process pool. - Auth pool: getAuthPool() backs the Better Auth adapter, getAuthDatabase() outside a transaction, the OAuth adapter guard defaults, and the API-key hash lookup. - isPrimaryPoolClient() in @sim/db replaces the `executor === db` checks in the security, session, and SSO policy caches, so reads through the auth pool keep hitting them instead of costing two queries per cached session. - Integration test saturates the process pool and requires the billing write, an adapter session read, and an API-key lookup to finish anyway.
…ion commits checkNeedsRedeployment ran the full workflow diff inside its REPEATABLE READ transaction, holding a pooled connection idle in transaction during CPU work. Reads (active version, normalized draft, variables, credential migrations) stay in the snapshot; the pure diff now runs after commit.
…ead of per user every 5s The integration catalog cached each viewer's full schema set for 5s, so nearly every call rebuilt a user-tool schema for all ~5.3k exposed tools, sized the ~7.7MB result with JSON.stringify, structuredClone'd it, and re-tokenized every tool for BM25 - all synchronously. The schemas depend only on the registry and deployment config, so they are now built once per (surface, personal mode), frozen, and shared; the build yields between tools. Each request still applies the viewer's own block visibility and permission policy, and the plan-dependent email tagline is read per user with the same 5s TTL. Results are byte-identical. BM25 term counts are memoized per shared schema object.
Assert each frozen-schema mutation throws, rank BM25 by query terms instead of copying each tool's vocabulary, gate the reporting usage cache on isPrimaryPoolClient like the auth caches, and trim comments.
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
All reported issues were addressed across 31 files
Heads up: you’re close to your flex budget. Increase your flex budget so reviews don’t pause.
Turn on auto-fix | Re-trigger cubic
 [High impact] The PR appears safe to merge; no blocking issues were found. SummaryThe PR separates billing and auth database pools, moves workflow comparisons outside the database transaction, and shares frozen integration tool schemas.
Reviews (3) · Last reviewed commit: "test(db): close every pool the isolation..." · Reviewed by Greptile |
Give the memory harness integration mock a dbFor so the billing pool resolves.
|
@cubic-dev-ai review this PR |
@waleedlatif1 I have started the AI code review. It will take a few minutes to complete. |
There was a problem hiding this comment.
All reported issues were addressed across 33 files
Heads up: you’re close to your flex budget. Increase your flex budget so reviews don’t pause.
Turn on auto-fix | Re-trigger cubic
|
@cubic-dev-ai review this PR |
@waleedlatif1 I have started the AI code review. It will take a few minutes to complete. |
There was a problem hiding this comment.
No issues found across 33 files
Confidence score: 5/5
- Automated review surfaced no issues in the provided summaries.
- No files require special attention.
Heads up: you’re close to your flex budget. Increase your flex budget so reviews don’t pause.
Turn on auto-fix | Re-trigger cubic
Summary
dbFor('billing'), 4 connections;dbFor('auth'), 3). When slow work saturates a process's 10-connection pool, postgres.js pipelines new queries onto busy connections, so a billing transaction'sBEGINcould wait behind a large result and be killed by its 4stransaction_timeout. Settlement transactions (recordCumulativeUsage,recordUsagefallback, threshold billing, the execution-logger usage reconcile, job spend) and the auth adapter,getAuthDatabase()fallback and API-key lookup now run on their own clients. App-wide billing reads stay ondbso they don't crowd a small poolisPrimaryPoolClient()in@sim/dbreplaces theexecutor === dbchecks that gate the auth policy caches and the reporting usage cache, so those caches still apply on a sub-pool client and never on a transaction or the replicaDATABASE_URL_BILLING/DATABASE_URL_AUTHare optional; unset, the pools fall back to the process URL, which already gives the client-side isolationcheckNeedsRedeploymentkeeps every read in its REPEATABLE READ snapshot but runs the state diff after commit, so the connection is no longer held idle in transaction during the comparisonType of Change
Testing
lib/db/pool-isolation.integration.ts: fills every connection of the realdbpool, then requires a cumulative usage write, an auth adapter session read and an API-key lookup to finish within 1.5s. Fails on the pre-fix code (all three stall), passes with the fixpool-isolation,usage-log,reporting-usage-cache,get-organization-plan-seats(57 tests) against disposable Postgresdeployment-status.test.ts(diff no longer runs while the transaction is open; red pre-fix),db.test.ts, auth cache-identity case insecurity-policy.test.ts(red withexecutor === db), payload tests (one build per process, cross-viewer isolation, tagline per viewer; red pre-fix), plus 54 neighbouring files (635 tests)check:tool-registry-boundary,check:schema-mock,check:test-patterns,check:utils,check:comment-hygiene,check:unused-exports,check:file-names; generators in sync. Type-check and full audits run in CIChecklist
test-auditauthoring gate)