Skip to content

feat(projects): enforce Project membership and retire the connector - #8590

Draft
mzxchandra wants to merge 96 commits into
feat/project-workspace-column-expandfrom
codex/project-entity-enforcement
Draft

mzxchandra wants to merge 96 commits into
feat/project-workspace-column-expandfrom
codex/project-entity-enforcement

Conversation

@mzxchandra

@mzxchandra mzxchandra commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Complete the migration to workspace.project_id after #8830 is deployed and incompatible membership activity has drained. Release-2 migrations run before application promotion, while release 1 serves traffic.

  • The existing registered 0031_project_membership runner switches the checked singleton to column authority under the workspace barrier, then performs bounded, resumable assignment and reconciliation. SQL 0407 remains a placeholder because SQL migrations precede script migrations.
  • Preserve existing Project identity, archive state and ownership; stop for ambiguous grouping or incomplete provider cleanup. Retain reviewed operator repair and resume paths.
  • Validate membership, enforce workspace.project_id NOT NULL and its restrictive FK, then atomically remove both project_workspace and project_membership_rollout. The authority-aware feat(projects): move Project membership to the workspace column #8830 runtime remains the supported rollback version after contraction.

Permanent enforcement

Replace all custom Project integrity triggers, trigger-specific advisory locks, deferred row-version dispatch, and UPDATE project SET updated_at = updated_at with native constraints:

  • A generated organization scope key on Project/workspace encodes NULL as personal and IDs as organization:<id>. The expression always yields a value, cannot be overridden by writers, and supports a composite organization-consistency FK without the nullable-FK loophole.
  • A composite workspace self-FK requires each connected fork to share its parent's Project. The existing parent-ID FK still clears only the parent pointer on deletion; Project membership never cascades automatically.
  • Both composite FKs are deferred so atomic organization transfers and subtree moves can complete before validation. Their supporting unique indexes are built concurrently and attached as named unique constraints. Fresh schema push uses the same finalizer to restore deferred timing, which Drizzle does not model.

Project non-emptiness and archive lifecycle remain application-owned. Keep existing application Project/lineage/edge/workspace locks and transactional workflow admission. Backfill still validates legacy lifecycle state once; it does not install permanent lifecycle triggers. Keep project.updated_at as ordinary metadata, and exclude the new internal scope key from Project presentation.

Migration trade-off and rollback

On PostgreSQL 16/17, adding stored generated columns rewrites the Project/workspace tables. This phase refuses busy tables and bounds each statement to five seconds; a timeout rolls it back before connector removal. This is a bounded blocking operation, not a zero-interruption column addition. Assess table size/capacity before deployment; do not silently extend its lock budget. Interrupted concurrent unique-index builds are repaired on replay.

Use the existing all-at-once traffic cutover and verify the concrete old membership operations/workers have drained. No new maintenance mechanism, dual writes, synchronization triggers, extra compatibility release, or Project-specific PostgreSQL16 CI is introduced. Once authority switches, keep column authority on retries/rollback and never deploy pre-#8830 code. Updated #8830 recognizes the completed schema without the marker, so rollback requires no retained rollout table. Fresh schema push creates no marker; migration replay recognizes completed contraction even when recording its receipt previously failed.

Validation

Focused local checks only; full CI is delegated to GitHub Actions.

Current marker-removal revision: 17 real PostgreSQL checks passed for authority switching, missing-marker refusal, contraction cleanup, failed-receipt replay, and fresh push/replay. The application creation/disconnect/organization-deletion/archive integration case also passed against the migrated database without the marker. DB package type-check, SQL migration safety, schema mock generation, and schema drift checks passed.

Earlier validation for the native-constraint implementation (before marker removal):

  • 65 PostgreSQL17 migration/constraint/push checks passed, including replay, invalid-index recovery, null scope transitions, immediate/deferred validation, parent deletion, subtree moves, stale snapshots, and absence of custom triggers/Project row touches.
  • 11 targeted PostgreSQL16 checks passed for the native constraints and replay behavior.
  • 27 application integration checks passed with native constraints, covering creation rollback, concurrent last-environment archival, workflow restoration versus archival, organization transfers and account teardown. The final creation/disconnect/organization-deletion/archive case was rerun with an assertion that the internal scope key is not exposed.
  • Actual local HTTP proof: connector-only create/fork through feat(projects): move Project membership to the workspace column #8830; the real migration runner while the same server remains running; preserved assignments and completion receipt; nine post-contraction smoke checks on each of the feat(projects): move Project membership to the workspace column #8830 and feat(projects): enforce Project membership and retire the connector #8590 dev servers. Real account-deletion preview/POST and complete teardown passed on both servers.
  • DB package type-check, affected-file Biome, API validation audit, migration and maintenance SQL safety checks, generated schema mocks, and schema-generation drift check passed.

Local HTTP proof does not establish production drain completion or external-provider cleanup. GitHub Actions results for the new heads are separate from these local results. Downstream #8609/#8610 still need synchronization; the shared decision document records superseded trigger/lifecycle/CI requirements.

Current hosted status

Head 204e5f1fac fixes repair lookups that selected generated columns before contraction, updates the newly merged inbox fixture to create mandatory Project membership, bounds short DDL separately from longer scans/index builds, and flushes child stdout before exit. All 9 operator-repair integration cases, 5 inbox integration cases, and 3 focused database cases passed locally. The two previously failing detach-repair cases were reproduced before the fix. The generated-column ordering review finding did not reproduce with the unchanged fresh-push fixture; its thread includes the passing database evidence.

Both PRs are mergeable. CI and both reviewers are running on this head; no clean hosted result is claimed: https://github.com/simstudioai/sim/actions/runs/38085328277.

Latest archive-result correction

Head 88977f3931 restores Project name to the explicit lookup used by workspace archival. The existing concurrent archive integration case now checks the returned Project identity and name: it fails before the correction and passes afterward. Both pre-contraction reviewed-detach repair cases still pass (3 focused database-backed cases total). CI and both reviews have restarted; results are pending.

@vercel

vercel Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Oct 10, 2026 8:57pm UTC

Request Review

@mzxchandra mzxchandra changed the title feat(projects): enforce membership after the staged backfill feat(projects): backfill and enforce membership in SQL Oct 3, 2026
@mzxchandra

Copy link
Copy Markdown
Contributor Author

@greptile Please re-review the current head and the individual thread evidence. Downstream import adaptations are explicitly recorded as required follow-up outside these PRs; no downstream completion is claimed.

@mzxchandra

Copy link
Copy Markdown
Contributor Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 10, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@mzxchandra I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 171 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.

You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic

…t-entity-enforcement

# Conflicts:
#	apps/sim/lib/billing/organizations/lock-order.test.ts
#	apps/sim/lib/projects/__integration__/foundation.integration.ts
#	apps/sim/lib/projects/environment-source.ts
@mzxchandra

Copy link
Copy Markdown
Contributor Author

@greptile Please review current head d2f0893, including atomic removal of both project_workspace and project_membership_rollout, completed-schema detection for retries and fresh push, and compatibility with the updated #8830 rollback release.

@mzxchandra

Copy link
Copy Markdown
Contributor Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 10, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@mzxchandra I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 172 files

Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.

You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic

Comment thread packages/db/scripts/push.integration.ts
Comment thread packages/db/maintenance/project-membership.sql
Comment thread packages/db/scripts/project-contract.integration.ts
@mzxchandra

Copy link
Copy Markdown
Contributor Author

@greptile Please re-review head 204e5f1. Repair queries now select only pre-contraction fields, the inbox fixture supplies mandatory Project membership, DDL timeout scopes are explicit, and child stdout is flushed. All 14 affected application integration tests and three focused database tests pass locally.

@mzxchandra

Copy link
Copy Markdown
Contributor Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 10, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@mzxchandra I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 173 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.

You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic

Comment thread apps/sim/lib/projects/membership.ts
@mzxchandra

Copy link
Copy Markdown
Contributor Author

@greptile Please re-review head 88977f3. The Project name is restored in the explicit projection. The archive result regression was reproduced before the fix; it and both pre-contraction repair cases pass afterward.

@mzxchandra

Copy link
Copy Markdown
Contributor Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 10, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@mzxchandra I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 173 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.

You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.

Turn on auto-fix | Re-trigger cubic

This branch was previously deployed

1 inactive deployment
Preview — 88977f39 Deployed Oct 10, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant