Skip to content

perf: wake Loom imports from the worker webhook instead of polling - #3120

Merged
richiemcilroy merged 1 commit into
mainfrom
perf/loom-import-hooks
Oct 24, 2026
Merged

richiemcilroy merged 1 commit into
mainfrom
perf/loom-import-hooks

Conversation

@richiemcilroy

@richiemcilroy richiemcilroy commented Oct 24, 2026 •

Copy link
Copy Markdown
Member

Loom import workflows used to poll the database every 5 to 30 seconds until their media job finished. Each poll is a workflow step plus a replay on Vercel, so a ten minute import cost about 25 invocations, and a batch of 1,200 imports cost tens of thousands.

Now each processing attempt creates a workflow hook whose token goes into the worker's webhook URL. When the worker reports complete, error or cancelled, the progress route saves the state first and then resumes the hook. The workflow then reads the status once. A typical import now uses two status reads instead of dozens.

Safety nets:

  • If the signal is lost, the workflow still checks every 5 minutes, within the same 1 hour limit.
  • If the signal arrives before the saved state is visible, it falls back to the old short polling.
  • Tokens only resume hooks that belong to the same video. Unknown or disposed tokens are ignored.
  • Each hook is disposed when its attempt ends.
  • Runs already in flight stay on the deployment that started them.

Recording, instant and web uploads still use waitForVideoProcessing and are unchanged.

Follows #3118. Replaces #3119, which closed when its base branch was deleted.

Tests: video-processing-status 15, media-server-progress 59, media-processing-workflows 27 pass. The 7 share-stage-identity failures also fail on main.

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge; no actionable issues were found.

Summary

Loom imports now wait for a worker webhook instead of repeatedly checking the database.

  • Loom imports wake when their media worker reports a final state.

Reviews (2) · Last reviewed commit: "perf: wake Loom import workflows from th..." · Reviewed by Greptile

@richiemcilroy

Copy link
Copy Markdown
Member Author

@greptileai

@richiemcilroy
richiemcilroy marked this pull request as ready for review October 24, 2026 17:06
@richiemcilroy
richiemcilroy merged commit 4d718aa into main Oct 24, 2026
59 checks passed
@richiemcilroy
richiemcilroy deleted the perf/loom-import-hooks branch October 24, 2026 17:06

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

Open in Capy

This branch was successfully deployed

1 active and 1 inactive deployments
Preview – cap-web — ead7b39e Deployed Oct 24, 2026 by vercel[bot]
Preview – cap-status — ead7b39e Deployed Oct 24, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant