[richiemcilroy]

Instant-mode live transcription sends each 5 to 10 s chunk on its own, so the first words of every chunk start mid-sentence and the last word is often cut in half. Those chunks are promoted to the final transcript, so the damage at every cut ends up in the video's transcript.

This adds opt-in left context: each chunk is sent with the already-transcribed segments just before it, and the next chunk re-transcribes the last 2 s before its cut, where it hears the cut words whole.

sequenceDiagram
    participant W as liveTranscribeWorkflow
    participant P as transcription API
    Note over W: chunk N = segments 11..14, context = segments 4..10 (up to N s)
    W->>P: init + context + chunk audio
    P-->>W: words for all of it (ms from audio start)
    Note over W: offsetChunkWords places all words on the<br/>recording timeline (context words before the cut)
    Note over W: applyChunkToLiveTranscript joins at a word both<br/>chunks heard near (cut - 2 s), else cuts there

Loading

• planNextLiveChunk (apps/web/lib/live-transcribe-core.ts:74) returns context / contextMs: the trailing already-processed segments until contextSeconds is reached. The first chunk has none.
• offsetChunkWords takes contextMs and places every word on the recording timeline; context words land before the chunk start instead of being clamped to it.
• applyChunkToLiveTranscript joins the previous words to the new chunk about 2 s before the cut (joinAtHandover), so the new chunk replaces the previous chunk's cut-off tail:
  • the two calls time the same word slightly differently, so the join lands on a word both heard (same text, start within 300 ms, searched within 1 s of the handover point);
  • repeated words ("no, no, no") pick the copy whose surrounding words line up (alignedRun), then the closest start;
  • if the chunks share no word there, it cuts at the handover point. Still idempotent on retry.
• chunkHandoverMs never hands over more than the context actually sent, so a short context can't leave a hole.
• orderLiveTranscriptWords keeps word starts non-decreasing and gives every word at least 1 ms inside the recording, which parseEditTranscript requires. A join can reorder starts by up to 300 ms. A word clamped to a chunk's end is zero-length, and that already happens without context.
• promoteLiveTranscript parses the canonical transcript before claiming the video and falls back to the full post-stop pass if it would not read back.
• The workflow reads LIVE_TRANSCRIBE_CONTEXT_SECONDS (apps/web/workflows/live-transcribe.ts:548, clamped to 0 to 60). Unset or 0 sends exactly the audio it sends today. A retried chunk is sent without context in case the earlier audio is what failed.

Context multiplies the audio sent per chunk (about 2.7x with 20 s), which is why it is off by default.

Measured

A 5 minute public-domain LibriVox reading, cut into 3 s fMP4 AAC segments, replayed through planNextLiveChunk / offsetChunkWords / applyChunkToLiveTranscript one segment at a time as a recording in progress, transcribed by AssemblyAI. Compared with AssemblyAI's own full-file transcript of the same audio:

	context 0 (today)	context 20 s
Word difference from the full-file transcript	5.56%	1.81%
Words inserted at chunk joins (duplicates)	9	0
Word start offset from full-file, p90 / max	69 / 1691 ms	65 / 238 ms
Trailing 10 ms segment as the final chunk	Audio duration is too short	transcribed

A failed final chunk is retried, then skipped as a gap, which blocks promotion and falls back to the full post-stop pass. With context the tail chunk carries enough audio to be accepted.

Confidence Score: 5/5

The PR appears safe to merge; no new blocking issues were found.

Summary

Adds optional earlier audio to live transcription chunks and joins overlapping words to improve chunk boundaries.

• Live chunks can use earlier audio to replace words near their cuts.
• Live transcript word times stay readable when chunks are joined.

_{Reviews (7) · Last reviewed commit: "fix(web): keep promoted live transcripts..." · Reviewed by Greptile}

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated
cap-status	Skipped		Oct 28, 2026 11:51pm UTC

[richiemcilroy]

@greptileai review

[richiemcilroy]

@greptileai review

[richiemcilroy]

@greptileai review

[richiemcilroy]

@greptileai review

[capy-ai]

Capy found 1 potential issue (1 medium).

View 1 other finding in Capy.

[richiemcilroy]

@greptileai review

[richiemcilroy]

@greptileai review

[capy-ai]

Capy found no issues.