Skip to content

One source model owned by the worker (--probe-source) (task #50) - #207

Merged
StuartCameronCode merged 4 commits into
mainfrom
stuart/50-one-source-model-owned-by-the-worker-pro
Nov 16, 2026
Merged

StuartCameronCode merged 4 commits into
mainfrom
stuart/50-one-source-model-owned-by-the-worker-pro

Conversation

@StuartCameronCode

Copy link
Copy Markdown
Owner

Summary

Every fact about a source (size, rate, frame count, video start, decoder delay, seek plan, grid, VFR times, colour tags, Dolby Vision, audio streams, timestamp-reset scan) is now computed once, by the worker. This covers R3a and R5 from task #42's plan.

  • vapourbox-worker --probe-source <path> prints a SourceModel (worker/src/source_model.rs). It reuses the existing pure rules (video_start_from_probe, decoder_delay_from_probe, seek_plan_from_probe, grid_from_probe, the continuity scan, the VFR timeline). The independent probes run in parallel.
  • Jobs carry it (VideoJob.source inline, or --source <file> for previews). One resolve_source replaces the five fill_* probes. It fills only the fields a job left unset, so every decoder's arguments stay byte-for-byte the same. A model whose file has changed (path, size or mtime differ) is probed again rather than trusted. A job without a frame count is still counted by decoding.
  • The app stores the model on QueueItem.sourceModel at analysis and reads every source fact from it (app/lib/models/source_model.dart). This removes the Dart copies of the probe rules: video start, decoder delay, seek-plan and grid values, the frame-count derivations, the rate choice, SAR and colour-tag reading. FieldOrderDetector keeps only idet and soft-telecine detection, which look at pictures.
  • R5: docs/TWINS.md lists every remaining Rust↔Dart twin with its disposition. twin_registry_test.dart fails when an app file declares a twin that isn't registered there.
  • Review fixes: the model's grid is re-rated to a job's own inputFrameRate when the two differ, as grid_from_probe did. The preview writes one model file at a time.
  • Report: docs/reports/task-50-source-model/ covers frame-count parity between the app's old rules and the model, and analysis/preview timing. CLAUDE.md, README and ENGINEERING_NOTES are updated.

Not in this PR: R3b. That means --source-frame / --thumbnails, and deleting the Dart SeekPlan/SourceGrid argument builders, decodeNeutraliser and the HDR display chain. It is waiting on a go-ahead and a thumbnail-latency measurement. TWINS.md marks those rows "Delete with R3b", and they stay pinned by their vector files until then.

Behaviour change: if the worker cannot read a file, its queue item gets no video info or thumbnails, where the app's own ffprobe call used to fill them. The log says why.

Testing

  • cargo test: 1432 passed, 1 failed. The failure is the Whisper subtitle test, which needs the add-on (not installed locally). New unit tests cover the count rules, the model round-trip, the VFR timeline and apply_source_model.
  • flutter test --exclude-tags heavy,gpu: 1603 passed.
  • Heavy suites (seek alignment, source grid, video-start offset, VFR, reserved colour tags, HDR, frame mapping, aspect, audio restoration, noise/halo, crop detect): 257 passed and 2 timed out under load from concurrent timing scripts. Both pass on reruns.
  • flutter analyze: no new warnings.

🤖 Generated with Claude Code

StuartCameronCode and others added 4 commits November 16, 2026 17:36
…) (task #50)

SourceModel holds every fact the decoders need about a source: size, rate,
frame count, video start offset, decoder delay, seek plan, grid, VFR
timeline times, colour tags, Dolby Vision, audio streams and the
timestamp-reset scan. --probe-source prints it; a job carries it inline
or by file (--source). resolve_source replaces the five fill_* probes and
fills only fields the job left unset, so every decoder argument list is
unchanged (checked: identical preview plan and PNG on every fixture at
five frames, before and after). A VFR job's timeline comes from the
model's stored times, so a preview no longer reads every packet.

The app's frame-count rules are ported (frame_count): the model's count
equals the app's on all 30 fixtures. The -count_frames fallback a job
without totalFrames used is gone: on interlaced_test.avi it said 75 where
the grid decode produces the 79 the header stores.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first commit filled a job's missing total_frames from the source
model. Against what the worker's own decode produces, the model is right
except on an MP4 whose edit list discards pre-roll (small_clip.mp4: 3154
vs 3151), where -count_frames is right; two heavy AVI tests that send no
count caught it. Until the model's count handles edit lists, a job
without a count (CLI, tests; the app always sends one) is counted by
decoding again, in the encode and --audio-plan only, as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he remaining twins (task #50)

The app probes a source once, with --probe-source, at analysis
(QueueItem.sourceModel; the reset scan comes with it, and a joined copy
is probed again). VideoInfo, the preview generator and every job (encode,
detectors inline; previews by --source file) take their facts from it,
and the model goes back to the worker exactly as it came. Deleted: the
Dart probe rules for the video start, decoder delay, seek plan, grid,
frame count, rate choice, VFR test, SAR and colour tags, the preview
generator's own probe, and the metadata field-order ffprobe call.
SeekPlan/SourceGrid keep only their argument builders until the Original
and thumbnails go through the worker (R3b, not built).

docs/TWINS.md lists every remaining Rust<->Dart twin with its disposition;
twin_registry_test.dart fails on an unregistered one. Report:
docs/reports/task-50-source-model (no decode changed on 30 fixtures; the
model equals the app's old values; app previews 89-182 ms faster; frame
count findings).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…task #50 review)

apply_source_model re-rates the model's grid to the job's inputFrameRate
when they differ beyond GRID_RATE_TOLERANCE, as grid_from_probe did for a
job carrying its own rate. PreviewGenerator no longer writes a new model
file on every selection and deletes the one it replaces. Unit tests for
apply_source_model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@StuartCameronCode
StuartCameronCode force-pushed the stuart/50-one-source-model-owned-by-the-worker-pro branch from 8b4b9eb to 959ec93 Compare November 16, 2026 06:43
@StuartCameronCode
StuartCameronCode merged commit 4cdb0e2 into main Nov 16, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant