Skip to content

One output capability table; the output conformance matrix (task #49) - #211

Merged
StuartCameronCode merged 5 commits into
mainfrom
stuart/49-output-conformance-matrix-container-audi
Nov 16, 2026
Merged

StuartCameronCode merged 5 commits into
mainfrom
stuart/49-output-conformance-matrix-container-audi

Conversation

@StuartCameronCode

Copy link
Copy Markdown
Owner

Task #49, from task #42's plan (R4). Tasks #22, #24, #30, #38, #39, #41 and #43 each found one cell of the same matrix: sound against picture, per container and audio mode. This PR makes that matrix one table and one generated test.

The table: Tests/TestResources/output_capabilities.json

It widens task #39's Convert table into one statement of what each output container holds and how it times it:

  • Video codec × container verdicts: offered, notOffered or refused, measured against the bundled FFmpeg. FFmpeg can't write HuffYUV, FFVHuff, ProRes or v210 into MP4, nor AV1 into MOV. AV1 into AVI writes a non-standard AV01 tag, so it is refused too.
  • Convert codecs: Monterey 12.7.6 vspipe-bin Symbol not found #39's table.
  • The Passthrough copy table: issue MP2 audio is not supported in any container #118's. The worker now refuses a copy the container can't hold, before any work, in the same words as the app.
  • Container traits: storesStreamStart, videoTimeBase, fieldFlag, audioPacketTiming, restoredPassthrough, subtitles, isoMedia.

Both sides read it:

  • Worker: build.rs checks the table's shape and compiles it in; models::output_capabilities reads it.
  • App: reads a generated, committed twin (app/lib/models/output_capabilities_table.dart, regenerated by dart run tool/generate_output_capabilities.dart, with a staleness test).

The scattered booleans are now reads of the table: stores_stream_start, restored_passthrough_codec, checks_block_align, field_order_note, the subtitle mapping, faststart/hvc1, allows_container, supportsContainer and storesStreamStart. A lint on each side fails on any new == ContainerFormat::… comparison. output_capabilities_test.dart (push gate) still checks every row against the bundled FFmpeg, both ways.

The app also gets a pre-run dialog for a refused video pair (only a preset or an imported job can still make one). It offers the codec's own container.

The matrix: integration_output_conformance_test.dart (heavy)

444 cases, generated from the enums and the table (test/support/output_matrix.dart):

  • Containers: every one.
  • Audio modes: Passthrough, Convert × each codec the container takes, Restore, Restore + Convert AAC.
  • Source classes: plain MP4, late-video TS, delayed AVI, and a reset PS read through the app's joined copy.
  • Trims: none, from frame 2, from frame 40.
  • Extra rows: ProRes/MOV and AV1/MP4.

The sources come from generate_output_conformance.sh: one flash/click content in four containers.

Each case checks:

  1. the header writes, with the expected audio codec;
  2. the picture count and the first picture's time;
  3. no null frames;
  4. each flash's click within one audio packet;
  5. audio packet timing, plus ci: bump actions to Node 24 majors (@v5) #41's block size in AVI;
  6. lossless paths are lossless: a copy is byte-exact, FLAC/PCM sample-exact.

Run one case with:
flutter test --tags heavy test/integration_output_conformance_test.dart --plain-name 'avi/convert-libmp3lame/delayed-avi/near-start'

What it found (task #58)

On macOS arm64, 425 of 446 pass.

Finding Cases Every run?
Restore + Convert AAC into AVI, trimmed: 49 pictures for 50 8 yes
Restore of a seeked MP2 source: up to 2⁻¹⁵ off the source decode (Convert to FLAC/PCM of the same seek is exact) 10 yes
Delayed AVI's MP3 copied into AVI from picture 2: click 31 ms late 1 yes
pipe_source returns an evicted frame as the last one (pictures 3–7 came out as picture 99) 1–3 no, varies
A reset whose sound ends off the picture grid joins 16 ms late 1 test yes

The 19 cases that fail every run are knownFailures: they still run, and fail the suite if they start passing. The pipe_source fault can't be listed, so the heavy matrix fails intermittently until #58 fixes it. It silently writes wrong pictures into users' files, so it is #58's priority.

Review fixes (in this PR)

The first full run failed 109 cases, mostly because the matrix's checks were wrong:

  • The sample check aligned on the first match of a repeating tone, so trimmed outputs were compared a whole tone period off. It now keeps the best whole-output alignment.
  • AVI's first audio chunk has no duration, which the test read as 0.
  • ffprobe names MP2 in MP4 mp3 or mp2; either is now accepted.

The review also added the missing report (docs/reports/task-49-output-conformance/) and moved a doc comment back to its own function.

The branch merges main. The two conflicts were the pipeline_executor.rs imports and two appended ENGINEERING_NOTES entries, both resolved by keeping both sides.

Tests

  • cargo test: all pass except subtitle_integration_test, which needs the whisper add-on and this worktree doesn't have it.
  • flutter test --exclude-tags heavy,gpu: 1,664 passed, 7 skipped, 0 failed. Run after the merge with main.
  • Heavy matrix: three full runs plus a targeted run of the known-failure set, all on macOS arm64. Other platforms will first run it in nightly.

Report: docs/reports/task-49-output-conformance/index.html.

🤖 Generated with Claude Code

@sanchandhana-commits

Copy link
Copy Markdown

Excellent work! This is a very important improvement for overall export reliability, audio-video synchronization, and container compatibility. The comprehensive testing and attention to these edge cases are impressive.

One thing I’d love to see in future updates is more noticeable improvements to the actual video processing quality, especially AI RIFE frame interpolation, temporal and spatial noise reduction, and natural sharpening.

Currently, these features don’t seem to deliver a significant improvement in real-world results. I hope future updates focus not only on export reliability but also on making the AI processing quality noticeably better, especially for 4K HDR and Dolby Vision content.

Keep up the great work!

StuartCameronCode and others added 3 commits November 17, 2026 01:11
…ision (task #49)

Tests/TestResources/output_capabilities.json widens task #39's audio table:
video codec x container verdicts (offered / notOffered / refused, measured
against the bundled FFmpeg), the Passthrough copy table and each container's
traits. build.rs checks and compiles it in; models::output_capabilities reads
it. stores_stream_start, restored_passthrough_codec, field_order_note, the
subtitle form, faststart/hvc1 and the AVI block-align fix now read it, and a
lint test keeps container comparisons out of the rest of the worker.

The worker now refuses every pair FFmpeg can't write (and AV1 in AVI) before
any work, notes pairs the app doesn't offer, and refuses a Passthrough copy
the container can't hold.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… conformance matrix (task #49)

The app's video and audio container checks, storesStreamStart and the
Passthrough copy table now read app/lib/models/output_capabilities_table.dart,
generated from the same JSON the worker compiles in (tool/
generate_output_capabilities.dart; a freshness test fails when it is stale).
A lint keeps container comparisons out of the rest of lib/. The pre-run
check offers the codec's own container for a pair the worker refuses.

output_capabilities_test.dart (was audio_compatibility_test.dart) now also
muxes every video codec x container row with the bundled FFmpeg, both ways.

integration_output_conformance_test.dart (heavy) encodes every container x
audio mode x source class x trim from generate_output_conformance.sh's
flash/click sources and checks header, pictures, first picture, null frames,
flash-to-click sync, packet timing and lossless paths; the cases and oracle
are in test/support/output_matrix.dart (checked in the push gate). The sync
helpers move to test/support/av_sync.dart.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the report (task #49 review)

The first full run of the output conformance matrix failed 109 cases, most of
them the oracle's:
- the sample check aligned on the first offset whose first 4800 samples
  matched; the tone repeats, so trimmed outputs were compared a whole period
  off. It now compares every matching offset in full and keeps the best.
- AVI stores no duration for its first audio chunk (nor any FLAC chunk);
  read as 0, packet 1 looked one packet late. Use the stream's usual one.
- ffprobe names MP2 in MP4 mp3 or mp2; accept either for MP4.

What is left (425/446 pass) is filed as task #58: the 19 deterministic
failures are knownFailures; pipe_source's intermittent wrong frames cannot be.

Also: the report's index.html, its README row and ENGINEERING_NOTES entry
(required by the reports rules and cited by CLAUDE.md). On rebasing onto
task #50, the worker's copy check reads the source model's audio streams
instead of its own ffprobe, and the matrix sends each job its source model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@StuartCameronCode
StuartCameronCode force-pushed the stuart/49-output-conformance-matrix-container-audi branch from 02613d8 to 01a37a1 Compare November 16, 2026 14:26
StuartCameronCode and others added 2 commits November 17, 2026 02:14
… checkout (task #49)

Windows CI checks out with CRLF: the Rust lint's search for "#[cfg(test)]\nmod"
never matched, so it scanned every file's tests and flagged their own
container comparisons. The Dart freshness check would have compared LF output
with the CRLF committed file. Both normalise line endings first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#49)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@StuartCameronCode
StuartCameronCode merged commit dc9e36c into main Nov 16, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants