Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
121 changes: 96 additions & 25 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ VapourBox/
| `worker/src/crop_detect.rs` | `--detect-crop`: black borders, blanking, letterbox bars and the head-switching band, for Crop & Resize's Detect (see "Crop Detection"); also the detectors' shared sampling and decode (`decode_samples`) |
| `worker/src/noise_halo.rs` | `--detect-noise-halo`: a source's noise sigma and halo score, for the pass list's suggestions (see "Noise and Halo Scores") |
| `worker/src/audio_restoration.rs` | Audio restoration: probes, the one `AudioPlan` (every `-af` filter, in order), the render before the encode, and the encoder's audio map/codec args, including the NUT sound copy, the AVI copy cut (`SoundCut`, trimmed or a primed head: `sound_cut_applies`) and the encoder priming for Convert into AVI (`EncoderPriming`) (see "Audio Restoration", "Source Frame Size") |
| `Tests/TestResources/audio_codec_containers.json` | Which Convert audio codec each container holds, and the refusal's wording; read by both sides (see "Convert audio: codec × container") |
| `Tests/TestResources/output_capabilities.json` | What each container holds and how it times it: video codec verdicts, Convert and copied audio codecs, the container traits; compiled into the worker (`worker/src/models/output_capabilities.rs`) and generated into the app (`app/lib/models/output_capabilities_table.dart`) (see "Output capabilities: one table") |
| `worker/src/avi_header.rs` | An AVI output's AAC block size raised to its largest packet after the encode (task #41; see "Source Frame Size") |
| `worker/templates/pipeline_template.vpy` | The one VapourSynth script template, for the encode and the preview (see "One script template") |
| `worker/templates/gauss_blur.py` | TCanny's Gaussian blur (`mode=-1`) on `std.Convolution`; never call `tcanny.TCanny` from a template (see "One script template") |
Expand Down Expand Up @@ -2333,29 +2333,78 @@ model (`worker/templates/colorize.py`, on `ai_runtime`). Rules:
and on a colour source), strength 0 = input, both models, and preview = encode.
See docs/ENGINEERING_NOTES.md (2026-11-08) and docs/reports/2026-11-colourisation.

### Convert audio: codec × container (task #39)

The bundled FFmpeg cannot mux Opus into AVI or MOV, nor FLAC into MOV (no
option fixes it; the encode died at "Could not write header"). Rules:

- **One table, two sides**: `AudioCodec::supports_container` /
`container_problem` (Rust, exhaustive match) and
`AudioCodec.supportsContainer` / `containerProblem` (Dart), both pinned to
`Tests/TestResources/audio_codec_containers.json`, message included. The
push-gate `audio_compatibility_test.dart` encodes every codec into every
container with the real binary, both ways; the Convert table must agree
with the Passthrough one (`audio_codec_container_test.dart`).
- **The app offers only what fits**: the Codec dropdown lists
`audioCodecChoices` (the container's codecs, plus a saved unsupported one so
the dropdown keeps its value, shown with the error); the container switch is
`EncodingSettings.withContainer`, which applies `fallbackFor` (lossless →
PCM, lossy → AAC, never lossless → lossy). A preset that still pairs them
gets a "Use <fallback>" dialog before the queue runs.
- **The worker refuses before any work**:
`EncodingSettings::audio_container_problem` in `run_worker`, right after the
config is parsed. Under Convert the restored track is encoded with the
chosen codec, so the one check covers it; under Passthrough it is
`restored_passthrough_codec`, asserted to fit every container.
### Output capabilities: one table (task #49)

`Tests/TestResources/output_capabilities.json` is the one statement of what
each output container holds and how it times it. It widened task #39's Convert
table. Rules:

- **Both sides read the file; nothing restates it.** `worker/build.rs` checks
its shape (a malformed table fails the build) and compiles it in;
`models::output_capabilities` reads it (`ContainerFormat::traits`,
`VideoCodec::container_verdict`, `AudioCodec::supports_container`,
`audio_copy_allowed`). The app reads
`app/lib/models/output_capabilities_table.dart`, **generated and committed**:
after editing the JSON run `dart run tool/generate_output_capabilities.dart`
in `app/` (and don't `dart format` it). `output_capabilities_table_test.dart`
fails when it is stale.
- **Container traits** (`containers`): `storesStreamStart` (AVI: no, so a late
picture, a copy's pre-roll and encoder priming all land at 0 — read by
`stores_stream_start` and so by `output_video_delay`, `needs_sound_cut`,
`may_cut_sound_head`, `needs_priming_shift`), `videoTimeBase` (AVI one tick
per frame), `fieldFlag` (AVI none: `field_order_note`), `audioPacketTiming`
(AVI `blockAlign`: `avi_header::checks_block_align`), `restoredPassthrough`
(`restored_passthrough_codec`, after the editing intermediates' PCM
override), `subtitles` (copy / mov_text / none) and `isoMedia` (faststart,
`hvc1`). **No code outside the table module compares or matches on a
container**: `no_container_is_compared_outside_the_table` (Rust) and its
Dart twin fail on `== ContainerFormat::…` / `case ContainerFormat.…`. A new
container fact is a new column, not a boolean.
- **Video: three verdicts per codec × container** (`video`), measured against
the bundled FFmpeg (docs/reports/task-49-output-conformance): `offered`
(the app offers it: `supportsContainer`), `notOffered` (FFmpeg writes it,
the app keeps it off, the worker runs it with the row's note — the
field-order test needs HuffYUV in MOV), `refused` (HuffYUV, FFVHuff, ProRes
and v210 in MP4 and AV1 in MOV, which FFmpeg can't write; AV1 in AVI, which
it writes with a non-standard tag, `ffmpegWrites: true`). The worker refuses
a refused pair in `run_worker` before any work, with the row's message; the
app's pre-run check offers the codec's own container
(`EncodingSettings.videoContainerProblem`).
- **Audio**: `audioConvert` is #39's table (Opus into AVI/MOV and FLAC into
MOV refused; the Codec dropdown lists `audioCodecChoices`, the container
switch applies `fallbackFor`, a preset that still pairs them gets a "Use
<fallback>" dialog, the worker refuses via `audio_container_problem`).
`audioCopy` is the Passthrough copy table (issue #118), which the worker now
checks too: the source model's audio codecs (`audioStreams`), refused with
`audioCopyMessage` (`audio_copy_problem`, Dart
`AudioCompatibilityService.copyProblem`); a stream Audio Restoration
re-encodes is not checked.
- **Checked against FFmpeg both ways** in the push gate
(`output_capabilities_test.dart`): every video row muxed (hardware encoders
through their software twin, `muxEncoder`), every Convert codec, every
copied codec. The table is self-consistent (`the_table_is_consistent`: the
restored track is a codec its container copies, every codec's preferred
container is offered, the container switch lands on offered pairs).
- **The output conformance matrix** (`integration_output_conformance_test.dart`,
heavy) encodes every container × audio mode (Passthrough, Convert × each
codec the container takes, Restore, Restore + Convert AAC) × source class
(plain MP4, late-video TS, delayed AVI, reset PS through the continuity
copy) × trim (none, from frame 2, from frame 40), plus ProRes/MOV and
AV1/MP4 rows, from `generate_output_conformance.sh`'s flash/click sources:
~440 cases, generated from the enums and the table
(`test/support/output_matrix.dart`, which also holds the oracle; the push
gate's `output_matrix_cases_test.dart` checks the cases). Each checks the
header, picture count and first picture, no null frames, every flash's
click within one audio packet, packet timing, and lossless paths sample-
or byte-exact. **A new container fact or source rule gets a case here, not
another one-cell heavy test.** Cases that fail today are `knownFailures`
(id → task, today all task #58); they still run and fail when they pass.
One fault can't be listed: `pipe_source` intermittently serves an evicted
frame as the last one (#58), so the matrix fails a different case or two
per run until that is fixed. One case:
`flutter test --tags heavy test/integration_output_conformance_test.dart
--plain-name 'avi/convert-aac/delayed-avi/near-start'`. The shared
measuring helpers are `test/support/av_sync.dart`.

### Audio Restoration

Expand Down Expand Up @@ -2423,7 +2472,8 @@ Rules, from docs/reports/2026-11-audio-restoration:
codec and in any container beside an editing intermediate, ProRes or v210
(`restored_passthrough_codec`, which returns its own `RestoredTrackCodec`:
`AudioCodec::Pcm` is the user's Convert choice and stays `pcm_s16le` on the
wire). No restored track is 16-bit. The render is f32; ffmpeg's FLAC from it
wire; the per-container choice is the table's `restoredPassthrough`, see
"Output capabilities"). No restored track is 16-bit. The render is f32; ffmpeg's FLAC from it
is already 24-bit, and the same render as FLAC/MKV and 24-bit PCM in
MOV/MKV/AVI decodes bit-identical. 24-bit PCM in AVI can only be
`WAVE_FORMAT_EXTENSIBLE`; Video for Windows, DirectShow, Media Foundation
Expand Down Expand Up @@ -3015,6 +3065,27 @@ for the full audit and the specific licence/year errors it found.
- `json_annotation` + `json_serializable` for models
- MVVM pattern (models, viewmodels, views, services)

### Line endings: accept both CRLF and LF, everywhere

**Any code that reads a text file from the repo or the user, in the app,
the worker, a test or a script, must work on CRLF (Windows) and LF (macOS
/ Linux) line endings alike.** Windows CI checks the repo out with CRLF,
so a file that is LF on a Mac is CRLF there, and the failure only shows on
Windows. Normalise first (`text.replace("\r\n", "\n")` in Rust,
`replaceAll('\r\n', '\n')` in Dart), or split with `.lines()` /
`LineSplitter`, which accept both. Never:

- search for a multi-line pattern containing `"\n"` in raw file text (task
#49's container lint looked for `"#[cfg(test)]\nmod "` and scanned every
test module on Windows);
- compare a file byte for byte against generated text (task #49's
generated-table freshness check);
- substitute a pattern that spans a line break in a loaded template
(`load_template_by_name` normalises for this; see Debugging Tips #5).

Text the code writes may use `\n`; it is what it reads that must accept
both.

## havsfunc Compatibility Patches

All three `download-deps-*` scripts apply these automatically, and they must stay
Expand Down
128 changes: 0 additions & 128 deletions Tests/TestResources/audio_codec_containers.json

This file was deleted.

Binary file added Tests/TestResources/conf_delayed.avi
Binary file not shown.
Binary file added Tests/TestResources/conf_late_video.ts
Binary file not shown.
Binary file added Tests/TestResources/conf_plain.mp4
Binary file not shown.
Binary file added Tests/TestResources/conf_reset.mpg
Binary file not shown.
Binary file added Tests/TestResources/conf_reset_offgrid.mpg
Binary file not shown.
108 changes: 108 additions & 0 deletions Tests/TestResources/generate_output_conformance.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
#!/bin/bash
# Generate the output conformance matrix's sources (task #49): one content,
# four source classes, so one oracle covers every case
# (app/test/support/output_matrix.dart, integration_output_conformance_test.dart).
#
# Content, identical in all four:
# video testsrc2 (every frame differs), 160x120, 25 fps, 100 pictures (4 s),
# a white frame at pictures 5, 30, 55 and 80
# audio 48 kHz mono, a quiet 200 Hz tone with a 20 ms 1 kHz click at the
# moment each white frame is shown
# 25 fps x 48 kHz: one frame is exactly 1920 samples, so every trim lands on
# a whole sample and the lossless checks can be exact.
#
# conf_plain.mp4 H.264 GOP 25 with B-frames + AAC (primed, edit list);
# sound and picture start together
# conf_late_video.ts H.264 GOP 25 with B-frames + MP2 in MPEG-TS, the video
# 1 s after the sound (a DVB-T capture)
# conf_delayed.avi MPEG-4 ASP with B-frames (Xvid-style: decode-order
# timestamps, decoder delay 1) + MP3
# conf_reset.mpg MPEG-2 + MP2 in MPEG-PS (DVD-style), two parts each
# with its own clock, joined: the clock resets at
# picture 51 (2.04 s: a whole number of MP2 frames, so
# sound and picture end the first part together)
# conf_reset_offgrid.mpg the same, reset at picture 50 (2.0 s: 83.3 MP2
# frames), where the first part's sound ends 16 ms
# after its picture, as a DVD cell's may
#
# Usage: ./generate_output_conformance.sh [output_dir] [ffmpeg]

set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
OUTPUT_DIR="${1:-$SCRIPT_DIR}"
FFMPEG="${2:-ffmpeg}"
FFPROBE="$(dirname "$FFMPEG")/ffprobe"
[ -x "$FFPROBE" ] || FFPROBE=ffprobe

FLASHES="eq(n\,5)+eq(n\,30)+eq(n\,55)+eq(n\,80)"
VIDEO="testsrc2=size=160x120:rate=25:duration=4,drawbox=c=white:t=fill:enable='$FLASHES'"
# The clicks at 0.2, 1.2, 2.2 and 3.2 s on the picture's timeline, shifted by
# the sound's lead ($1 s).
audio() {
local lead="$1" dur="$2" clicks="" t
for t in 0.2 1.2 2.2 3.2; do
local at
at="$(echo "$t + $lead" | bc -l)"
clicks="$clicks+between(t\,$at\,$at+0.02)"
done
echo "aevalsrc=0.05*sin(2*PI*200*t)+0.6*sin(2*PI*1000*t)*(${clicks#+}):s=48000:d=$dur:c=mono"
}

frames_of() {
"$FFPROBE" -v error -select_streams v:0 -count_frames -show_entries stream=nb_read_frames \
-of csv=p=0 "$1" | head -n1
}

# Plain: both streams start at 0.
"$FFMPEG" -y -v error -f lavfi -i "$VIDEO" -f lavfi -i "$(audio 0 4)" \
-map 0:v -map 1:a -pix_fmt yuv420p \
-c:v libx264 -preset veryfast -crf 30 -g 25 -bf 2 -c:a aac -b:a 96k \
"$OUTPUT_DIR/conf_plain.mp4"

# Late video: the picture 1 s after the sound.
"$FFMPEG" -y -v error -itsoffset 1.0 -f lavfi -i "$VIDEO" -f lavfi -i "$(audio 1 5)" \
-map 0:v -map 1:a -pix_fmt yuv420p \
-c:v libx264 -preset veryfast -crf 30 -g 25 -bf 2 -c:a mp2 -b:a 96k \
-f mpegts "$OUTPUT_DIR/conf_late_video.ts"

# Delayed AVI: decode-order timestamps.
"$FFMPEG" -y -v error -f lavfi -i "$VIDEO" -f lavfi -i "$(audio 0 4)" \
-map 0:v -map 1:a -pix_fmt yuv420p \
-c:v mpeg4 -vtag XVID -bf 2 -g 25 -q:v 4 -c:a libmp3lame -b:a 96k \
"$OUTPUT_DIR/conf_delayed.avi"

# Reset: pictures 0..at-1 and at..99 as two program streams, each from 0.
tmp="$(mktemp -d)"
trap 'rm -rf "$tmp"' EXIT
reset_at() {
local at="$1" out="$2" part first count
for part in 0 1; do
first=$((part * at)); count=$(( part == 0 ? at : 100 - at ))
"$FFMPEG" -y -v error \
-f lavfi -i "$VIDEO,trim=start_frame=$first:end_frame=$((first + count)),setpts=PTS-STARTPTS" \
-f lavfi -i "$(audio 0 4),atrim=start=$(echo "$first / 25" | bc -l):duration=$(echo "$count / 25" | bc -l),asetpts=PTS-STARTPTS" \
-map 0:v -map 1:a -pix_fmt yuv420p \
-c:v mpeg2video -b:v 400k -g 12 -bf 2 -flags -cgop -sc_threshold 1000000000 \
-c:a mp2 -b:a 96k -f vob "$tmp/$part.mpg"
done
cat "$tmp/0.mpg" "$tmp/1.mpg" > "$OUTPUT_DIR/$out"
}
reset_at 51 conf_reset.mpg
reset_at 50 conf_reset_offgrid.mpg

for f in conf_plain.mp4 conf_late_video.ts conf_delayed.avi; do
n="$(frames_of "$OUTPUT_DIR/$f")"
[ "$n" = 100 ] || { echo "$f: expected 100 pictures, got $n" >&2; exit 1; }
done
vstart="$("$FFPROBE" -v error -select_streams v:0 -show_entries stream=start_time -of csv=p=0 "$OUTPUT_DIR/conf_late_video.ts" | head -n1)"
astart="$("$FFPROBE" -v error -select_streams a:0 -show_entries stream=start_time -of csv=p=0 "$OUTPUT_DIR/conf_late_video.ts" | head -n1)"
echo "conf_late_video.ts: video starts at $vstart, sound at $astart"
b="$("$FFPROBE" -v error -select_streams v:0 -show_entries stream=has_b_frames -of csv=p=0 "$OUTPUT_DIR/conf_delayed.avi")"
[ "$b" -gt 0 ] || { echo "conf_delayed.avi: no B-frames" >&2; exit 1; }
for f in conf_reset.mpg conf_reset_offgrid.mpg; do
seq="$("$FFMPEG" -v error -i "$OUTPUT_DIR/$f" -map 0:v -f null - -progress pipe:1 \
| awk -F= '$1 == "frame" { f = $2 } END { print f }')"
[ "$seq" = 100 ] || { echo "$f: expected 100 pictures in a sequential decode, got $seq" >&2; exit 1; }
done
echo "Wrote conf_plain.mp4, conf_late_video.ts, conf_delayed.avi, conf_reset.mpg, conf_reset_offgrid.mpg to $OUTPUT_DIR"
Loading
Loading