Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 24 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ VapourBox/
|------|---------|
| `worker/src/models/video_job.rs` | Job config, EncodingSettings, processing passes |
| `worker/src/models/qtgmc_parameters.rs` | All 70+ QTGMC parameters (serde) |
| `worker/src/models/processing_pipeline.rs` | Pipeline passes + `FrameMap` (source↔output frame mapping: `output_count` drives the progress total, `invert`/`total_radius` drive frame-accurate preview) |
| `worker/src/models/processing_pipeline.rs` | Pipeline passes + `FrameMap` (source↔output frame mapping: `output_count` drives the progress total, `invert`/`total_radius`/`preview_output_in` drive frame-accurate preview, including a decimation's phase) |
| `worker/src/script_generator.rs` | Template substitution for .vpy |
| `worker/src/pipeline_executor.rs` | vspipe \| ffmpeg execution |
| `worker/src/pixel_format.rs` | Source `pix_fmt` → pipe format (see "Source Pixel Formats") |
Expand Down Expand Up @@ -1537,8 +1537,10 @@ that havsfunc's QTGMC takes none of them (setting one failed every encode).

- **The few lines that genuinely differ** sit in `{{#ENCODE_ONLY}}` /
`{{#PREVIEW_ONLY}}` blocks: the pipe-source reads, Unblend's and the Frame
Rate pass's phase padding, DDColor's `preview=True`, IVTC's and Soft
Telecine's VDecimate (encode only), and the preview's final frame pick.
Rate pass's phase padding, DDColor's `preview=True`, the arrangement of
IVTC's and Soft Telecine's VDecimate input and the read of its drop
(task #46: VDecimate itself runs on both sides), and the preview's final
frame pick.
`script_generator::select_side` keeps one side and drops the other when the
file is **loaded**, before any substitution, so no other block ever sees the
side tags. Never nest them, and never write the tags anywhere else (the
Expand Down Expand Up @@ -1781,6 +1783,25 @@ time span `[t, t+1)`; the worker reports `sub_frame`, `sub_frames` and
preview job therefore carries as `startFrame` (the preview uses it for
nothing else). `integration_frame_mapping_test.dart` checks every output
frame against a lossless encode, and fails by 13 levels without the pad.
- **A decimation runs in the encode's phase too (task #46).** IVTC and Soft
Telecine at 29.97 run VDecimate in the preview, which used to skip it while
`preview_output` assumed it (the preview was `radius / 5` film frames
early). VDecimate counts cycles from its clip's first frame, reads the
frame before each cycle, and never drops a clip's frame 0, so
`preview_output_in` arranges its input as the encode's
(`DecimatePreview`): the window's frames before the in-point dropped (for
IVTC before VFM), `e0 mod cycle` copies of the first frame in front, and
the clip ended where the encode's ends (the preview job carries `endFrame`,
else the worker takes `totalFrames`; Soft Telecine's last cycle filled as
`pipe_frames` does). `FrameMap::Decimate` declares radius `cycle + 1`.
Which frame of the target's cycle goes depends on the picture: each frame
is tagged with its index, the script reads the tag where the target would
be, picks between the two outputs the walk computed (`after_drop`) and
prints `PREVIEW_DECIMATE:corr=N` for `PREVIEW_INFO`. Stabilize's pitch
lookup starts at `preview_source_offset` (`first + skip - pad`).
`integration_ivtc_preview_test.dart` (heavy) checks previews against a
lossless encode by the film frame they hold
(docs/reports/issue-46-ivtc-preview).
- A source frame with no output of its own (down-conversion, decimation)
shows the output just before it, as the preview always has.

Expand Down
11 changes: 9 additions & 2 deletions app/lib/services/preview_generator.dart
Original file line number Diff line number Diff line change
Expand Up @@ -446,6 +446,7 @@ class PreviewGenerator {
required EncodingSettings encodingSettings,
int subFrame = 0,
int? startFrame,
int? endFrame,
String? pitchFactorsPath,
CancelToken? cancelToken,
}) async {
Expand Down Expand Up @@ -501,9 +502,15 @@ class PreviewGenerator {
inputColorRange: _colorRange,
inputSar: _sar,
// Not a trim: the worker's preview only uses it as the point the
// encode's retime starts from, so a generated frame is previewed at
// the moment the encode makes it.
// encode's retime and decimation start from, so a generated frame is
// previewed at the moment the encode makes it, and a decimated one
// is the frame the encode keeps (task #46).
startFrame: startFrame,
// Not a trim either: where the encode's picture ends, so a
// decimation (IVTC, Soft Telecine) cuts its last cycle where the
// encode's is cut (task #46). Without it the worker takes the end
// of the source (`totalFrames`).
endFrame: endFrame,
// The selected video's factor file for Stabilize's "From File" pitch
// mode, so the preview corrects with the same factors as the encode.
pitchFactorsPath: pitchFactorsPath,
Expand Down
1 change: 1 addition & 0 deletions app/lib/viewmodels/main_viewmodel.dart
Original file line number Diff line number Diff line change
Expand Up @@ -1544,6 +1544,7 @@ class MainViewModel extends ChangeNotifier {
encodingSettings: _encodingSettings,
subFrame: _previewSubFrame,
startFrame: item.inPointFrame,
endFrame: item.outPointFrame,
pitchFactorsPath: item.pitchFactorsPath,
cancelToken: cancelToken,
);
Expand Down
305 changes: 305 additions & 0 deletions app/test/integration_ivtc_preview_test.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,305 @@
/// IVTC / Soft Telecine preview (task #46): the preview shows the frame the
/// encode keeps for the target, not one a fifth of the window radius early.
///
/// The preview used to run VFM without VDecimate while the worker picked the
/// output index as if it had decimated, so it showed source frame
/// `target - ceil(radius / 5)`'s film frame: one early at radius 5, three at
/// 12. A frame rate conversion after IVTC converted the preview's undecimated
/// 29.97 at the wrong ratio as well. The preview now runs VDecimate on a clip
/// arranged as the encode's (docs/reports/issue-46-ivtc-preview).
///
/// Every fixture is testsrc2 at 24000/1001, so each picture is named by the
/// film frame it holds, field by field, against the film regenerated from
/// lavfi — independently of the worker. A preview is right when it is one of
/// the lossless encode's frames (MAD < 1.5) and holds one of the target
/// slot's film frames, or, when VDecimate dropped every one of them from the
/// encode, the kept frame before them (what a frame with no output of its own
/// always shows). After a frame rate conversion it must be the encode's k-th
/// output of the slot's film frame.
///
/// Heavy (full encodes + previews) — nightly, not the push gate.
@Tags(['heavy'])
library;

// ignore_for_file: avoid_print — these tests print diagnostics to the test log.

import 'dart:io';
import 'dart:math' as math;
import 'dart:typed_data';

import 'package:flutter_test/flutter_test.dart';
import 'package:path/path.dart' as p;
import 'package:uuid/uuid.dart';

import 'package:vapourbox/models/deflicker_parameters.dart';
import 'package:vapourbox/models/encoding_settings.dart';
import 'package:vapourbox/models/frame_rate_parameters.dart';
import 'package:vapourbox/models/processing_pipeline.dart';
import 'package:vapourbox/models/qtgmc_parameters.dart';
import 'package:vapourbox/models/video_job.dart';

import 'support/worker_harness.dart';

String get _outDir => '${WorkerHarness.outputDir}/ivtc_preview';

const _tw = 88, _th = 30;
const _fullRgb = 'scale=in_range=tv:out_range=pc,format=rgb24';
const _same = 1.5;

class _Fixture {
final String file;
final int width, height, slots, film;
final String pixFmt;

/// Filters placing a soft-telecined stream's frames on the 29.97 grid.
final String slotFilters;
const _Fixture(this.file, this.width, this.height, this.pixFmt, this.slots,
this.film, [this.slotFilters = '']);
String get path =>
p.join(WorkerHarness.repoRoot, 'Tests', 'TestResources', file);
}

const _hard =
_Fixture('hard_telecine_test.avi', 720, 480, 'yuv420p', 90, 72);
const _prores =
_Fixture('prores422_10bit_telecine.mov', 720, 480, 'yuv422p10le', 30, 24);
const _dvdMkv = _Fixture('soft_telecine_dvd.mkv', 352, 240, 'yuv420p', 150,
120, 'setpts=PTS-STARTPTS,fps=30000/1001,');
const _dvdVob = _Fixture('soft_telecine_dvd.vob', 352, 240, 'yuv420p', 150,
120, 'setpts=PTS-STARTPTS,fps=30000/1001,');

const _ivtc = QTGMCParameters(
enabled: true, method: DeinterlaceMethod.ivtc, tff: true, ivtcCycle: 5);
const _soft =
QTGMCParameters(enabled: true, method: DeinterlaceMethod.softTelecine);

/// One field of every picture of [input], as a mean/std-normalised luma
/// thumbnail. `[(top, bottom), ...]`.
Future<List<List<Float64List>>> _fieldThumbs(List<String> input,
{String head = '', int? count}) async {
final fields = <List<Float64List>>[];
for (final parity in ['top', 'bottom']) {
final r = await Process.run(
WorkerHarness.ffmpegPath,
[
'-v', 'error', ...input,
'-vf', '$head$_fullRgb,field=$parity,scale=$_tw:$_th:flags=area,format=gray',
if (count != null) ...['-frames:v', '$count'],
'-f', 'rawvideo', '-',
],
environment: WorkerHarness.ffmpegEnv,
stdoutEncoding: null,
);
expect(r.exitCode, 0, reason: '${r.stderr}');
final raw = r.stdout as List<int>;
const size = _tw * _th;
fields.add([
for (var i = 0; i + size <= raw.length; i += size)
_norm(raw.sublist(i, i + size)),
]);
}
return [
for (var i = 0; i < math.min(fields[0].length, fields[1].length); i++)
[fields[0][i], fields[1][i]],
];
}

Float64List _norm(List<int> b) {
final mean = b.reduce((a, c) => a + c) / b.length;
var v = 0.0;
for (final x in b) {
v += (x - mean) * (x - mean);
}
final sd = math.sqrt(v / b.length);
return Float64List.fromList([for (final x in b) (x - mean) / (sd == 0 ? 1 : sd)]);
}

double _dist(Float64List a, Float64List b) {
var d = 0.0;
for (var i = 0; i < a.length; i++) {
d += (a[i] - b[i]).abs();
}
return d;
}

/// The film frame each field holds.
List<int> _name(List<Float64List> fields, List<List<Float64List>> film) => [
for (var f = 0; f < 2; f++)
List.generate(film.length, (i) => i).reduce(
(a, b) => _dist(fields[f], film[a][f]) <= _dist(fields[f], film[b][f]) ? a : b),
];

Future<List<Uint8List>> _rgbFrames(String path, int w, int h) async {
final r = await Process.run(
WorkerHarness.ffmpegPath,
['-v', 'error', '-i', path, '-map', '0:v:0', '-vf', _fullRgb, '-f', 'rawvideo', '-'],
environment: WorkerHarness.ffmpegEnv,
stdoutEncoding: null,
);
expect(r.exitCode, 0, reason: '${r.stderr}');
final raw = Uint8List.fromList(r.stdout as List<int>);
final size = w * h * 3;
return [for (var i = 0; i + size <= raw.length; i += size) raw.sublist(i, i + size)];
}

Future<void> _check(
String name,
_Fixture fx,
ProcessingPipeline pipeline,
List<int> targets, {
int? start,
int? end,
bool retime = false,
}) async {
final film = await _fieldThumbs(
['-f', 'lavfi', '-i', 'testsrc2=size=${fx.width}x${fx.height}:rate=24000/1001,format=yuv420p'],
count: fx.film + 2);
final slots = [
for (final s in await _fieldThumbs(['-i', fx.path], head: fx.slotFilters)) _name(s, film),
];

final job = VideoJob(
id: const Uuid().v4(),
inputPath: fx.path,
outputPath: '$_outDir/$name.mkv',
processingPipeline: pipeline,
qtgmcParameters: pipeline.deinterlace,
encodingSettings: const EncodingSettings(
codec: VideoCodec.ffv1, container: ContainerFormat.mkv, audioMode: AudioMode.none),
detectedFieldOrder: pipeline.deinterlace.method == DeinterlaceMethod.ivtc
? FieldOrder.topFieldFirst
: FieldOrder.progressive,
inputWidth: fx.width,
inputHeight: fx.height,
inputPixelFormat: fx.pixFmt,
inputFrameRate: 30000 / 1001,
totalFrames: start == null ? fx.slots : end! - start + 1,
startFrame: start,
endFrame: end,
);
final r = await WorkerHarness.runJob(job.toJson(), label: name);
expect(r.success, isTrue, reason: r.error);
final encodeThumbs = await _fieldThumbs(['-i', r.outputPath!]);
final encodeFilm = [for (final e in encodeThumbs) _name(e, film)];
final encodeRgb = await _rgbFrames(r.outputPath!, fx.width, fx.height);
final tops = [for (final e in encodeFilm) e[0]];

final wrong = <String>[];
for (final t in targets) {
var subFrames = 1;
for (var k = 0; k < subFrames; k++) {
final pv = await WorkerHarness.runPreview(job.toJson(),
frame: t, subFrame: k, label: '${name}_$t');
expect(pv.success, isTrue, reason: pv.toString());
if (retime) {
subFrames = int.parse(
RegExp(r'sub_frames=(\d+)').firstMatch(pv.logs)!.group(1)!);
}
final png = File(p.join(Directory.systemTemp.path,
'vb_ivtc_${name}_${t}_$k.png'));
await png.writeAsBytes(pv.png!);
final pvThumb = (await _fieldThumbs(['-i', png.path]))[0];
final pvRgb = await WorkerHarness.imageToRgb24(pv.png!, label: '${name}_$t');
await png.delete();
final pvFilm = _name(pvThumb, film);

// The encode frame it is: nearest thumbnail, then measured.
final order = List.generate(encodeThumbs.length, (j) => j)
..sort((a, b) => (_dist(pvThumb[0], encodeThumbs[a][0]) +
_dist(pvThumb[1], encodeThumbs[a][1]))
.compareTo(_dist(pvThumb[0], encodeThumbs[b][0]) +
_dist(pvThumb[1], encodeThumbs[b][1])));
final j = order.first;
final mad = WorkerHarness.meanAbsDiff(pvRgb, encodeRgb[j]);

final slot = slots[t].toSet();
bool right;
String want;
if (retime) {
// 23.976 -> 59.94 is x5/2: film frame f's outputs start at ceil(5f/2).
final wanted = {for (final f in slot) (5 * f + 1) ~/ 2 + k};
right = wanted.contains(j) && mad < _same;
want = 'encode $wanted';
} else if (slot.contains(pvFilm[0])) {
right = mad < _same;
want = 'film $slot';
} else if (slot.any(tops.contains)) {
right = false;
want = 'film $slot';
} else {
// The encode kept none of the slot's film frames: the one before.
final before = tops.where((f) => f < slot.reduce(math.min));
final shown = before.isEmpty ? tops.reduce(math.min) : before.reduce(math.max);
right = pvFilm[0] == shown && mad < _same;
want = 'film $shown (the slot\'s was dropped)';
}
print(' ${right ? 'ok ' : 'BAD'} $name source $t'
'${retime ? ' output $k/$subFrames' : ''}: preview film $pvFilm = '
'encode $j (MAD ${mad.toStringAsFixed(2)}), want $want');
if (!right) wrong.add('$t/$k');
}
}
expect(wrong, isEmpty,
reason: '$name: these previews are not the frame the encode shows '
'for their source frame');
}

void main() {
setUpAll(() async {
await WorkerHarness.ensureReady();
await Directory(_outDir).create(recursive: true);
});

group('IVTC / Soft Telecine preview shows the frame the encode keeps (task #46)', () {
test('IVTC at the minimum radius: start, every cycle phase, end', () async {
await _check('ivtc_r5', _hard, const ProcessingPipeline(deinterlace: _ivtc),
[0, 1, 2, 3, 4, 5, 6, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 85, 86, 87, 88, 89]);
}, timeout: const Timeout(Duration(minutes: 10)));

test('IVTC with a temporal pass widening the window to 12', () async {
// The old preview was three film frames early here.
await _check(
'ivtc_r12',
_hard,
const ProcessingPipeline(
deinterlace: _ivtc,
deflicker: DeflickerParameters(enabled: true, method: DeflickerMethod.global, window: 12),
),
[40, 41, 42, 43, 44]);
}, timeout: const Timeout(Duration(minutes: 10)));

test('IVTC trimmed: after the in-point and in the short last cycle', () async {
// 23..70 is 48 frames: the encode's last cycle holds 3, and drops one.
await _check('ivtc_trim', _hard, const ProcessingPipeline(deinterlace: _ivtc),
[23, 24, 25, 26, 27, 66, 67, 68, 69, 70],
start: 23, end: 70);
}, timeout: const Timeout(Duration(minutes: 10)));

test('IVTC on 10-bit ProRes (VFM metrics from an 8-bit copy)', () async {
await _check('ivtc_prores10', _prores, const ProcessingPipeline(deinterlace: _ivtc),
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 25, 26, 27, 28, 29]);
}, timeout: const Timeout(Duration(minutes: 10)));

test('Soft Telecine at 29.97: DVD program stream and its MKV remux', () async {
await _check('soft_vob', _dvdVob, const ProcessingPipeline(deinterlace: _soft),
[0, 1, 2, 3, 4, 60, 61, 62, 63, 64]);
// The VOB's last picture has no timestamp, so the tail is the MKV's.
await _check('soft_mkv', _dvdMkv, const ProcessingPipeline(deinterlace: _soft),
[143, 144, 145, 146, 147, 148, 149]);
}, timeout: const Timeout(Duration(minutes: 10)));

test('IVTC then 59.94: every output frame is the encode\'s', () async {
// The preview used to convert its undecimated 29.97 at about x2.
await _check(
'ivtc_fps5994',
_hard,
const ProcessingPipeline(
deinterlace: _ivtc,
frameRate: FrameRateParameters(
enabled: true, target: FrameRateTarget.ntsc5994, method: FrameRateMethod.flowFps),
),
[40, 41, 42, 43, 44],
retime: true);
}, timeout: const Timeout(Duration(minutes: 10)));
});
}
Loading
Loading