Skip to content

Format sweep findings: width 2 mod 4, field order after Rotate/Flip, source depth, Bifrost, CLAHE, MVTools two-row crash (task #56) - #214

Open
StuartCameronCode wants to merge 4 commits into
mainfrom
stuart/56-format-sweep-findings-task-47-width-2-mo
Open

StuartCameronCode wants to merge 4 commits into
mainfrom
stuart/56-format-sweep-findings-task-47-width-2-mo

Conversation

@StuartCameronCode

Copy link
Copy Markdown
Owner

Fixes every row the generated format sweep (task #47, #212) listed under task #56 in knownFailures. The list is now empty, so the sweep fails if any of these breaks again.

Findings and fixes

What failed Fix
F1 MCTemporalDenoise at a 4:2:0 / 4:2:2 width ≡ 2 mod 4 (e.g. 854x480): "cropped area needs to have mod 2 width offset" MCTD runs through run_within with a width_step=CHROMA contract. The frame is padded to whole chroma columns and cropped back.
F2 QTGMC Super Fast / Ultra Fast at the same widths: "image dimensions must be divisible by subsampling factor" Only SrchClipPP = 1 halves the frame for the motion search, so only those presets get a width step: a separate contract key, havsfunc.QTGMC[SrchClipPP=1]. The other presets keep their output unchanged. Padding them would move columns well inside the frame; the edge cost is measured in the report.
F3 A quarter turn kept _FieldBased, so a turned 4:2:2 picture was resampled per "field" (its columns) ScriptGenerator::field_order_at_framing and GeometryParameters::field_order_after: a quarter turn makes the picture progressive, and a vertical flip or half turn swaps TFF and BFF. The geometry block stamps the new order. The border alignment, the crop detector, the resize step and output_field_order all read it.
F4 Deband's default 16-bit working depth made an 8-bit source's "original" output 16-bit Under Color Format "original", {{#DEPTH_RESTORE}} dithers back to the source's depth after the passes. Only the depth is restored, never the colour family, and it runs before the borders.
F5 Resizing a still-interlaced 4:2:0 picture to a height ≡ 2 mod 4 framing_steps: the resize height is a multiple of 4 there, in the script's _height() and in Rust's output_size_stepped.
F6 Bifrost v3.0 returned uninitialised chroma past its last whole 4x4 block, which differed run to run, including at 720x486 4:2:2 A contract with width_step=4, height_step=8. Padded, the output is identical across runs at every size tested. Task #60 (Bifrost v4.0) can drop this.
F7 CLAHE parity on genuinely deep content: 3–22 levels from the 8-bit result Local Contrast decides on an 8-bit copy at every depth and adds only its change at the clip's depth (plugin_formats.change_at). Genuine 10–16-bit content now lands about 1 level from the 8-bit result.
F8 The preview's YUV→RGB conversion interpolated chroma at some depths and not at 8-bit 4:2:0, about 4.5 levels apart The preview scale uses accurate_rnd+full_chroma_int at every depth.
F9 QTGMC Very Fast / Super Fast / Ultra Fast / Draft crashed on x86 at 64-line frames MVTools' Compensate / Degrain / BlockFPS divide by zero ((by + nBlkY - 3) / (nBlkY - 2)) when a picture is two block rows tall. That is SIGFPE on x86 and up to 106 levels off, silently, on arm64. Every MVTools caller with an overlap now goes through _pf.mvtools_blocks, which pads a picture smaller than 64x96 and crops back. No real source is that small, so real jobs get no extra node. test_256 lints that every caller is wrapped.
F9b A crash in the preview was reported as the first warning on vspipe's stderr ("plugin … already loaded") The preview uses the encode's vspipe_failure_message, so a native crash is reported as one, with its signal or Windows status.

The frame-size steps use task #57's pad_around / Contract helper, copied byte for byte from its commit so the two branches carry the same text. The probe (Scripts/probe-plugin-formats.py) and plugin_format_table.json are re-run.

The evidence is in docs/reports/task-56-format-sweep-findings/ (index.html, scripts, data), with a dated ENGINEERING_NOTES entry. CLAUDE.md is updated: field order after Rotate / Flip, depth restore, CLAHE, contract steps, the MVTools rule, and SIGFPE / 0xC0000094 in Debugging Tips.

Review fix: output_size_stepped now steps Pad to Fill's box height as the script does (486 becomes 484 when the step is 4). Before, it disagreed with the script, which affected the BT.709 Auto decision and the encoder's colour tags.

Testing

  • cargo test: 1475 passed, 0 failed. New tests are test_253–256 plus units for field_order_after, output_size_stepped, the preview scale filter and the crash-after-warnings message.
  • flutter test --exclude-tags heavy,gpu: 1771 passed (run before the one-line review fix).
  • After the review fix: format_sweep_cases_test, plugin_format_contracts_test (including the quick probe against the table and pad_around on real frames) and template_call_arguments_test all passed.
  • Measured on macOS arm64. The x86 F9 crash was reproduced and fixed under Rosetta with the macOS x64 bundle; this PR's CI on all four platforms is the native x86 proof.

Coordination

Task #57 (720x486, height ≡ 2 mod 4) shares plugin_formats.py. Since the commit this branch copied, it has added a field_based_only contract field, read_padded and a 96x66 sweep size. Whichever of the two merges second must merge the Contract fields and drop #57's two task-56 knownFailures entries (Bifrost 96x66, padToAspect 96x66 TFF); this branch's Bifrost step and F5 height step should cover them.

🤖 Generated with Claude Code

StuartCameronCode and others added 4 commits November 17, 2026 09:34
…, source depth under "original", preview chroma (task #56)

- F3: a quarter turn makes a still-interlaced picture progressive and a
  vertical mirror (or half turn) swaps its field order. One decision,
  ScriptGenerator::field_order_at_framing (GeometryParameters::
  field_order_after), stamped on the clip before the geometry pass's format
  restore and read by output_field_order, the borders' field alignment and
  the crop/resize steps. test_221 lists the new SetFieldBased site; test_253.
- F5: a still-interlaced 4:2:0 picture is resized to a height that is a
  multiple of 4 (framing_steps, shared with the crop detector);
  CropResizeParameters::output_size_stepped mirrors it. test_254.
- F4: Color Format "original" restores the source's depth (dithered) after
  the passes, so Deband's output_depth is the working depth of the passes
  after it, not the file's. Schema label "Working Depth". test_255.
- F8: the preview's YUV->RGB conversion interpolates chroma at every depth
  (accurate_rnd+full_chroma_int).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s, CLAHE at 8-bit, readable preview crashes (task #56)

- F9: MVTools' Compensate/Degrain/BlockFPS pick the overlap window with
  (by + nBlkY - 3) / (nBlkY - 2): a picture two block rows tall is SIGFPE on
  x86 and up to 127 levels silently wrong on arm64; one row or column is
  wrong everywhere. Every MVTools overlap caller (QTGMC, MCTD, SMDegrain,
  MCDegrainSharp, SpotLess, mClean, TemporalDegrain2) runs through
  plugin_formats.mvtools_blocks, which pads a picture under 64x96 and crops
  back (no node at real sizes). test_256.
- F9b: the preview reports a vspipe failure with the encode's
  vspipe_failure_message, so a native crash says so instead of quoting the
  first plugin warning.
- F1/F2/F6: task #57's pad_around (verbatim) and Contract steps. MCTD and
  QTGMC with SrchClipPP 1 pad a 4:2:x width 2 mod 4; QTGMC's other presets
  keep their width (padding moved their pictures inside the frame).
  Bifrost pads to its 4x8 blocks, whose leftover chroma v3.0 never
  initialises (720x486 4:2:2 differed run to run). test_257; probe table
  re-measured through run_within (havsfunc.QTGMC[SrchClipPP=1] added).
- F7: Local Contrast's CLAHE decides on an 8-bit copy at every depth and
  only its change is applied (plugin_formats.change_at): parity with 8-bit
  3-22 -> ~1 level at SD.
- pipeline_executor: preview_scale_filter no longer splits
  field_order_note's doc comment.
- The sweep's knownFailures are empty; report
  docs/reports/task-56-format-sweep-findings, CLAUDE.md, ENGINEERING_NOTES.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t_size (task #56 review)

The script pads to _height(target_h); output_size_stepped used the unstepped
target, so a still-interlaced 4:2:0 job padded to 486 predicted 486 where the
script delivers 484.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The padded cells and the SrchClipPP=1 QTGMC call took the quick probe from
550 s to ~800 s on the push gate's runners; macOS x64 passed 15 min.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant