[lgirdwood]

Add on-DSP libavfilter audio filter graph support (src/audio/ffmpeg_dec/ffmpeg-filter.c) to the ffmpeg_dec module:

• Filter Graph Pipeline: Builds an abuffer(src) -> <filter> -> abuffersink(sink) graph with deferred DP-thread initialization so heavy filter setup (such as afftdn FFT table generation) runs on the module's deep DP stack rather than the IPC/IDC worker stack.
• Supported Audio Filters:
  • CONFIG_FFMPEG_FILTER_AFFTDN (afftdn): FFT-based audio noise reduction (AV_SAMPLE_FMT_FLTP).
  • CONFIG_FFMPEG_FILTER_ALIMITER (alimiter): Lookahead brickwall peak limiter (AV_SAMPLE_FMT_DBL).
• Runtime Selection: Selectable via Kconfig default (CONFIG_FFMPEG_FILTER_DEFAULT_AFFTDN / CONFIG_FFMPEG_FILTER_DEFAULT_ALIMITER), IPC4 snd_codec.profile, or runtime module configuration (<filter_name>[:<filter_args>]).
• PCM Format & HiFi VFPU Support: Supports S16_LE, S24_4LE, and S32_LE input/output PCM formats, using packed HiFi4/HiFi5 VFPU int32 <-> float conversion intrinsics (float.sx2 / trunc.sx2) on VFPU-capable Xtensa cores.
• LLEXT strtod / strtof: Provides C90 strtod() and strtof() in ffmpeg_dec-shims.c (matching ffmpeg_dec-builtin-libc.c) so libavfilter option parsing works in both built-in (=y) and LLEXT (=m) builds.

[Copilot]

🟡 Changes recommended

Critical defects remain in end-of-stream draining, 24-bit PCM scaling, configuration synchronization, and IPC3 filter dispatch, causing lost audio, incorrect output, race conditions, or initialization failure.

6 open issues

• Synchronize configuration updates with filter graph opening · New
• Use valid PCM formats for filter conversions · New
• Drain final partial input and filter output before signaling EOS · New
• Dispatch IPC3 filter instances to the default filter · New
• Saturate float-to-int32 conversion at the positive limit · New
• Preserve exponent scale and handle overflow and subnormals · New

What changed in this PR

Adds on-DSP libavfilter processing to ffmpeg_dec, extending its decoder and encoder support with PCM noise reduction and peak limiting.

Changes:

• Adds afftdn and alimiter graphs with deferred initialization, PCM conversion, and runtime filter selection.
• Integrates filters into Kconfig, built-in and LLEXT builds, module lifecycle dispatch, and PCM capability reporting.
• Implements LLEXT floating-point string parsing and updates built-in parsing helpers.

File	Description
src/​kernel/​audio/​base_fw.c	Advertises PCM playback when filters are enabled.
src/​audio/​ffmpeg_dec/​llext/​CMakeLists.txt	Includes filter code and adjusts library link order.
src/​audio/​ffmpeg_dec/​Kconfig	Adds filter selection and default-filter options.
src/​audio/​ffmpeg_dec/​ffmpeg.cmake	Enables selected libavfilter components in the cross-build.
src/​audio/​ffmpeg_dec/​ffmpeg-filter.c	Implements filter graphs, PCM conversion, buffering, and lifecycle handling.
src/​audio/​ffmpeg_dec/​ffmpeg_dec.h	Declares filter entry points and instance state.
src/​audio/​ffmpeg_dec/​ffmpeg_dec.c	Routes initialization, processing, and configuration to filter mode.
src/​audio/​ffmpeg_dec/​ffmpeg_dec-shims.c	Implements LLEXT strtod() and strtof().
src/​audio/​ffmpeg_dec/​ffmpeg_dec-builtin-libc.c	Updates special-value handling and bounds exponent parsing.
src/​audio/​ffmpeg_dec/​CMakeLists.txt	Adds filter sources and libavfilter linkage for built-in builds.

_{Review effort: Balanced}

[intel-sofci]

PR 11423: test results

Run date: 2026-11-09 10:20 UTC

Tested commit: 7319a8f2ea59989acd831e5bed250c4c62d0aa03

[Copilot]

🟡 Changes recommended

Unresolved critical defects can discard final audio, mis-scale IPC4 24-bit samples, and cause runtime faults in LLEXT builds with the FPU disabled.

9 open issues

• Apply the soft-float override to LLEXT filter glue · New
• Dispatch IPC3 filter instances to the default filter
• Drain final partial input and filter output before signaling EOS
• Use valid PCM formats for filter conversions
• Synchronize configuration updates with filter graph opening
• Avoid EOS-only wakeups when output cannot drain · New
• Preserve requested exponents when handling range limits · New
• Preserve exponent scale and handle overflow and subnormals
• Saturate float-to-int32 conversion at the positive limit

_{Review effort: Balanced}

[lgirdwood]

All 18 review comments have been addressed and pushed in commit 7319a8f2e (with inline replies on each thread):

Review Comment Fixes

1. Separate HiFi SIMD file (#discussion_r4506789481):
   • Moved all Xtensa HiFi3 / HiFi4 VFPU SIMD conversion intrinsics into a dedicated src/audio/ffmpeg_dec/ffmpeg-convert-hifi.c file (ffmpeg_hifi_fltp_to_s32(), ffmpeg_hifi_s32_to_fltp(), ffmpeg_hifi_s32p_to_s32_stereo()).
2. Reuse existing conversion APIs and SOF Q-format macros (#discussion_r4506787736, #discussion_r4506806570, #discussion_r4506807306, #discussion_r4506630174):
   • Removed all duplicate conversion routines and custom scale/clamp/sign-extend macros from ffmpeg-filter.c.
   • Extended ffmpeg_dec-convert.c / ffmpeg_dec-convert.h with ffmpeg_dec_pcm_to_frame() and AV_SAMPLE_FMT_DBL / SOF_IPC_FRAME_S24_4LE support in ffmpeg_dec_convert_frame(), reusing <sof/audio/format.h> macros (Q_CONVERT_QTOF, Q_CONVERT_QTOD, Q_SHIFT, Q_SHIFT_LEFT, sign_extend_s24, sat_int32) and f32_to_q31_scalar() (avoiding float-cast overflow at +1.0f).
3. IPC4-only filter code & Kconfig (#discussion_r4506810503, #discussion_r4506630162):
   • Added depends on IPC_MAJOR_4 to CONFIG_FFMPEG_FILTER_AFFTDN and CONFIG_FFMPEG_FILTER_ALIMITER in src/audio/ffmpeg_dec/Kconfig and removed all #if CONFIG_IPC_MAJOR_4 preprocessor conditionals from ffmpeg-filter.c.
4. Inline phase comments in ffmpeg_af_open() (#discussion_r4506796084):
   • Added inline phase comments (Phase 1 through Phase 5) documenting descriptor lookup/parameter validation, graph/source allocation, filter/sink instantiation, graph linking/format negotiation, and persistent AVFrame/staging buffer allocation.
5. Channel layout buffer size macro (#discussion_r4506798796):
   • Defined #define FFMPEG_AF_LAYOUT_STR_SIZE 64 and used it for both char layout_str[FFMPEG_AF_LAYOUT_STR_SIZE] and av_channel_layout_describe().
6. Warn before applying default parameter fallbacks (#discussion_r4506803161):
   • Added comp_warn() (LOG_WRN) before applying fallback defaults for rate, channels, in_frame_bytes, out_frame_bytes, and frame_size in ffmpeg_af_open().
7. Why strtod / strtof are provided in libc shims & exponent scaling (#discussion_r4506814274, #discussion_r4506630185, #discussion_r4506803895):
   • Documented above strtod() in ffmpeg_dec-builtin-libc.c and ffmpeg_dec-shims.c that Zephyr's CONFIG_MINIMAL_LIBC (zephyr/lib/libc/minimal/source/stdlib/) only provides atoi/strtol/strtoul and omits strtod/strtof, while the toolchain's prebuilt libc.a is compiled without -mlongcalls / PIC and cannot be linked into LLEXT or --no-relax firmware images (libavfilter's av_opt_set_from_string() calls av_strtod() -> strtod()).
   • Fixed exponent scaling in strtod() to track fractional digit offsets (frac_exp) and scale in chunks (1e8/10.0 and 1e-8/0.1) without clamping exp_val at 308, preserving overflow to +/-inf and subnormal values.
8. Synchronize ffmpeg_af_mod_set_config() with active graph (#discussion_r4506630101):
   • Rejected runtime reconfiguration with -EBUSY when the module is active or the graph is already open (dev->state == COMP_STATE_ACTIVE || cd->configured || g->graph).
9. Valid sample format vs. container format validation (#discussion_r4506630122):
   • Updated ffmpeg_af_mod_prepare() to query both container (source_get_frm_fmt / sink_get_frm_fmt) and valid sample formats (source_get_valid_fmt / sink_get_valid_fmt) and validate container width and frame_bytes via ffmpeg_af_validate_pcm_fmt().
10. EOS partial input block & filter graph EOF draining (#discussion_r4506630141, #discussion_r4506803883):
    • Updated ffmpeg_af_mod_process() to submit any remaining < frame_size input block at EOS, push a NULL EOF frame to abuffer, and drain buffered filter frames until av_buffersink_get_frame() returns AVERROR_EOF before signaling EOS, and tightened ffmpeg_af_is_ready_to_process() so EOS wakeups only occur when progress can be made.
11. LLEXT Clang soft-float -fp override (#discussion_r4506803860):
    • Added -Xclang -target-feature -Xclang -fp for Clang builds in src/audio/ffmpeg_dec/llext/CMakeLists.txt (and added ffmpeg-convert-hifi.c and ffmpeg_dec-builtin-libc.c to _ffmpeg_dec_sources in src/audio/ffmpeg_dec/CMakeLists.txt).

[Copilot]

🟡 Changes recommended

Unresolved defects can break Clang LLEXT compilation, block the low-latency pipeline, cause alignment faults, prevent limiter initialization, and silently drop audio tails.

5 open issues

• Fall back to scalar stores for misaligned VFPU destinations · New
• Reject low-latency instances during filter initialization · New
• Preserve paired -Xclang flags in the LLEXT build · New
• Use a boolean value accepted by the LLEXT option parser · New
• Limit pulled frames to FIFO capacity to avoid dropping the tail · New

9 resolved since last review

• Apply the soft-float override to LLEXT filter glue
• Dispatch IPC3 filter instances to the default filter
• Drain final partial input and filter output before signaling EOS
• Use valid PCM formats for filter conversions
• Synchronize configuration updates with filter graph opening
• Preserve requested exponents when handling range limits
• Avoid EOS-only wakeups when output cannot drain
• Preserve exponent scale and handle overflow and subnormals
• Saturate float-to-int32 conversion at the positive limit

_{Review effort: Balanced}

[lgirdwood]

CI failures unrelated to PR.