Repository navigation
Conversation
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Decoder heap undersizing, excess DMA bytes published downstream, and backend initialization on the shared LL thread risk allocation failures, payload corruption, and audio stalls; benchmark routing and SoundWire integration are also incomplete.
5 open issues
What changed in this PR
This PR updates SOF’s ffmpeg_dec runtime integration, topology configuration, and test tooling for compressed audio decoding.
Changes:
- Adjusts SRC allocation alignment, primary-core DP scheduler initialization, and compressed-stream drain alignment.
- Stores codec extradata inline and extracts container headers before lazy backend configuration.
- Parameterizes decoder core selection, adds an Opus target and benchmark fragments, and provides FLAC test-vector preparation.
| File | Description |
|---|---|
tools/topology/topology2/platform/intel/compr-ffmpeg.conf |
Uses configurable decoder core selection. |
tools/topology/topology2/platform/intel/compr-default.conf |
Updates comments to cover both decoder implementations. |
tools/topology/topology2/include/bench/ffmpeg_dec_s32.conf |
Adds S32 decoder benchmark instances. |
tools/topology/topology2/include/bench/ffmpeg_dec_s24.conf |
Adds S24 decoder benchmark instances. |
tools/topology/topology2/include/bench/ffmpeg_dec_s16.conf |
Adds S16 decoder benchmark instances. |
tools/topology/topology2/include/bench/ffmpeg_dec_route.conf |
Adds playback and capture benchmark routes. |
tools/topology/topology2/include/bench/ffmpeg_dec_controls_playback.conf |
Adds commented playback control placeholders. |
tools/topology/topology2/include/bench/ffmpeg_dec_controls_capture.conf |
Adds commented capture control placeholders. |
tools/topology/topology2/development/tplg-targets.cmake |
Registers the FFmpeg Opus compressed-playback target. |
tools/topology/topology2/development/tplg-targets-bench.cmake |
Adds FFmpeg to S32 benchmark target generation. |
tools/topology/topology2/cavs-benchmark-hda.conf |
Includes the decoder class and S32 benchmark selection. |
tools/test/audio/ffmpeg_dec_prepare.sh |
Generates FLAC data, STREAMINFO, and reference PCM. |
src/platform/intel/cavs/platform.c |
Initializes the enabled DP scheduler during platform startup. |
src/audio/src/src_common.c |
Requests eight-byte alignment for SRC delay lines. |
src/audio/host-zephyr.c |
Rounds drain remainder to DMA copy alignment. |
src/audio/ffmpeg_dec/ffmpeg_dec.h |
Replaces the extradata pointer with inline storage. |
src/audio/ffmpeg_dec/ffmpeg_dec.c |
Updates extradata handling and strips headers before backend configuration. |
Review effort: Balanced
| (uint64_t)hd->dma_copy_align); | ||
|
|
||
| remaining = ALIGN_UP(remaining, (uint64_t)dma_sample_bytes); | ||
| remaining = ALIGN_UP(remaining, align); |
| "sof-hda-generic\;sof-hda-generic-ffmpeg-compr\;HDA_CONFIG=mix,COMPR_FFMPEG=true" | ||
| "sof-hda-generic\;sof-hda-generic-ffmpeg-mp3-compr\;HDA_CONFIG=mix,COMPR_FFMPEG=true,COMPR_FFMPEG_CODEC=mp3" | ||
| "sof-hda-generic\;sof-hda-generic-ffmpeg-aac-compr\;HDA_CONFIG=mix,COMPR_FFMPEG=true,COMPR_FFMPEG_CODEC=aac,COMPR_FFMPEG_DEC_HEAP=901120" | ||
| "sof-hda-generic\;sof-hda-generic-ffmpeg-opus-compr\;HDA_CONFIG=mix,COMPR_FFMPEG=true,COMPR_FFMPEG_CODEC=opus" |
| @@ -0,0 +1,13 @@ | |||
| # Created with script "bench_comp_generate.sh ffmpeg_dec" | |||
| Object.Widget.ffmpeg_dec.1 { | |||
Comment on lines
74
to
84
| set(components_s32 | ||
| "micsel_multich" | ||
| "ffmpeg_dec" | ||
| ) | ||
|
|
||
| set(component_parameters_s32 | ||
| "BENCH_MICSEL_PARAMS=default" | ||
| "BENCH_FFMPEG_DEC_PARAMS=default" | ||
| ) | ||
|
|
||
| set (plat "mtl") |
Comment on lines
+12
to
+13
| source '$BENCH_CAPTURE_DAI_COPIER' | ||
| sink 'ffmpeg_dec.$BENCH_CAPTURE_HOST_PIPELINE.2' |
lgirdwood
commented
Nov 3, 2026
Comment on lines
+538
to
+547
| /* | ||
| * The base of delay_lines (and hence buffer_start, which is offset | ||
| * from it by a multiple of 8 bytes) must itself be 8-byte aligned: | ||
| * the HiFi FIR cores issue 64-bit AE_L32X2/AE_S32X2 circular | ||
| * accesses into it and fault (EXCCAUSE 9) on a 4-byte-aligned base. | ||
| * ALIGN_UP above only rounds the size; request the alignment on the | ||
| * allocation explicitly rather than relying on the allocator default | ||
| * (mod_alloc()/alignment 0 may return a 4-byte-aligned pointer). | ||
| */ | ||
| cd->delay_lines = mod_alloc_align(mod, delay_lines_size, sizeof(int32_t) * 2); |
Member
Author
There was a problem hiding this comment.
lets make this a single line comment and use an existing macro for vector size/alignment.
The HiFi FIR cores in the SRC filter issue 64-bit AE_L32X2/AE_S32X2 circular accesses into cd->delay_lines and fault (EXCCAUSE 9) on a 4-byte-aligned base. Allocate cd->delay_lines with SOF_FRAME_BYTE_ALIGN alignment rather than relying on the default mod_alloc() alignment. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Initialize scheduler_dp_init() in CAVS platform_init() when CONFIG_ZEPHYR_DP_SCHEDULER is enabled, matching ACE platform_init(). This allows data-processing (DP) scheduled modules such as ffmpeg_dec to run on the primary core on cAVS platforms (e.g. Tiger Lake). Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Use the COMPR_DECODER_CORE_ID define instead of a hardcoded core_id 1 on the ffmpeg_dec widget, size COMPR_FFMPEG_DEC_HEAP to 256 KiB to cover the 64 KiB PCM staging buffer and decoder working set, and add the sof-hda-generic-ffmpeg-opus-compr topology build target. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Add the ffmpeg_dec component to the playback benchmark harness (HDA and SoundWire) with DP scheduling and decoder stack/heap budgets so a topology instantiating the module can be built and driven by testbench or on target. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Add tools/test/audio/ffmpeg_dec_prepare.sh to generate FLAC test vectors, STREAMINFO extradata blobs, and bit-exact S32_LE reference PCM files for verifying the ffmpeg_dec module, as referenced in src/audio/ffmpeg_dec/TESTING.md. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
In host_get_copy_bytes_normal(), the remaining committed compress drain bytes (hd->drain_bytes - hd->total_data_processed) were rounded up only to dma_sample_bytes before clamping dma_copy_bytes. However, the return value of host_get_copy_bytes_normal() is rounded down to hd->dma_copy_align (e.g. 32 bytes on HDA). When the final tail of a compressed stream is smaller than hd->dma_copy_align, ALIGN_DOWN() rounds dma_copy_bytes down to zero, preventing hd->total_data_processed from ever reaching hd->drain_bytes and stalling compress drain EOS. Round the remaining drain byte count up to MAX(dma_sample_bytes, hd->dma_copy_align) for the DMA transfer/reload, and clamp the bytes copied into the downstream sink in host_common_update() to the actual payload remainder (consuming any trailing DMA padding from the source DMA buffer) so stale HDA bytes are never published to the decoder. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
For container-wrapped FLAC streams without topology-supplied extradata, ffmpeg_dec_skip_container() dynamically extracts the 34-byte STREAMINFO header from the stream. Running the lazy backend configure block before ffmpeg_dec_skip_container() caused avcodec_open2() to open the FLAC decoder with empty extradata before STREAMINFO could be parsed. Additionally, ffmpeg_dec_skip_container() runs inside ffmpeg_dec_process() on the user-mode DP thread, where calling mod_alloc() in ffmpeg_dec_store_extradata() faults for built-in DP modules. Embed the extradata buffer inline in struct ffmpeg_dec_comp_data and run container header stripping before lazy backend configuration. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
lgirdwood
force-pushed
the
topic/ffmpeg-decode-fixes
branch
from
November 3, 2026 20:48
2c699be to
33474be
Compare
Member
Author
|
Addressed review comments:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This series contains the remaining core, topology, and tooling updates for
ffmpeg_deccompressed audio decode support on top of the mergedffmpeg_decintegration (#11284 / #11343):audio: src: align the SRC delay line allocation to SOF_FRAME_BYTE_ALIGNmod_alloc_align(..., SOF_FRAME_BYTE_ALIGN)forcd->delay_linesso SIMD circular buffer accesses in the SRC filter do not fault whenffmpeg_decis paired with an SRC stage.platform: cavs: initialize DP scheduler on primary corescheduler_dp_init()unconditionally duringplatform_init()on CAVS/ACE platforms whenCONFIG_SOF_USERSPACE_APPLICATION=yso DP modules (such asffmpeg_dec) can be scheduled on core 0 as well as secondary cores.topology2: compr-ffmpeg: use COMPR_DECODER_CORE_ID and add Opus targetffmpeg_decpipeline and componentcore_idviaCOMPR_DECODER_CORE_ID, sizeCOMPR_FFMPEG_DEC_HEAPto 256 KiB to cover the 64 KiB PCM staging buffer and decoder working set, and add thesof-hda-generic-ffmpeg-opus-comprbuild target.topology2: add ffmpeg_dec benchmark test topologyffmpeg_dec32, HDA and SoundWire) with DP scheduling and decoder stack/heap budgets forffmpeg_dec.tools: test: add ffmpeg_dec FLAC test vector generator scripttools/test/audio/ffmpeg_dec_prepare.shreferenced bysrc/audio/ffmpeg_dec/TESTING.mdto generate raw FLAC frames, 34-byteSTREAMINFOextradata, and reference S32_LE PCM vectors.audio: host: align remaining compress drain bytes to dma_copy_alignhost_get_copy_bytes_normal()up toMAX(dma_sample_bytes, hd->dma_copy_align)for the DMA transfer/reload, and clamp the bytes copied into the downstream sink inhost_common_update()to the actual payload remainder (consuming any trailing DMA padding from the source DMA buffer).audio: ffmpeg_dec: strip container header before lazy backend configureextradatainline instruct ffmpeg_dec_comp_data(avoidingmod_alloc()on the user-mode DP thread) and strip the container header before lazy backend configuration so stream-embedded setup headers (such as FLACSTREAMINFO) are extracted beforeavcodec_open2().