Skip to content

audio: webrtc: add 32/48 kHz AGC support, AEC DP scheduling, and Kconfig overlays - #11419

Merged
lgirdwood merged 3 commits into
thesofproject:mainfrom
lgirdwood:topic/webrtc-pr1
Nov 8, 2026
Merged

lgirdwood merged 3 commits into
thesofproject:mainfrom
lgirdwood:topic/webrtc-pr1

Conversation

@lgirdwood

@lgirdwood lgirdwood commented Nov 7, 2026 •

Copy link
Copy Markdown
Member

Summary

Follow-up improvements for the WebRTC audio processing module suite merged in #11044:

  1. audio: webrtc_agc: add 32 kHz and 48 kHz sample rate support

    • WebRtcAgc_ProcessDigital() natively accepts 8, 16, 32, and 48 kHz sample rates using 160-sample sub-bands (num_bands = 1 at 8/16 kHz, 2 at 32 kHz, 3 at 48 kHz).
    • Extend webrtc_agc and webrtc-agc.conf to support 32 kHz and 48 kHz streams alongside 8 kHz and 16 kHz by downsampling each 10 ms frame to 16 kHz for VAD/envelope tracking, running a separate full-rate overload limiter pass across all 10 subframes, and applying piecewise-linear Q16 gain interpolation directly to the full-rate samples.
  2. audio: webrtc_aec: configure module for DP domain scheduling

    • Configure the webrtc_aec LLEXT manifest (.domain_dp = 1), rimage TOML (domain_types = "1"), and topology2 widget definition (scheduler_domain "DP" with stack_bytes_requirement 16384) to run in the Data Processing (DP) scheduling domain rather than on the 1 ms Low Latency (LL) thread.
    • Add webrtc_aec_is_ready_to_process() so the DP scheduler triggers processing whenever a full 10 ms microphone capture frame is available and output space is free, without blocking when the playback echo reference stream is idle.
  3. app: add WebRTC Kconfig overlays and COMP_STUBS defaults

    • Default webrtc_aec, webrtc_agc, and webrtc_hpf to their pass-through stub backends when CONFIG_COMP_STUBS=y (matching webrtc_ns, webrtc_ns2, and webrtc_vad), and enable all six WebRTC modules in app/stub_build_all_ipc4.conf.
    • Add app/webrtc_overlay.conf (LLEXT real backends + heap sizing) and app/webrtc_stubs_overlay.conf (LLEXT stub backends).

@lgirdwood lgirdwood left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commit messages are not line wrapped correctly, and webrtc modules will be deployed as llext raher than built in so we need to make sure we can build as llext.

Comment thread src/audio/webrtc_agc/webrtc_agc-agc.c Resolved
Comment thread app/stub_build_all_ipc4.conf Resolved
@lgirdwood

Copy link
Copy Markdown
Member Author

Updated the branch to address the review comments:

  1. Wrapped all commit messages to <= 72 characters per line.
  2. Added detailed step-by-step inline comments (2a–2e) to the 32/48 kHz multi-band block in src/audio/webrtc_agc/webrtc_agc-agc.c.
  3. Configured all six WebRTC modules (webrtc_vad, webrtc_ns, webrtc_aec, webrtc_ns2, webrtc_agc, webrtc_hpf) as LLEXT modules (=m) across app/stub_build_all_ipc4.conf, app/webrtc_overlay.conf, and app/webrtc_stubs_overlay.conf, and verified PTL LLEXT builds (sof-ptl-openmodules.ri) in both real-backend and stub modes.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Critical issues can silently discard 240 audio samples during consecutive 48 kHz calls and leave the advertised real-AEC overlay using a pass-through stub, while unfiltered band splitting also distorts VAD-driven gain decisions.

3 open issues
What changed in this PR

Extends SOF’s WebRTC AGC to higher sample rates and adds configuration options for WebRTC deployment and scheduler stack sizing.

Changes:

  • Adds 32/48 kHz AGC processing and matching topology formats.
  • Makes the per-core LL scheduler stack configurable, with a larger default for AEC3.
  • Adds real and stub WebRTC overlays and enables all six modules in the IPC4 stub configuration.
File Description
tools/​topology/​topology2/​include/​components/​webrtc-agc.conf Adds 32/48 kHz input and output formats.
src/​schedule/​zephyr_domain.c Uses the configurable LL stack size.
src/​schedule/​Kconfig Defines LL stack sizing and the AEC3 default.
src/​audio/​webrtc_hpf/​Kconfig Defaults HPF to its stub under COMP_STUBS.
src/​audio/​webrtc_agc/​webrtc_agc.c Accepts 32/48 kHz streams.
src/​audio/​webrtc_agc/​webrtc_agc-agc.c Adds phase splitting and peak ordering for higher rates.
src/​audio/​webrtc_agc/​README.md Documents expanded frame sizes and rates.
src/​audio/​webrtc_agc/​Kconfig Defaults AGC to its stub under COMP_STUBS.
src/​audio/​webrtc_aec/​Kconfig Defaults AEC to its stub under COMP_STUBS.
app/​webrtc_stubs_overlay.conf Adds a six-module pass-through overlay.
app/​webrtc_overlay.conf Adds real-backend settings and heap sizing.
app/​stub_build_all_ipc4.conf Enables all six WebRTC modules.

Review effort: Balanced

Comment thread app/webrtc_overlay.conf Resolved
Comment thread src/audio/webrtc_agc/webrtc_agc-agc.c Resolved
Comment thread src/audio/webrtc_agc/webrtc_agc-agc.c Resolved
Comment thread src/audio/webrtc_agc/webrtc_agc-agc.c Resolved
Comment thread src/audio/webrtc_agc/webrtc_agc-agc.c Resolved
Comment thread src/audio/webrtc_agc/webrtc_agc-agc.c Resolved
Comment thread src/audio/webrtc_agc/webrtc_agc-agc.c Resolved
Comment thread src/audio/webrtc_agc/webrtc_agc-agc.c Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Two critical AGC issues can overflow audio and probe calculations, causing wrapped output samples or reconstructed negative gains.

2 open issues
3 resolved since last review

Review effort: Balanced

Comment thread src/audio/webrtc_agc/webrtc_agc-agc.c Resolved
Comment thread src/audio/webrtc_agc/webrtc_agc-agc.c Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread src/audio/webrtc_agc/webrtc_agc-agc.c Resolved
Comment thread src/audio/webrtc_agc/webrtc_agc-agc.c Resolved
Comment thread src/schedule/Kconfig Resolved
Comment thread src/audio/webrtc_agc/webrtc_agc-agc.c Resolved
Comment thread src/audio/webrtc_agc/webrtc_agc-agc.c Resolved
Comment thread src/audio/webrtc_agc/webrtc_agc-agc.c Resolved
Comment thread src/audio/webrtc_agc/webrtc_agc-agc.c Resolved
Comment thread src/audio/webrtc_agc/webrtc_agc.h Resolved
Comment thread src/audio/webrtc_agc/webrtc_agc.h Resolved
Comment thread src/schedule/Kconfig Resolved
Comment thread src/audio/webrtc_agc/webrtc_agc-agc.c Resolved
Comment thread src/audio/webrtc_agc/webrtc_agc.h Resolved
Comment thread src/audio/webrtc_agc/webrtc_agc.h Resolved
Extend webrtc_agc to accept 32 kHz and 48 kHz streams in addition to
8 kHz and 16 kHz. Downsample 32/48 kHz 10 ms frames to an anti-aliased
16 kHz signal via WebRtcSpl_DownsampleBy2 and
WebRtcSpl_Resample48khzTo16khz for Band 0 VAD and adaptive gain
analysis, while tracking full-rate 1 ms subframe peaks separately for
the overload limiter.

Interleave FIFO filling, 10 ms processing, and draining in chunks so
calls larger than 10 ms never overflow the framing FIFOs. Update the
webrtc-agc topology2 widget definition to list 8, 16, 32, and 48 kHz.

Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Configure the webrtc_aec LLEXT manifest (.domain_dp = 1), rimage TOML
(domain_types = "1"), and topology2 widget definition (scheduler_domain
"DP" with stack_bytes_requirement 16384) to run in the Data Processing
(DP) scheduling domain rather than on the 1 ms Low Latency (LL) thread.

Add webrtc_aec_is_ready_to_process() so the DP scheduler triggers
processing whenever a full 10 ms microphone capture frame is available
and output space is free, without blocking when the playback echo
reference stream is idle.

Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Select the pass-through stub backends by default for webrtc_aec,
webrtc_agc, and webrtc_hpf when CONFIG_COMP_STUBS=y (matching webrtc_ns,
webrtc_ns2, and webrtc_vad), and include all six WebRTC modules as LLEXT
modules (=m) in app/stub_build_all_ipc4.conf.

Add app/webrtc_overlay.conf (LLEXT real backends + heap sizing) and
app/webrtc_stubs_overlay.conf (LLEXT stub backends) for firmware builds.

Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
@lgirdwood lgirdwood changed the title audio: webrtc: add 32/48 kHz AGC support, LL stack config, and Kconfig overlays audio: webrtc: add 32/48 kHz AGC support, AEC DP scheduling, and Kconfig overlays Nov 8, 2026
@lgirdwood
lgirdwood marked this pull request as ready for review November 8, 2026 19:36
@lgirdwood
lgirdwood requested a review from kv2019i as a code owner November 8, 2026 19:36
@intel-sofci

intel-sofci commented Nov 8, 2026 •

Copy link
Copy Markdown

PR 11419: test results

Run date: 2026-11-09 10:58 UTC

Tested commit: 1bf9c2043f1ab8f06687a154e8bd3146337b6a0c

mtl pass rate lnl pass rate ptl pass rate wcl pass rate nvl pass rate

@lgirdwood

Copy link
Copy Markdown
Member Author

Unrelated CI failures. Feature not yet tested in CI.

@lgirdwood
lgirdwood merged commit 24592d0 into thesofproject:main Nov 8, 2026
42 of 45 checks passed
@lgirdwood
lgirdwood deleted the topic/webrtc-pr1 branch November 8, 2026 21:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants