audio: ffmpeg_dec: add FFmpeg (libavcodec/libavfilter) module

Add a loadable (LLEXT) audio module that wraps FFmpeg. Two build modes:
 - decoder (default): a compressed elementary stream (FLAC/AAC/Opus,
   selected via Kconfig) is parsed and decoded to PCM (process_raw_data);
 - filter (CONFIG_FFMPEG_DEC_FILTER_MODE): a PCM source/sink effect that
   runs an FFmpeg audio filter graph (afftdn noise reduction).

FFmpeg is a west-pinned source (see west.yml) cross-built by ffmpeg.cmake
as a CMake ExternalProject, enabling only the Kconfig-selected decoders and
filters. The module supplies the libc surface FFmpeg needs that the SOF core
does not export to LLEXT (ffmpeg_dec-shims.c), a SOF-heap-backed malloc
(ffmpeg_dec-alloc.c), fast single-precision float math (fastmathf.c), and
routes av_log() into the Zephyr log. The avfilter graph backend and the PCM
effect ops are in ffmpeg_dec-filter.c.

Also adds the rimage module manifest (ffmpeg_dec.toml + per-platform
includes), the topology widget (ffmpeg_dec.conf), host test tooling
(ffmpeg_dec_prepare.sh) and docs (README/TESTING).

Verified building and load-ready (all symbols resolved, signed) for ace30
(ptl) with the Zephyr SDK: FLAC and FLAC+AAC decoders, and the afftdn filter
effect. Runtime decode/denoise verification is pending hardware.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: lrgirdwo

src/audio/CMakeLists.txt

@@ -41,6 +41,9 @@ if(NOT CONFIG_COMP_MODULE_SHARED_LIBRARY_BUILD)
 	if(CONFIG_COMP_DRC)
 		add_subdirectory(drc)
 	endif()
+	if(CONFIG_COMP_FFMPEG_DEC)
+		add_subdirectory(ffmpeg_dec)
+	endif()
 	if(CONFIG_COMP_FIR)
 		add_subdirectory(eq_fir)
 	endif()

src/audio/Kconfig

@@ -139,6 +139,7 @@ rsource "dcblock/Kconfig"
 rsource "drc/Kconfig"
 rsource "eq_fir/Kconfig"
 rsource "eq_iir/Kconfig"
+rsource "ffmpeg_dec/Kconfig"
 rsource "google/Kconfig"
 rsource "igo_nr/Kconfig"
 rsource "mfcc/Kconfig"

src/audio/ffmpeg_dec/CMakeLists.txt

@@ -0,0 +1,18 @@
+# SPDX-License-Identifier: BSD-3-Clause
+
+if(CONFIG_COMP_FFMPEG_DEC STREQUAL "m" AND DEFINED CONFIG_LLEXT)
+  add_subdirectory(llext ${PROJECT_BINARY_DIR}/ffmpeg_dec_llext)
+  add_dependencies(app ffmpeg_dec)
+else()
+  add_local_sources(sof ffmpeg_dec.c)
+
+  if(CONFIG_COMP_FFMPEG_DEC_STUB)
+    add_local_sources(sof ffmpeg_dec-stub.c)
+  else()
+    add_local_sources(sof ffmpeg_dec-ffmpeg.c)
+    # NOTE: ffmpeg_dec-shims.c / ffmpeg_dec-alloc.c are LLEXT-only (see
+    # llext/CMakeLists.txt). They provide libc/libm symbols the isolated LLEXT
+    # cannot import; in a built-in (=y) or native/testbench build the real libc
+    # provides those, so including them here would multiply-define malloc etc.
+  endif()
+endif()

src/audio/ffmpeg_dec/Kconfig

@@ -0,0 +1,97 @@
+# SPDX-License-Identifier: BSD-3-Clause
+
+config COMP_FFMPEG_DEC
+	tristate "FFmpeg (libavcodec) audio decoder"
+	select COMP_FFMPEG_DEC_STUB if COMP_STUBS
+	help
+	  Select to include the FFmpeg audio decoder module. It wraps
+	  libavcodec decoders behind the SOF module interface, decoding a
+	  compressed elementary stream to PCM. The real backend cross-builds
+	  libavcodec/libavutil/libswresample from the FFmpeg source pulled in
+	  by west (see west.yml), enabling only the decoders selected below.
+	  Without the source (or for CI) select COMP_FFMPEG_DEC_STUB to build
+	  the dependency-free passthrough stub.
+
+config COMP_FFMPEG_DEC_STUB
+	bool "FFmpeg decoder stub backend"
+	depends on COMP_FFMPEG_DEC
+	help
+	  Build the ffmpeg_dec module against a dependency-free passthrough
+	  backend instead of libavcodec. Intended for testing and CI so the
+	  SOF glue and LLEXT packaging can be validated without the FFmpeg
+	  source or a cross-built archive.
+
+if COMP_FFMPEG_DEC && !COMP_FFMPEG_DEC_STUB
+
+comment "FFmpeg decoders to build (footprint scales with selection)"
+
+config FFMPEG_DEC_FLAC
+	bool "FLAC decoder"
+	default y
+	help
+	  Lossless integer decoder. Smallest footprint, no floating-point
+	  math. Enables --enable-decoder=flac in the libavcodec build.
+
+config FFMPEG_DEC_AAC
+	bool "AAC-LC decoder"
+	help
+	  AAC Low Complexity decoder. Pulls in the single-precision float
+	  math layer (see FFMPEG_DEC_FLOAT_MATH). Enables
+	  --enable-decoder=aac in the libavcodec build.
+
+config FFMPEG_DEC_OPUS
+	bool "Opus decoder"
+	help
+	  Opus (SILK+CELT) decoder. Pulls in the single-precision float math
+	  layer. Enables --enable-decoder=opus in the libavcodec build.
+
+comment "FFmpeg audio filters (libavfilter; need the avfilter-graph backend to run)"
+
+config FFMPEG_FILTER_AFFTDN
+	bool "FFT noise reduction (afftdn)"
+	help
+	  Build FFmpeg's FFT-based denoiser (afftdn) into libavfilter. Pure DSP,
+	  no external model file (unlike arnndn). Enabling any filter turns on
+	  libavfilter in the cross-build. NOTE: a filter is only usable at runtime
+	  once the module gains an avfilter-graph backend; selecting it here just
+	  builds the filter into the archive.
+
+config FFMPEG_DEC_FILTER_MODE
+	bool "Build as a PCM filter effect instead of a decoder [EXPERIMENTAL]"
+	select FFMPEG_FILTER_AFFTDN
+	help
+	  Build the module as a PCM source->sink effect that runs an FFmpeg
+	  audio filter graph (default afftdn noise reduction) via the modern
+	  .process interface, instead of the compressed-stream decoder. Turns on
+	  libavfilter and the fast float math. Structurally complete and
+	  load-ready; real-time latency/format tuning needs on-hardware bring-up.
+
+# Turns on libavfilter in the FFmpeg cross-build when any filter is selected.
+config FFMPEG_BUILD_AVFILTER
+	bool
+	default y if FFMPEG_FILTER_AFFTDN
+
+# Selected automatically when a decoder or filter that needs IEEE float math is
+# built. Gates compilation of the fast single-precision libm (fastmathf.c).
+config FFMPEG_DEC_FLOAT_MATH
+	bool
+	default y if FFMPEG_DEC_AAC || FFMPEG_DEC_OPUS || FFMPEG_FILTER_AFFTDN
+
+config FFMPEG_DEC_COLD_SPLIT
+	bool "Place cold FFmpeg code (av_cold init/setup) in DRAM [EXPERIMENTAL]"
+	depends on COLD_STORE_EXECUTE_DRAM
+	default n
+	help
+	  EXPERIMENTAL. Build libavcodec with -mtext-section-literals and rename
+	  FFmpeg's cold functions (its av_cold init/table-generation code, which GCC
+	  emits into .text.unlikely) into SOF's .cold section so the LLEXT loader
+	  places them in DRAM, keeping the hot per-frame decode in fast SRAM.
+
+	  Measured cold code is small (~1 KB for FLAC, ~26 KB for AAC) because
+	  FFmpeg only marks init functions av_cold. Placing that much cold code in a
+	  -shared LLEXT currently hits Xtensa linker limits (cold->hot call8 range
+	  needs -mlongcalls; the orphan .cold lands at a bad LMA and overlaps .hash),
+	  so this is off by default pending proper cold-region placement in the LLEXT
+	  memory map. See git history / TESTING notes.
+
+endif # COMP_FFMPEG_DEC && !COMP_FFMPEG_DEC_STUB

src/audio/ffmpeg_dec/README.md

@@ -0,0 +1,59 @@
+# FFmpeg audio decoder module (ffmpeg_dec)
+
+Wraps FFmpeg's `libavcodec` audio decoders behind the SOF `module_interface`,
+decoding a compressed elementary stream to PCM inside the DSP. First target
+codec is **FLAC**.
+
+## Design
+
+The SOF glue and the decoder are separated so the integration can be validated
+without libavcodec:
+
+- `ffmpeg_dec.c` — SOF module core: `init` / `prepare` / `process_raw_data` /
+  `set_configuration` / `reset` / `free`, plus the LLEXT manifest. Input is the
+  compressed byte stream (raw data), output is interleaved PCM. Decoding is
+  delegated to a `struct ffmpeg_dec_backend`.
+- `ffmpeg_dec-stub.c` — dependency-free passthrough backend. Lets CI build and
+  exercise the module (pipeline, IPC config, LLEXT packaging) with **no** FFmpeg
+  libraries. Selected by `CONFIG_COMP_FFMPEG_DEC_STUB`.
+- `ffmpeg_dec-ffmpeg.c` — real backend driving the standalone `libavcodec`
+  send-packet / receive-frame API. A raw stream is framed on-DSP with an
+  `AVCodecParser` (`av_parser_parse2`); decode is forced single-threaded
+  (`thread_count = 1`). Requires pre-compiled static libraries and headers under
+  `third_party/` (see below).
+- `ffmpeg_dec.h` — private data (`struct ffmpeg_dec_comp_data`) and the backend
+  interface.
+
+The codec setup header (e.g. FLAC STREAMINFO) is delivered as a binary control
+via `set_configuration` and stored as `extradata`, which the libavcodec backend
+installs before `avcodec_open2()`.
+
+## Build
+
+- **Stub (default for testing/CI):** `CONFIG_COMP_FFMPEG_DEC=y` (or `=m` for
+  LLEXT) with `CONFIG_COMP_FFMPEG_DEC_STUB=y`. No external dependencies.
+- **Real decoder:** `CONFIG_COMP_FFMPEG_DEC=m`, `CONFIG_COMP_FFMPEG_DEC_STUB=n`.
+  Requires cross-compiled decoder-only FFmpeg static libraries installed as:
+  - `third_party/lib/libavcodec.a`, `libavutil.a`, `libswresample.a`
+  - `third_party/include/libavcodec/…`, `libavutil/…`, `libswresample/…`
+  Configure FFmpeg with `--disable-everything --disable-avformat
+  --disable-pthreads --enable-decoder=flac --enable-parser=flac` (plus the
+  target cross-compile flags).
+
+## Files
+
+- `Kconfig` — `COMP_FFMPEG_DEC` (tristate) and `COMP_FFMPEG_DEC_STUB`.
+- `CMakeLists.txt` / `llext/CMakeLists.txt` — static and LLEXT builds; the LLEXT
+  target name is `ffmpeg_dec` and links the FFmpeg libraries in the non-stub
+  branch.
+- `ffmpeg_dec.toml` / `llext/llext.toml.h` — rimage module manifest
+  (`UUIDREG_STR_FFMPEG_DEC`); OBS is sized above IBS since PCM out ≫ compressed
+  in.
+
+## Status / TODO
+
+- Codec id is currently hard-coded to FLAC in `init`; wire it from topology/IPC
+  init config for other codecs.
+- Output is assumed to already match the sink sample format; add
+  `libswresample` (or a fixed-point path) for format/rate conversion.
+- Topology and host test tooling for end-to-end (bit-exact) FLAC decode.

src/audio/ffmpeg_dec/TESTING.md

@@ -0,0 +1,66 @@
+# Testing the ffmpeg_dec module
+
+This describes how to verify the FFmpeg FLAC decoder module end to end. FLAC is
+lossless, so a correct decode is **bit-exact** against a reference decode.
+
+## Test vectors
+
+Generate vectors with the helper (needs `ffmpeg` on the host):
+
+```
+tools/test/audio/ffmpeg_dec_prepare.sh -i song.wav -o vectors/
+```
+
+Produces in `vectors/`:
+- `streaminfo.bin` — 34-byte FLAC STREAMINFO = the codec **extradata** blob to
+  load into the module's bytes control (`set_configuration`).
+- `stream.raw` — the raw FLAC elementary stream fed to the decoder input.
+- `ref_s32le.raw` — reference interleaved S32_LE PCM; the decoder output must
+  match this byte-for-byte.
+
+## Topology
+
+The component widget is `tools/topology/topology2/include/components/ffmpeg_dec.conf`
+(models the DTS codec: `type "effect"`, one input/one output pin, a `bytes`
+control for STREAMINFO). Its `uuid` must match the firmware's registry entry for
+`ffmpeg_dec` (see `uuid-registry.txt`).
+
+A runnable pipeline must deliver **compressed bytes** to the module and take PCM
+out. Two paths:
+
+1. **ALSA compress-offload** (the production path for a decoder): the host writes
+   the FLAC elementary stream to a compress DMA gateway; the module decodes; PCM
+   goes to a DAI or host capture. This is the correct model (a decoder is not a
+   fixed-rate PCM effect) and is the remaining integration to wire on real ptl
+   hardware.
+2. **Direct raw-data injection** (bring-up shortcut): a minimal pipeline where a
+   host copier delivers `stream.raw` as an opaque byte buffer into the module's
+   `process_raw_data`, capturing PCM on the other side.
+
+## Verification paths
+
+### A. On target (ptl hardware) — full verification
+1. Build + flash firmware with the real backend:
+   `CONFIG_COMP_FFMPEG_DEC=m`, `CONFIG_COMP_FFMPEG_DEC_STUB=n` (requires the
+   FFmpeg static libs in `third_party/`; see the module README).
+2. Load the topology; set the STREAMINFO bytes control:
+   `amixer -c0 cset name='FFMPGDEC...' -- $(od -An -tx1 vectors/streaminfo.bin)`
+   (or via `tinymix`).
+3. Play `stream.raw` through the pipeline, capture the PCM output.
+4. `cmp captured.raw vectors/ref_s32le.raw` — must be identical.
+
+### B. Native testbench (no hardware) — decode-only check
+The module builds native (built-in, real libc — the LLEXT-only shims/alloc are
+excluded automatically, see `CMakeLists.txt`). Linking the **host** libavcodec
+and driving `process_raw_data` over `stream.raw` with `ref_s32le.raw` as the
+oracle gives a bit-exact check without a DSP. This harness is not yet wired (the
+testbench file component is PCM-oriented; feeding compressed bytes needs a small
+adapter) and is the recommended next step for CI-friendly verification.
+
+## Status
+
+- Component widget conf: **done**.
+- Host vector/reference tooling: **done** (`ffmpeg_dec_prepare.sh`).
+- Runnable pipeline (compress-offload) + on-target bit-exact run: **pending
+  hardware**.
+- Native testbench harness: **pending** (small compressed-input adapter).