Skip to content

Commit 10f174f

Browse files
lrgirdwoclaude
andcommitted
ffmpeg_dec: HIFI.md: add AAC-LC decode hot-spot appendix
Document the per-frame AAC-LC decode hot spots for Xtensa HiFi work: IMDCT (libavutil/tx), the AVFloatDSPContext vector kernels (vector_fmul_window and friends), x^(4/3) inverse quant, and TNS, plus SBR/PS for HE-AAC. Includes a toolchain-requirement note: the HiFi intrinsic headers (xt_hifi*.h) and an ace30 core-isa.h with XCHAL_HAVE_HIFI4=1 are only available via the LLVM/xt-clang Xtensa toolchain, not the Zephyr-SDK GCC currently used by ffmpeg.cmake, so the ff_*_init_xtensa kernels need an intrinsic path plus a scalar C fallback. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent e9da0d5 commit 10f174f

1 file changed

Lines changed: 73 additions & 0 deletions

File tree

src/audio/ffmpeg_dec/HIFI.md

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -134,6 +134,79 @@ per-sample loop), so they are a secondary target. HiFi wins come from:
134134
vector of them (e.g. AAC scalefactor gains), and use FMA for the polynomials.
135135
Mirror `src/math/exp_fcn_hifi.c`.
136136

137+
## Appendix: AAC-LC decode hot spots (per-frame detail)
138+
139+
Where the cycles actually go decoding one AAC-LC frame (1024 samples long, or
140+
8×128 short) per channel, mapped to the code in the tree. Everything below runs as
141+
**scalar C** on Xtensa today (`--disable-asm`, no `_xtensa` DSP init).
142+
143+
### Toolchain requirement (applies to all FFmpeg-side HiFi work)
144+
145+
The Cadence HiFi intrinsic headers (`<xtensa/tie/xt_hifi{3,4,5}.h>`) and a
146+
`core-isa.h` with `XCHAL_HAVE_HIFI4 == 1` are **only** available through the
147+
**LLVM/xt-clang Xtensa** toolchain (on this build host under
148+
`llvm-project/.../clang/lib/Headers/xtensa/tie/` plus the ace30 core config at
149+
`modules/hal/xtensa/.../soc/intel_ace30_ptl/xtensa/config/core-isa.h`). The
150+
Zephyr-SDK GCC that `ffmpeg.cmake` currently drives does **not** ship these headers.
151+
152+
Consequence: the `ff_*_init_xtensa` kernels must be written with `#if
153+
XCHAL_HAVE_HIFI*` intrinsics **and** a scalar `#else` fallback. Under the current
154+
GCC cross-build the guard is 0, so the fallback compiles (correct, but no speedup).
155+
Actual HiFi codegen requires cross-building FFmpeg with the LLVM Xtensa clang and
156+
adding the ace30 core-config include dir to the FFmpeg `CFLAGS` — the same
157+
toolchain SOF's own `src/math/*_hifi*.c` already rely on.
158+
159+
### Tier 1 — dominant, best return
160+
161+
1. **IMDCT**`mdct1024_fn` / `mdct128_fn` (short blocks) via `libavutil/tx`,
162+
driven from `imdct_and_windowing()` (`libavcodec/aac/aacdec_dsp_template.c:341-343`).
163+
One 1024-pt (or 8×128) inverse MDCT per channel per frame — the single largest
164+
compute. Dispatch: `AVTXContext`. HiFi = complex-FMA butterflies, via HiFi `tx`
165+
codelets or a bridge to SOF's HiFi3 FFT (`src/math/fft/`). Highest cost, most work.
166+
167+
2. **`AVFloatDSPContext` vector kernels** — pervasive for windowing / overlap-add /
168+
stereo, and the easiest high-leverage target:
169+
- `vector_fmul_window` — windowed overlap-add, ~10 call sites (`:354-419`); *the*
170+
per-sample AAC output kernel.
171+
- `vector_fmul` / `vector_fmul_reverse` — window application (`:235-308`).
172+
- `vector_fmul_scalar` — intensity-stereo gain (`:146`).
173+
- `butterflies_float` — M/S stereo (`:102`).
174+
One `ff_float_dsp_init_xtensa` (in `libavutil/`) covers all of them and also
175+
accelerates MP3 and Opus. **Start here** (lowest effort, cross-codec).
176+
177+
### Tier 2 — AAC-specific loops (in `aacdec_dsp_template.c`, HiFi'd inline in the fork)
178+
179+
3. **Inverse quantization `x^(4/3)`**`decode_spectrum_and_dequant()`
180+
(`libavcodec/aac/aacdec.c:1768`) + `dequant_scalefactors()` (`:41`): up to 1024
181+
coeffs, each `ff_cbrt_tab` lookup for `|x|^(4/3)` × scalefactor gain
182+
`2^(0.25·sf)`. HiFi = vectorised table gather + FMA (per-SFB gain is a
183+
`vector_fmul_scalar`).
184+
4. **TNS**`apply_tns()` (`:164`): LPC all-pole filter (order ≤ ~20) over spectral
185+
coeffs per window. HiFi = MAC filter (mirror `src/math/iir_df1_hifi*`).
186+
187+
### Tier 3 — HE-AAC only (skip unless SBR/PS enabled)
188+
189+
5. **SBR QMF**`libavcodec/sbrdsp.c`: `sbr_qmf_pre/post_shuffle`, `sbr_hf_gen`,
190+
`sbr_hf_g_filt`, `sbr_sum64x5` + a 64-pt QMF transform. Dispatch:
191+
`AACSBRDSPContext` (`sbrdsp.h`, `ff_sbrdsp_init`). Nearly as heavy as the base IMDCT.
192+
6. **PS**`libavcodec/aacps.c` `hybrid_analysis` / `stereo_interpolate` via
193+
`PSDSPContext` (`aacpsdsp.h`, `ff_psdsp_init`).
194+
195+
### Where the HiFi hooks land + priority
196+
197+
| Priority | Target | Dispatch / location | Effort |
198+
|---|--------|---------------------|--------|
199+
| 1 | `vector_fmul_window` + friends | `AVFloatDSPContext``ff_float_dsp_init_xtensa` (libavutil) | low; AAC+MP3+Opus |
200+
| 2 | IMDCT (mdct128/1024) | `libavutil/tx` — HiFi codelets or SOF-FFT bridge | high |
201+
| 3 | dequant `x^(4/3)` | `aac/aacdec.c` + dsp_template (cbrt gather+FMA) | med |
202+
| 4 | TNS | `apply_tns` (LPC MAC) | med |
203+
| 5 | SBR / PS | `sbrdsp`/`aacpsdsp` `_xtensa` inits | med, HE-AAC only |
204+
205+
AAC-LC order: **float_dsp (2) → IMDCT (1) → dequant (3) → TNS (4)**; SBR/PS only if
206+
HE-AAC. `float_dsp`/`tx`/`sbrdsp`/`psdsp` are `libavutil`/`libavcodec` and take
207+
`_xtensa` inits mirroring the existing `aarch64`/`x86` dirs; dequant/TNS are AAC
208+
decoder code patched in place. All on the `lgirdwood/FFmpeg` `sof-hifi` branch.
209+
137210
## Recommendations
138211

139212
1. **Quick wins we own** — hand-vectorise the four conversion loops (§A) with

0 commit comments

Comments
 (0)