|
| 1 | +# Multi-Slot Wake-On-Voice (WOV) Architecture & Arbitration |
| 2 | + |
| 3 | +## Overview |
| 4 | + |
| 5 | +The Multi-Slot Wake-On-Voice (WOV) subsystem in Sound Open Firmware (SOF) provides a scalable, low-power architecture for running multiple concurrent keyphrase/voice detectors (e.g., Male, Female, and Child voice frequency profiles) over a shared audio capture source. |
| 6 | + |
| 7 | +It combines energy-based voice activity gating, multi-channel ring buffering (KPB), frequency-differentiated detectors, and exclusive host PCM stream arbitration. |
| 8 | + |
| 9 | +--- |
| 10 | + |
| 11 | +## System Architecture |
| 12 | + |
| 13 | +```mermaid |
| 14 | +graph TD |
| 15 | + subgraph Capture_Input["Pipeline 100: Capture & Gating (Core 0)"] |
| 16 | + DAI["DAI Copier (3.5mm HDA Jack / DMIC)"] --> VAD["VAD Gate (Energy Estimator)"] |
| 17 | + VAD --> MIXIN["Mixin (1-to-3 Fan-out)"] |
| 18 | + end |
| 19 | +
|
| 20 | + subgraph Slot0["Pipeline 101: Slot 0 (Core 0)"] |
| 21 | + MIXOUT0["Mixout 0"] --> KPB0["KPB 0 (Ring Buffer)"] |
| 22 | + KPB0 -- "sel_sink" --> WOV0["Detector 0: Male (80-170 Hz)"] |
| 23 | + KPB0 -- "host_sink" --> ARB_IN0["Arbiter Input 0"] |
| 24 | + end |
| 25 | +
|
| 26 | + subgraph Slot1["Pipeline 102: Slot 1 (Core 0)"] |
| 27 | + MIXOUT1["Mixout 1"] --> KPB1["KPB 1 (Ring Buffer)"] |
| 28 | + KPB1 -- "sel_sink" --> WOV1["Detector 1: Female (175-270 Hz)"] |
| 29 | + KPB1 -- "host_sink" --> ARB_IN1["Arbiter Input 1"] |
| 30 | + end |
| 31 | +
|
| 32 | + subgraph Slot2["Pipeline 103: Slot 2 (Core 1)"] |
| 33 | + MIXOUT2["Mixout 2"] --> KPB2["KPB 2 (Ring Buffer)"] |
| 34 | + KPB2 -- "sel_sink" --> WOV2["Detector 2: Child (275-500 Hz)"] |
| 35 | + KPB2 -- "host_sink" --> ARB_IN2["Arbiter Input 2"] |
| 36 | + end |
| 37 | +
|
| 38 | + subgraph Arbiter_Output["Pipeline 104: Arbitration & Host Capture (Core 0)"] |
| 39 | + ARB_IN0 --> ARB["WOV Arbiter"] |
| 40 | + ARB_IN1 --> ARB |
| 41 | + ARB_IN2 --> ARB |
| 42 | + ARB --> HOST["Host Copier (Card 0 PCM 11)"] |
| 43 | + end |
| 44 | +
|
| 45 | + subgraph Messaging["AMS Inter-Module Control"] |
| 46 | + WOV0 -- "AMS_WOV_DETECT (Slot 0)" --> ARB |
| 47 | + WOV1 -- "AMS_WOV_DETECT (Slot 1)" --> ARB |
| 48 | + WOV2 -- "AMS_WOV_DETECT (Slot 2)" --> ARB |
| 49 | + ARB -- "AMS_WOV_CTRL (PAUSE/RESUME)" --> WOV0 |
| 50 | + ARB -- "AMS_WOV_CTRL (PAUSE/RESUME)" --> WOV1 |
| 51 | + ARB -- "AMS_WOV_CTRL (PAUSE/RESUME)" --> WOV2 |
| 52 | + end |
| 53 | +
|
| 54 | + MIXIN --> MIXOUT0 |
| 55 | + MIXIN --> MIXOUT1 |
| 56 | + MIXIN --> MIXOUT2 |
| 57 | +
|
| 58 | + style VAD fill:#2d5a27,stroke:#333,stroke-width:2px |
| 59 | + style ARB fill:#1c4966,stroke:#333,stroke-width:2px |
| 60 | + style WOV0 fill:#663300,stroke:#333,stroke-width:1px |
| 61 | + style WOV1 fill:#660033,stroke:#333,stroke-width:1px |
| 62 | + style WOV2 fill:#003366,stroke:#333,stroke-width:1px |
| 63 | +``` |
| 64 | + |
| 65 | +--- |
| 66 | + |
| 67 | +## Event Sequence & Arbitration Flow |
| 68 | + |
| 69 | +```mermaid |
| 70 | +sequenceDiagram |
| 71 | + autonumber |
| 72 | + participant Audio as Audio Source (HDA Jack / DMIC) |
| 73 | + participant VAD as VAD Gate |
| 74 | + participant KPB as KPB (Slots 0..2) |
| 75 | + participant WOV as WOV Detectors (0..2) |
| 76 | + participant Arbiter as WOV Arbiter |
| 77 | + participant Host as Host PCM (arecord) |
| 78 | +
|
| 79 | + Note over Audio, Host: Initial Listening State (All slots active) |
| 80 | + Audio->>VAD: Audio Stream |
| 81 | + VAD->>VAD: Check energy > threshold |
| 82 | + VAD->>KPB: Forward PCM frames |
| 83 | + KPB->>WOV: Continuous audio copy |
| 84 | + WOV->>WOV: Track zero-crossings & energy |
| 85 | +
|
| 86 | + Note over WOV, Arbiter: Keyword Detection (e.g. Female Voice 220Hz on Slot 1) |
| 87 | + WOV->>Arbiter: Send AMS_WOV_DETECT (slot_id = 1) |
| 88 | + Arbiter->>Host: Switch active stream to KPB 1 host_sink |
| 89 | + Arbiter->>WOV: Broadcast AMS_WOV_CTRL (PAUSE sibling slots 0 & 2) |
| 90 | + KPB->>Host: Drain pre-roll buffered audio to host |
| 91 | +
|
| 92 | + Note over Audio, Host: Capture Stream Finish / Stop Command |
| 93 | + Host->>Arbiter: Trigger STOP / CLOSE |
| 94 | + Arbiter->>WOV: Broadcast AMS_WOV_CTRL (RESUME all slots) |
| 95 | + Note over Audio, Host: System returns to initial listening state |
| 96 | +``` |
| 97 | + |
| 98 | +--- |
| 99 | + |
| 100 | +## Component Roles & Responsibilities |
| 101 | + |
| 102 | +### 1. `vad_gate` (Voice Activity Detector Gate) |
| 103 | +- **File**: `src/audio/vad_gate/vad_gate.c` |
| 104 | +- **Role**: First-stage low-power energy estimator. Drains silence to prevent DAI DMA stall while returning `PPL_STATUS_PATH_STOP` during silence to keep downstream DSP pipelines idled. |
| 105 | + |
| 106 | +### 2. `mixin_mixout` (Fan-Out Mixer) |
| 107 | +- **File**: `src/audio/mixin_mixout/mixin_mixout.c` |
| 108 | +- **Role**: Duplicates input capture audio from Pipeline 100 into parallel detector pipelines 101, 102, and 103. |
| 109 | + |
| 110 | +### 3. `kpb` (Keyphrase Buffer) |
| 111 | +- **File**: `src/audio/kpb.c` |
| 112 | +- **Role**: Continuous ring-buffer maintaining history (pre-roll). In listening mode, forwards real-time audio to `sel_sink` (`detect_test`). Upon keyword detection, switches to high-speed draining over `host_sink` to host memory. |
| 113 | + |
| 114 | +### 4. `detect_test` (`wov` detector) |
| 115 | +- **File**: `src/samples/audio/detect_test.c` |
| 116 | +- **Role**: Frequency-differentiated keyphrase detector stub: |
| 117 | + - **Slot 0**: Male voice range (80 Hz – 170 Hz) |
| 118 | + - **Slot 1**: Female voice range (175 Hz – 270 Hz) |
| 119 | + - **Slot 2**: Child voice range (275 Hz – 500 Hz) |
| 120 | +- Sends `AMS_WOV_DETECT` message containing `slot_id` to `wov_arbiter` upon match. |
| 121 | + |
| 122 | +### 5. `wov_arbiter` (Arbitration Engine) |
| 123 | +- **File**: `src/audio/wov_arbiter/wov_arbiter.c` |
| 124 | +- **Role**: Accepts 3 `host_sink` inputs from KPB 0..2 and routes the winning slot's pre-roll buffer to `host-copier`. Broadcasts `AMS_WOV_CTRL` messages (`PAUSE`/`RESUME`) to ensure only one slot drains while preserving listening state. |
| 125 | + |
| 126 | +--- |
| 127 | + |
| 128 | +## Topology & Configuration |
| 129 | + |
| 130 | +### Kconfig Options |
| 131 | +Enable the required components in `app/prj.conf` or board configuration (`app/boards/intel_adsp_cavs25.conf`): |
| 132 | + |
| 133 | +```kconfig |
| 134 | +CONFIG_COMP_WOV_ARBITER=y |
| 135 | +CONFIG_COMP_VAD_GATE=y |
| 136 | +CONFIG_COMP_KPB=y |
| 137 | +CONFIG_COMP_MIXIN_MIXOUT=y |
| 138 | +CONFIG_COMP_KWD_DETECT=y |
| 139 | +CONFIG_AMS=y |
| 140 | +``` |
| 141 | + |
| 142 | +### Topology 2 Syntax |
| 143 | +Topology configuration file: `tools/topology/topology2/dmic-wov-multi-manifest.conf` |
| 144 | + |
| 145 | +To compile topology into `.tplg` binary: |
| 146 | + |
| 147 | +```bash |
| 148 | +ALSA_CONFIG_DIR=tools/topology/topology2 \ |
| 149 | +alsatplg -I tools/topology/topology2 -p \ |
| 150 | +-c tools/topology/topology2/dmic-wov-multi-manifest.conf \ |
| 151 | +-o /lib/firmware/intel/sof-ipc4-tplg/sof-hda-generic-4ch.tplg |
| 152 | +``` |
| 153 | + |
| 154 | +--- |
| 155 | + |
| 156 | +## Testing & Verification Workflow |
| 157 | + |
| 158 | +### 1. Hardware Setup (Spider DUT) |
| 159 | +Audio is injected via the 3.5mm HDA Jack Mic input (`Card 0 PCM 11`): |
| 160 | + |
| 161 | +```bash |
| 162 | +ssh root@spider 'amixer -c 0 sset Capture 100% cap; amixer -c 0 sset "Mic Boost" 3' |
| 163 | +``` |
| 164 | + |
| 165 | +### 2. Frequency Sweep Test Execution |
| 166 | +Run capture on Spider while generating tone sweeps from testing host: |
| 167 | + |
| 168 | +```bash |
| 169 | +# Start capture on DUT |
| 170 | +ssh root@spider 'arecord -Dhw:0,11 -r 16000 -c 1 -f S16_LE -d 15 /tmp/wov_multi_capture.wav' & |
| 171 | + |
| 172 | +# Play frequency sweep (Male 120Hz -> Female 220Hz -> Child 350Hz) |
| 173 | +speaker-test -Dplughw:1,0 -c 1 -t sine -f 120 -l 2 |
| 174 | +speaker-test -Dplughw:1,0 -c 1 -t sine -f 220 -l 2 |
| 175 | +speaker-test -Dplughw:1,0 -c 1 -t sine -f 350 -l 2 |
| 176 | +``` |
| 177 | + |
| 178 | +### 3. Log Inspection (`mtrace`) |
| 179 | +Read trace output on Spider to observe frequency evaluations and arbitration events: |
| 180 | + |
| 181 | +```bash |
| 182 | +ssh root@spider 'grep -i -E "kd_test|TRIGGERED|wov_arb" /tmp/fw_wov_mtrace.log' |
| 183 | +``` |
| 184 | + |
| 185 | +**Expected Log Output**: |
| 186 | + |
| 187 | +```text |
| 188 | +[ 34.094740] kd_test.test_keyword_new: comp:1 0xd dev_id=0xd pipeline_id=1 (Slot 0 - MALE) |
| 189 | +[ 34.096583] kd_test.test_keyword_new: comp:3 0x1000d dev_id=0x1000d pipeline_id=3 (Slot 1 - FEMALE) |
| 190 | +[ 34.092470] kd_test.test_keyword_new: comp:4 0x2000d dev_id=0x2000d pipeline_id=4 (Slot 2 - CHILD) |
| 191 | +
|
| 192 | +[ 37.014193] kd_test.default_detect_test: comp:1 0xd kd_test eval: slot=0 (MALE), freq=120 Hz |
| 193 | +[ 52.000996] kd_test.default_detect_test: comp:4 0x2000d kd_test entry: slot=2 (CHILD), freq=375 Hz |
| 194 | +
|
| 195 | +[ 37.257318] wov_arbiter.wov_arb_trigger: comp:2 0x10 wov_arb: stream stopped, resuming all slots |
| 196 | +[ 37.257335] kd_test.on_wov_ctrl: comp:4 0x2000d kd: resumed |
| 197 | +``` |
0 commit comments