Skip to content

Commit a00c30e

Browse files
lbliiiclaude
andauthored
[codex] docs: add long-form audio cutting guide (#2136)
* docs: add long-form audio cutting guide Signed-off-by: Lawrence Lane <llane@nvidia.com> * docs: clarify audio pretrain finalization Signed-off-by: Lawrence Lane <llane@nvidia.com> * docs: address reviewer feedback on long-form audio cutting guide - state that source IDs must be unique after tar-key normalization (make_snippet_id replaces . / \ with _, so session.1 and session_1 collide) - clarify that a missing segments key fails OverlapFilterStage validation, while an empty list is accepted and yields zero snippets - warn that dry runs must use a fresh, nonexistent tar path so preview manifests are not reconciled against a stale archive - document the missing_audio/corrupted_audio counters as sparse and recommend defensive .get() access; drop the misleading zero-valued key from the example metrics JSON Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Lawrence Lane <llane@nvidia.com> --------- Signed-off-by: Lawrence Lane <llane@nvidia.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 0e0caa2 commit a00c30e

7 files changed

Lines changed: 416 additions & 0 deletions

File tree

fern/versions/main.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -474,6 +474,9 @@ navigation:
474474
- page: ALM Tutorial
475475
path: ./main/pages/curate-audio/tutorials/alm.mdx
476476
slug: alm
477+
- page: Long-Form Audio Cutting for ALM Pretraining
478+
path: ./main/pages/curate-audio/tutorials/audio-pretrain.mdx
479+
slug: audio-pretrain
477480
- page: ReadSpeech Tutorial
478481
path: ./main/pages/curate-audio/tutorials/readspeech.mdx
479482
slug: readspeech

fern/versions/main/pages/curate-audio/index.mdx

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -140,4 +140,11 @@ quality-filtering
140140
basic-workflow
141141
</Card>
142142

143+
<Card title="Long-Form Audio Cutting" href="/curate-audio/tutorials/audio-pretrain">
144+
Turn long-form diarized recordings into mono, resampled snippet archives and manifests for audio language model pretraining
145+
alm-pretraining
146+
long-form-audio
147+
webdataset
148+
</Card>
149+
143150
</Cards>

fern/versions/main/pages/curate-audio/process-data/alm/data-builder.mdx

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -124,3 +124,4 @@ The `stats` dictionary contains the following counters:
124124
- **[ALM Overlap Filtering](/curate-audio/process-data/alm/overlap-filtering)**: Next stage in the ALM pipeline
125125
- **[ALM Pipeline Concepts](/about/concepts/audio/alm-pipeline)**: Architectural overview
126126
- **[ALM Tutorial](/curate-audio/tutorials/alm)**: End-to-end walkthrough with sample data
127+
- **[Long-Form Audio Cutting](/curate-audio/tutorials/audio-pretrain)**: Cut selected diarized time ranges into reusable snippet audio and tar manifests

fern/versions/main/pages/curate-audio/process-data/alm/overlap-filtering.mdx

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -101,3 +101,4 @@ Monitor the yield by comparing `filtered_dur` to `total_dur_window` in the outpu
101101
- **[ALM Data Builder](/curate-audio/process-data/alm/data-builder)**: Previous stage in the ALM pipeline
102102
- **[ALM Pipeline Concepts](/about/concepts/audio/alm-pipeline)**: Architectural overview
103103
- **[ALM Tutorial](/curate-audio/tutorials/alm)**: End-to-end walkthrough with sample data
104+
- **[Long-Form Audio Cutting](/curate-audio/tutorials/audio-pretrain)**: Filter diarization overlap before extracting physical ALM pretraining snippets

fern/versions/main/pages/curate-audio/tutorials/alm.mdx

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -297,6 +297,7 @@ The `ALMManifestReader` discovers all `.jsonl` and `.json` files in the director
297297

298298
After completing this tutorial, explore:
299299

300+
- **[Long-Form Audio Cutting](/curate-audio/tutorials/audio-pretrain)**: Extract, resample, and package bounded audio snippets when you need physical audio assets rather than metadata windows
300301
- **[ALM Data Builder](/curate-audio/process-data/alm/data-builder)**: Detailed reference for window construction
301302
- **[ALM Overlap Filtering](/curate-audio/process-data/alm/overlap-filtering)**: Detailed reference for overlap filtering
302303
- **[ALM Pipeline Concepts](/about/concepts/audio/alm-pipeline)**: Architectural overview

fern/versions/main/pages/curate-audio/tutorials/audio-pretrain.mdx

Lines changed: 396 additions & 0 deletions
Large diffs are not rendered by default.

fern/versions/main/pages/curate-audio/tutorials/index.mdx

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,4 +34,11 @@ windowing
3434
speaker-diarization
3535
</Card>
3636

37+
<Card title="Long-Form Audio Cutting" href="/curate-audio/tutorials/audio-pretrain">
38+
Extract bounded mono and resampled snippets from long-form diarized audio, then package them as a training manifest and WebDataset-compatible tar.
39+
alm-pretraining
40+
snippet-extraction
41+
webdataset
42+
</Card>
43+
3744
</Cards>

0 commit comments

Comments
 (0)