You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: backlog/docs/architecture/transcript-import.md
+21-6Lines changed: 21 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -15,28 +15,43 @@ id: doc-008
15
15
16
16
## Automated YouTube Transcription (Async)
17
17
18
-
For missing transcripts of videos hosted on YouTube, we use a local asynchronous pipeline powered by `zdots-ctx` and `whisper.cpp`.
18
+
For missing transcripts of videos hosted on YouTube, we use a local asynchronous pipeline powered by `zdots-ctx` and `whisper.cpp`. This process is designed to be highly resilient and fully resumable.
19
19
20
20
1.**Enqueue Pending Videos:**
21
21
Identify all pending `video_assets` with a YouTube ID and enqueue them to the local `zdots-ctx` worker.
22
22
```bash
23
23
./bin/batch_ztranscribe.rb
24
24
```
25
-
2.**Start the Background Worker:**
26
-
Ensure the worker is running to process the queue asynchronously.
25
+
2.**Start the Background Worker (Safely):**
26
+
Ensure the worker is running to process the queue asynchronously.
27
+
**CRITICAL:** When running in the background, you MUST redirect stdin from `/dev/null`. If you do not, the underlying `ffmpeg` process (used by `yt-transcribe` to convert audio) will attempt to read from the terminal, immediately suspending the entire worker process.
3.**Pausing and Resuming (Interruption Recovery):**
33
+
If the process is interrupted (e.g., power loss, internet outage, or manual kill):
34
+
- Clear any jobs that were stuck in the "running" state when the interruption occurred:
35
+
```bash
36
+
zdots-ctx clear-stale-jobs
37
+
```
38
+
- Restart the worker using the safe command from Step 2. It will automatically pick up where it left off.
39
+
4. **Stage Completed Transcripts:**
31
40
As the worker finishes downloading and transcribing to `~/Downloads/transcripts/`, map them back to their canonical `video_asset_id` and stage them for ingestion.
32
41
```bash
33
42
./bin/stage_completed_transcripts.rb
34
43
```
35
-
4.**Ingest Staged Transcripts:**
44
+
5. **Ingest Staged Transcripts:**
36
45
Use the standard pipeline to ingest the staged files into `_data/transcripts/`.
Once ingested, prepare the newly available transcripts for the conversational audit to extract SEO metadata, separate speakers, and pull durable insights.
51
+
```bash
52
+
bundle exec rake audit:prepare_wave
53
+
```
54
+
Then activate the `transcript-conversational-audit` skill to process the generated prompts.
0 commit comments