Skip to content

Commit cb3957c

Browse files
author
Zenflow
committed
fix false gap regions: close cracks between adjacent seg_windows
The gap detection was firing on cracks between consecutive segment windows where unmatched (interpolated) segments sat between two matched anchors. Whisper words belonging to real content fell in the crack and were mis-classified as out_of_transcript. Root cause: seg_window[i].end < seg_window[i+1].start whenever the matched segment's LCS span ended before the next segment's start position. The 8s MIN_AD_GAP threshold was then applied to this crack, producing fake 'gaps'. Fix: after building all seg_windows, extend each window's end to the next window's start (max of current end and next start). The gap detector then only sees spans genuinely not covered by any window. Impact on ep152: Before: 88.6% assigned, 6 gap regions (5 fake + 1 real) After: 99.8% assigned, 1 gap region (real StepStone ad, 36s intro)
1 parent f441758 commit cb3957c

1 file changed

Lines changed: 14 additions & 0 deletions

File tree

scripts/podcast_to_moshi_dataset.py

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -546,6 +546,20 @@ def min_score(n: int) -> float:
546546
re = seg_raw_end[si] or 0.0
547547
seg_windows.append((rs, re, seg.get("speaker", ""), seg.get("text", "").strip()))
548548

549+
# ── Close cracks between adjacent windows ────────────────────────────────
550+
# After interpolation, consecutive segment windows may not be contiguous:
551+
# the end of window[i] < start of window[i+1]. Any whisper word that
552+
# falls in this crack gets misclassified as out_of_transcript even though
553+
# it belongs to real content. Extend each window's end to the next
554+
# window's start so cracks are closed. The gap-detection below then only
555+
# sees spans that are genuinely not covered by any window.
556+
closed = []
557+
for i, (rs, re, spk, txt) in enumerate(seg_windows):
558+
if i + 1 < len(seg_windows):
559+
re = max(re, seg_windows[i + 1][0])
560+
closed.append((rs, re, spk, txt))
561+
seg_windows = closed
562+
549563
# ── Detect real gaps ≥ MIN_AD_GAP ─────────────────────────────────────────
550564
gap_regions = []
551565
if seg_windows[0][0] > MIN_AD_GAP:

0 commit comments

Comments
 (0)