Skip to content

feat(video transcription): add generated video transcriptions to dash preview#1614

Open
kronosmichall wants to merge 3 commits into
box:masterfrom
kronosmichall:enable_generated_video_transcriptions
Open

feat(video transcription): add generated video transcriptions to dash preview#1614
kronosmichall wants to merge 3 commits into
box:masterfrom
kronosmichall:enable_generated_video_transcriptions

Conversation

@kronosmichall
Copy link
Copy Markdown
Contributor

@kronosmichall kronosmichall commented Feb 26, 2026

The fetched extracted_text rep is a vtt file with AI-generated transcriptions.
Waiting for the transcriptions to be generated should not stop the video. It should load when available.
It should be also silently ignored if generating transcriptions failed.

Currently video transcription generation is enabled on dev and staging only

@kronosmichall kronosmichall requested a review from a team as a code owner February 26, 2026 12:35
@kronosmichall kronosmichall force-pushed the enable_generated_video_transcriptions branch from 05a2731 to 733c0a0 Compare March 2, 2026 11:53
@kronosmichall kronosmichall changed the title feat: add generated video transcriptions to dash preview feat(video transcription): add generated video transcriptions to dash preview Mar 2, 2026
Comment thread src/lib/viewers/media/DashViewer.js
Comment thread src/lib/__tests__/Preview-test.js
Comment thread src/lib/viewers/media/DashViewer.js
@kronosmichall kronosmichall force-pushed the enable_generated_video_transcriptions branch 3 times, most recently from 7d9dcf8 to f849667 Compare April 21, 2026 10:13
feat: Refactor dashviewer transcription handling to await
@kronosmichall kronosmichall force-pushed the enable_generated_video_transcriptions branch from f849667 to 7f16525 Compare April 24, 2026 09:52
Comment thread src/lib/constants.js
export const X_REP_HINT_DOC_THUMBNAIL = '[jpg?dimensions=1024x1024&paged=false]';
export const X_REP_HINT_IMAGE = '[jpg?dimensions=2048x2048,png?dimensions=2048x2048]';
export const X_REP_HINT_VIDEO_DASH = '[dash,mp4][filmstrip]';
export const X_REP_HINT_VIDEO_DASH = '[dash,mp4][filmstrip][extracted_text]';
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are other places where we define x rep hints. Have we added this there as well?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't see that for dash. everywhere dash rep is used with X_REP_HINT_VIDEO_DASH which i updated

The non-React subtitle menu uses the array index as each menu item's
`data-value`, and `DashViewer.handleSubtitle` reads `cache.media-subtitles`
as an INDEX into `this.textTracks`. The previous implementation re-sorted
`this.textTracks` by id when appending the new transcription track, which:

- broke the user's cached selection (the cached index resolved to a
  different track — e.g. a user with Spanish selected silently started
  seeing the auto-generated transcription on the next handleSubtitle), and
- produced duplicate `data-value` attributes in the Settings DOM when
  the new track sorted in the middle of existing entries (since
  `Settings.subtitles` is push-appended at the end while the menu item
  was inserted with the new sorted index).

Append new tracks to the END of `this.textTracks` and pass
`this.textTracks.length - 1` as the new menu index so `Settings.subtitles`,
the DOM `data-value`s, and `this.textTracks` stay in lockstep with whatever
order existed before.
Sibling rep loaders in DashViewer (loadFilmStrip, requestFilter, manifest
fetch) gate URL construction on the migrateAccessTokenToHeader feature
flag and prefer createContentUrlV2 when enabled. loadTranscription was
hard-coded to createContentUrlWithAuthParams, which would keep sending
the access token in URL query params for the transcription request even
after the rest of the viewer migrated to header-based auth.

Mirror the loadFilmStrip pattern so the transcription request stays
consistent with the rest of the viewer when the flag is rolled out.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants