Repository navigation
feat(player): play the transcript as WebVTT caption tracks - #690
Merged
Merged
Conversation
A video's <video> now carries its transcript as a <track kind="captions">, and timed supplementary captions and translations as more tracks, in the workspace player and on the public page. The tracks are IIIF's WebVTT with links signed like media links, so visitors and share links can load them. In the workspace the existing overlay still draws captions; the track stays undrawn there and the browser draws it only in full screen or picture in picture, so captions never show twice. The public page has no overlay: the browser draws them, on by default, with a Captions button to turn them off. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QFUNzy1QZy5HhKAyJxTAYJ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by Adeel · project thread
Description
Before: a video's captions were only an overlay the page drew, so the browser's captions menu, full screen, picture in picture and assistive technology never saw them, and the public page had no captions at all.
After: the
<video>carries the transcript as a WebVTT<track kind="captions">, then each timed supplementary captions or translation file as another track (translations assubtitles).C) still draws the overlay. The track stays loaded but undrawn while the overlay shows, and the browser draws it only in full screen or picture in picture, so captions never show twice. Turning Captions off turns the track off.GET /api/v1/recordings/<id>/playerandGET /api/v1/public/recordings/<id>list the tracks ascaptions.How: the tracks use IIIF's existing WebVTT routes (
/iiif/<id>/transcript.vtt,/iiif/<id>/files/<fid>.vtt), signed with the existingiiif.signfor the media-link lifetime. A newCaptionTrackscomponent renders them and sets each track's mode.Motivation and Context
Item from the personal feature plan sheet (Library CSV export + captions
<track>).Screenshots (if appropriate):
n/a
Steps to reproduce (if appropriate):
Open a transcribed video, turn Captions on, then go full screen: the browser draws the transcript lines. On its public page, press Captions under the video.
Types of changes
!in the title)Checklist:
make hookschecks it on commit).tests/api/test_caption_tracks.py,__tests__/caption-tracks.test.tsx).docs/accordingly (docs/video.md#captions,docs/access.md).## UnreleasedinCHANGELOG.md. Written aschangelog.d/captions-track.md, per build(release): changelog fragments in changelog.d/ instead of one Unreleased section #581.Done questions (AGENTS.md)
.envedit or manual step.Cdrive the track; one button on the public page.docs/requirements.mdname the checks; "refine later" indocs/video.md(a translation picker in Lens's own controls; the audio-only embed).make openapi).<track>; no new dependencies.Local CI: the container runs several times slower than usual, so I ran the changed tests directly:
test_caption_tracks.py,test_iiif.py,test_public_pages.py,test_video.py(21 passed),caption-tracks.test.tsx(6 passed). ruff, mypy and tsc are clean, andmake openapishows no drift. GitHub CI decides the merge.Generated by Claude Code