adeelahmad · Oct 19, 2026 · Oct 19, 2026
diff --git a/README.md b/README.md
@@ -186,6 +186,8 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p
   beside the asset as a translation whose lines keep their times, found by search like an uploaded one.
   A date said ("let's meet next Tuesday at 3") becomes a calendar event in one click,
   prefilled from the line and added only once you approve it ([propose an event](docs/processing.md#propose-an-event)).
+  Videos play their transcript, and timed captions and translations added beside them, as captions the browser
+  itself shows (its captions menu, full screen, picture in picture), in the workspace and on public pages.
   ([Graph](docs/graph.md), [entity pages](docs/notes.md#entity-pages),
   [similar content](docs/processing.md#similar-content), [graph history](docs/graph-history.md),
   [topics](docs/topics.md), [video](docs/video.md), [NSFW classification](docs/nsfw.md),
@@ -194,7 +196,8 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p
   [your own names](docs/processing.md#your-own-names-in-transcription),
   [cleaning up transcripts](docs/processing.md#cleaning-up-transcripts), [dates](docs/processing.md#dates),
   [unsure words](docs/processing.md#unsure-words),
-  [translations](docs/api.md#translating-a-transcript); [checks](docs/requirements.md#understand-it))
+  [translations](docs/api.md#translating-a-transcript), [captions](docs/video.md#captions);
+  [checks](docs/requirements.md#understand-it))
 - **Learn from it.** Make a two-host podcast episode about what you pick, every claim cited and checked against its
   source, read aloud in two voices, by Kokoro on this machine or a speech provider. Turn a voice session or a meeting into a technical spec, in a note linked to it,
   that keeps every decision, every no and every constraint in the speaker's own words, linked to when they said them,

diff --git a/changelog.d/captions-track.md b/changelog.d/captions-track.md
@@ -0,0 +1,8 @@
+- **Captions the browser knows about.** A video now plays its transcript as a WebVTT `<track>`, in the workspace and on
+  its public page, so the browser's captions menu, full screen, picture in picture and assistive technology get it,
+  with timed supplementary captions and translations as more tracks to pick. The links are signed, so they work for
+  visitors and share links too. In the workspace the Captions toggle still draws the line and its speaker over the
+  video, and the browser draws the track only where that can't be seen (full screen, picture in picture), so captions
+  never show twice; on the public page the browser draws them once the visitor turns them on with the new
+  Captions button. `GET /api/v1/recordings/<id>/player` and `GET /api/v1/public/recordings/<id>` list the tracks as
+  `captions` ([Captions](docs/video.md#captions)).
diff --git a/docs/access.md b/docs/access.md
@@ -51,7 +51,8 @@ and translations are open when its text is, indexes when its index is, and thumb
 attachments (release forms, notes, anything else) always need permission. Decided with the project owner. People with
 permission download all of them; the public page lists the files a visitor may download and says how many more need
 permission, which signed-in visitors may ask for. IIIF lists every file, the closed ones behind the Authorization
-Flow.
+Flow. A video's player offers the ones it may read that say when their lines are as caption tracks, beside the
+transcript's ([Captions](video.md#captions)); with downloads off they still play, as the transcript does.
 
 ## Who is asking
 

diff --git a/docs/requirements.md b/docs/requirements.md
@@ -234,6 +234,7 @@ request, to another check that proves the same thing; loosening a requirement or
 | Dates said in an asset read as days ("next Tuesday", "last week"), off by default, with dates that read two ways left out | [test_dates.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_dates.py) `test_dates_are_read_as_days`, `test_what_could_be_two_dates_is_left_out`, `test_off_by_default_reads_none` |
 | A transcript translated into another language only when someone asks, the estimate first: in batches of lines that keep their times and speaker, the transcript marked as data, every call in the activity ledger and within budgets, kept as a translation file like an uploaded one; through the web app's Files tab, the API, MCP and the assistant (asking first) | [test_translations.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_translations.py) `test_translating_writes_a_translation_file`, `test_the_transcript_reaches_the_model_as_data`, `test_lines_go_in_batches_and_keep_their_place`, `test_a_spent_budget_refuses`, `test_the_estimate_comes_first`, `test_what_stops_a_translation`, `test_agents_translate`, `test_the_assistant_asks_first`; [recording-translate.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/recording-translate.test.tsx) `is the first the browser reads that the transcript isn't in`, `translates into the browser's language with Enter, after showing the estimate` |
 | A date said in an asset is proposed as a calendar event in one click, prefilled from its line (the day, a time said, the line as its title, who it names), and added only once the person approves it, signed in; declined, nothing is added; offered only to admins with a calendar connection that may add events, and agents propose over MCP but can't approve | [test_event_proposals.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_event_proposals.py) `test_a_date_said_is_proposed_and_added_only_once_approved`, `test_a_declined_proposal_adds_nothing`, `test_offered_only_to_admins_with_a_calendar_that_adds_events`, `test_agents_propose_over_mcp_but_cannot_approve`, `test_the_time_of_day_is_read_from_the_line`; [said-dates.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/said-dates.test.tsx) `proposes an event from a date in one click, and says it waits in Approvals`, `offers no Propose event without a calendar that adds events, and shows nothing without dates` |
+| Videos play their transcript and timed supplementary captions and translations as WebVTT tracks with signed links, in the workspace (drawn by the browser only where the overlay can't be seen) and on the public page (off until a button turns them on), with only what the viewer may read | [test_caption_tracks.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_caption_tracks.py) `test_the_player_offers_the_transcript_and_timed_files_as_tracks`, `test_the_public_page_offers_what_the_visitor_may_read_as_tracks`; jest [caption-tracks.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/caption-tracks.test.tsx) `stay undrawn while the page's own overlay shows the captions, and are drawn full screen`, `plays its text as captions the browser draws once the visitor turns them on` |
 
 ## Learn from it
 

diff --git a/docs/video.md b/docs/video.md
@@ -82,6 +82,35 @@ Video recordings open in their own layout.
   - **Shots:** a grid of shots.
 - Everything seeks the one video, and the layout works down to phone widths.
 
+## Captions
+
+A video plays its text as WebVTT `<track>`s, so the browser's own captions menu, full screen and picture in picture
+show them and assistive technology can read them, in the workspace and on its public page alike:
+
+- the transcript comes first, as `kind="captions"` in the recording's language when it's known, and is the default
+  while captions are on;
+- then each supplementary transcript, captions or translation that says when its lines are
+  ([files](api.md#files)), in the order they were added; a translation is `kind="subtitles"`. Pick them from the
+  browser's captions menu.
+
+The tracks are IIIF's WebVTT (`/iiif/<id>/transcript.vtt`, `/iiif/<id>/files/<file>.vtt`), with links signed for as long
+as media links last, because a `<track>` can't send the Authorization header. `GET /api/v1/recordings/<id>/player`
+lists them as `captions`, and so does `GET /api/v1/public/recordings/<id>`, with only what that visitor may read and
+only when they may play the media. A share link (`?s=`) gets the transcript's track; supplementary files stay with
+the namespace's members, as they do in the Files tab.
+
+Captions show once, never twice:
+
+- **Workspace (the video recording page):** the Captions toggle (`C`) draws the line and its speaker over the video, as
+  before. Its track stays loaded but undrawn while that overlay shows, and the browser draws it when the video is full
+  screen or picture in picture, where the overlay can't be seen. Turning Captions off turns the track off too.
+- **Public page:** there is no overlay, so the browser draws the captions. They're off when the page opens, as before,
+  and the Captions button under the video turns them on.
+
+Refine later: the embeddable player (`/embed/<id>`, `/s/<code>`) plays sound beside its transcript, without a video,
+so it has no tracks; Lens's own controls have no menu to pick a translation's track (the browser's menu does, where it
+shows its own controls); and a share link's player could offer the supplementary tracks its link opens.
+
 `The prototype's end-to-end browser test (`tests/e2e_video.py`, kept for reference in the history)` runs it end to end in Chromium, against a real server with face recognition on. It
 checks:
 - the lanes, and seeking from shots, lanes, the transcript and the grid

diff --git a/fastapi_backend/app/api/media.py b/fastapi_backend/app/api/media.py
@@ -19,7 +19,9 @@
 from collections.abc import Container
 from typing import Any, cast
 
+from app.config import settings
 from app.core.security import sign_path
+from app.domain import iiif
 from app.domain.store import API
 
 MEDIA_RX = re.compile(rf"^{re.escape(API)}/recordings/\d+/(audio|media|pdf|enhanced|wordcloud\.svg|frames/[\w.-]+|files/\d+/download)$")
@@ -71,3 +73,18 @@ def sign_page_links(page: str, recordings: Container[int], full: bool = False) -
     """Sign the media links (audio, video, frames, word clouds) of these recordings in a page, including inside its
     embedded JSON. Links to any other recording stay unsigned: the page's text may contain them."""
     return MEDIA_IN_HTML.sub(lambda m: sign_path(m.group(0), full=full) if int(m.group(1)) in recordings else m.group(0), page)
+
+
+TRACK_RX = re.compile(r"^/iiif/(\d+)/(?:transcript|files/(\d+))\.vtt$")
+
+
+def sign_tracks(cfg: dict[str, Any], tracks: list[dict[str, Any]]) -> list[dict[str, Any]]:
+    """Sign the links of a player's caption tracks (iiif.caption_tracks) as IIIF's probe service signs them, for as long
+    as media links last: a <track> can't send the Authorization header either, and share links and visitors have none.
+    Only hand them to someone who may read what they open."""
+    for t in tracks:
+        m = TRACK_RX.match(t["url"])
+        if m:
+            what = f"vtt{m.group(2)}" if m.group(2) else "transcript"
+            t["url"] = f"{t['url']}?{iiif.sign(cfg, int(m.group(1)), what, settings.MEDIA_URL_EXPIRE_SECONDS)}"
+    return tracks
diff --git a/fastapi_backend/app/api/v1/routes/public.py b/fastapi_backend/app/api/v1/routes/public.py
@@ -8,7 +8,7 @@
 from fastapi import APIRouter, BackgroundTasks, HTTPException, Query
 
 from app.api.deps import Acl, Cfg, Db, Writer, domain_errors
-from app.api.media import sign_url
+from app.api.media import sign_tracks, sign_url
 from app.domain import access as acc
 from app.domain import public, store
 from app.domain.store import R
@@ -110,6 +110,7 @@ def get_public_recording(rid: int, acl: Acl, db: Db, cfg: Cfg) -> PublicRecordin
             p["image"], p["thumb"] = sign_url(p["image"], member), sign_url(p["thumb"], member)
     for f in d["files"]:  # and only the files they may download
         f["url"] = sign_url(f["url"], save=owner)
+    d["captions"] = sign_tracks(cfg, d["captions"])  # and the text they may read, as caption tracks
     return PublicRecording.model_validate(d)
 
 

diff --git a/fastapi_backend/app/api/v1/routes/recordings.py b/fastapi_backend/app/api/v1/routes/recordings.py
@@ -15,16 +15,18 @@
 
 from app.api import evidence as evidence_api
 from app.api.deps import Access, Acl, Cfg, CurrentUser, Db, Writer, domain_errors
-from app.api.media import sign_url, sign_urls
+from app.api.media import sign_tracks, sign_url, sign_urls
 from app.api.streaming import LOCKED, file_response, range_response
 from app.domain import access as acc
 from app.domain import (
     analyze,
     auth,
     deletion,
+    documents,
     enhance,
     events_content,
     hierarchy,
+    iiif,
     ipgroups,
     jobs,
     keyring,
@@ -39,6 +41,7 @@
     video,
 )
 from app.domain import fields as fieldmod
+from app.domain import files as filemod
 from app.domain import metadata as md
 from app.domain.store import API, DB
 from app.schemas.common import AccessLevel, Ok
@@ -622,7 +625,12 @@ def get_player(rid: int, acl: Acl, db: Db, cfg: Cfg, s: str = "") -> Player:
     data = render.player_data(db, rid, audio_link(db, cfg, rid, s))
     data["nsfw"] = _nsfw(db, cfg, rec, rid)
     data["unsure_below"] = cfg["transcribe"].get("confidence_highlight") or None
-    return sign_urls(data, full=acl.member(rec))
+    member = acl.member(rec)
+    # its text as WebVTT tracks for the <video>: a document's has no times; a share link doesn't open its files
+    timed = rec.get("source") not in documents.KINDS
+    files = filemod.of(db, rid) if member and timed else []
+    data["captions"] = sign_tracks(cfg, iiif.caption_tracks(rid, rec.get("language"), files, timed and bool(data["segments"])))
+    return sign_urls(data, full=member)
 
 
 @router.get("/{rid}/embed-link")

diff --git a/fastapi_backend/app/domain/iiif.py b/fastapi_backend/app/domain/iiif.py
@@ -465,6 +465,23 @@ def _file_label(f):
     return f.get("label") or f"{filemod.LABELS.get(f['role'], 'File')}: {f['name']}"
 
 
+TRACK_KINDS = {"transcript": "captions", "captions": "captions", "translation": "subtitles"}  # a file's role: its <track> kind
+
+
+def caption_tracks(rid, language=None, files=(), transcript=True):
+    """The WebVTT tracks a player offers (docs/video.md#captions): the transcript's (`transcript`: when it has lines and
+    the viewer may read them), then each of `files` that is a transcript, captions or a translation saying when its
+    lines are, in the order they were added. The links are IIIF's, relative (the web app passes /iiif on to the API)
+    and plain: the routes sign them (app.api.media.sign_tracks), for someone who may read them."""
+    lang = _lang({"language": language})
+    out = [{"kind": "captions", "label": "Transcript", "language": lang, "url": f"/iiif/{rid}/transcript.vtt"}] if transcript else []
+    for f in files:
+        if f["role"] in TRACK_KINDS and f.get("timed"):
+            url = f"/iiif/{rid}/files/{f['id']}.vtt"
+            out.append({"kind": TRACK_KINDS[f["role"]], "label": _file_label(f), "language": f.get("language"), "url": url})
+    return out
+
+
 def _file_open(a, f):
     """Whether everyone may have a supplementary file: the part its role follows is open (attachments never are)."""
     part = filemod.PART.get(f["role"])

diff --git a/fastapi_backend/app/domain/public.py b/fastapi_backend/app/domain/public.py
@@ -36,7 +36,7 @@ def recording(db, cfg, rid, seen, a, member=False, granted=False, network=None):
 
     `closed` lists the parts the recording has that this visitor can't use; a locked recording closes all of them.
     """
-    rec = db.one("SELECT title, recorded_at, duration_ms, space, source, path, remote, media FROM $r", r=R("recording", rid))
+    rec = db.one("SELECT title, recorded_at, duration_ms, space, source, path, remote, media, language FROM $r", r=R("recording", rid))
     if not rec:
         raise KeyError(rid)
     meta = md.effective(db, cfg, rid)
@@ -70,6 +70,7 @@ def recording(db, cfg, rid, seen, a, member=False, granted=False, network=None):
         "closed": [p for p in acc.PARTS if has[p]],
         "files": [],
         "files_closed": len(filemod.of(db, rid)),
+        "captions": [],
     }
     if seen == "locked":
         return out
@@ -133,6 +134,9 @@ def recording(db, cfg, rid, seen, a, member=False, granted=False, network=None):
             if acc.is_open(a, "transcript")
             else [],
         }
+    if use["media"] and not paged:  # what they may read of it, as tracks for the <video> (the route signs them)
+        tracks = [f for f in kept if filemod.open_to(a, seen, f["role"])]
+        out["captions"] = iiif.caption_tracks(rid, rec.get("language"), tracks, use["transcript"])
     if use["index"]:
         out["chapters"] = [{"t0": s["t0"], "t1": s.get("t1"), "title": s.get("title")} for s in d["sections"]]
     return out

diff --git a/fastapi_backend/app/schemas/public.py b/fastapi_backend/app/schemas/public.py
@@ -7,7 +7,7 @@
 from pydantic import Field
 
 from app.schemas.common import AccessLevel, AccessPart, RequestModel, ResponseModel
-from app.schemas.recordings import MediaKind
+from app.schemas.recordings import CaptionTrack, MediaKind
 
 View = Literal["full", "public", "locked"]
 RequestStatus = Literal["pending", "approved", "declined"]
@@ -119,6 +119,11 @@ class PublicRecording(ResponseModel):
     closed: list[AccessPart] = Field(description="parts the asset has that this visitor can't use")
     files: list[PublicFile] = Field(default_factory=list, description="its supplementary files this visitor may download")
     files_closed: int = Field(0, description="how many of its files this visitor can't download")
+    captions: list[CaptionTrack] = Field(
+        default_factory=list,
+        description="with media this visitor may play: the text they may read as WebVTT tracks for the <video>, the "
+        "transcript first, then supplementary transcripts, captions and translations that say when their lines are",
+    )
     downloads: bool = Field(
         True,
         description="this visitor may save its files; false when its owners turned downloads off (it still plays and "

diff --git a/fastapi_backend/app/schemas/recordings.py b/fastapi_backend/app/schemas/recordings.py
@@ -401,6 +401,16 @@ class Page(ResponseModel):
     label: str | None = Field(None, description="the PDF's own name for it (iv, A-1, …) when it isn't its number")
 
 
+class CaptionTrack(ResponseModel):
+    """A WebVTT track for the player's <video> (docs/video.md#captions): the transcript, or a supplementary transcript,
+    captions or translation that says when its lines are."""
+
+    kind: Literal["captions", "subtitles"] = Field(description="the <track> kind: subtitles for a translation")
+    label: str = Field(description="its name in the browser's captions menu")
+    language: str | None = Field(None, description="its language (BCP 47), when known")
+    url: str = Field(description="a signed link to it as WebVTT, through IIIF (relative to Lens's address)")
+
+
 class Player(ResponseModel):
     """Everything the player shows: text, people, sections, entities and, for videos, shots and faces; for
     documents and images, their pages."""
@@ -443,6 +453,11 @@ class Player(ResponseModel):
     faces_mode: str = Field("off", description="the namespace's face mode: off, detect or recognize")
     faces_pixelate: bool = Field(False, description="the namespace pixelates faces in the pictures visitors see")
     nsfw: NsfwState | None = Field(None, description="NSFW classification, where it's on in its namespace and it was classified")
+    captions: list[CaptionTrack] = Field(
+        default_factory=list,
+        description="its text as WebVTT tracks, the transcript first (not a document's or an image's); supplementary "
+        "ones only for members of its namespace",
+    )
 
 
 class RecordingUpdate(RequestModel):