adeelahmad · Oct 19, 2026 · Oct 19, 2026
diff --git a/README.md b/README.md
@@ -145,7 +145,8 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p
   Drive can sign in through rclone on the Lens server, with no app of your own to register. Upload audio, video, PDFs,
   Office files, photos, JSON and zip or tar archives (photos and videos dated by when they were taken, from the file
   itself, and photos placed by their GPS on the Library's map, by API and MCP too), and transcripts made elsewhere
-  (SRT, WebVTT, text, Word, PDF), each line credited to the speaker the file names; and, with an Apache Tika server (an
+  (SRT, WebVTT, text, Word, PDF), each line credited to the speaker the file names, and one that came without times
+  lined up with its audio when you ask; and, with an Apache Tika server (an
   opt-in extra), documents nothing else here can read; capture web pages one at a time, from a link list or a
   bookmarks file, by API or by asking the assistant, from private networks you allow in Settings too; and import
   videos and podcasts by their address (off by default), taking a video's own captions as its transcript when people
@@ -156,7 +157,8 @@ listed under [In progress](#in-progress), and what comes next under [Planned](#p
   files, a link list, a source's files) is one run: the toast after it, and the Import screen's list of recent runs,
   undo it whole.
   ([Sensors](docs/sensors.md), [processing](docs/processing.md), [Apache Tika](docs/configuration.md#apache-tika),
-  [import webhooks](docs/api.md#import-webhooks), [playable copies](docs/video.md#playable-copies),
+  [import webhooks](docs/api.md#import-webhooks), [lining up a transcript](docs/processing.md#aligning),
+  [playable copies](docs/video.md#playable-copies),
   [where an asset was taken](docs/processing.md#where-an-asset-was-taken),
   [import runs](docs/api.md#import-runs);
   [checks](docs/requirements.md#bring-in-everything))

diff --git a/changelog.d/align-transcript.md b/changelog.d/align-transcript.md
@@ -0,0 +1,9 @@
+- **Line up a transcript with its audio.** A transcript imported without times (plain text, Word, PDF, Markdown) now
+  says its times are estimated, and once its audio is attached its page offers **Line up with the audio**: one click
+  transcribes the audio in the background with the engine transcription already uses (activity ledger, budgets and
+  quiet processing hours apply), matches the transcript's words to the words heard, and gives each line their times;
+  lines none of whose words were heard sit between their neighbours. The text and speakers stay exactly as they were.
+  Nothing is lined up unless someone asks. Agents use `POST /api/v1/resources/{id}/align` and the MCP tool
+  `align_transcript`; `GET /api/v1/resources/{id}` says `times` (estimated or aligned). Upgrading marks transcripts
+  imported before with estimated times (the `estimated-times` step)
+  ([Lining up a transcript with its audio](docs/processing.md#aligning)).
diff --git a/docs/api.md b/docs/api.md
@@ -639,6 +639,7 @@ GET    /api/v1/resources/{rid}/embed-link
 GET    /api/v1/resources/{rid}/audio
 GET    /api/v1/resources/{rid}/wordcloud.svg
 POST   /api/v1/resources/{rid}/reprocess
+POST   /api/v1/resources/{rid}/align
 POST   /api/v1/resources/{rid}/share
 DELETE /api/v1/resources/{rid}/share
 GET    /api/v1/resources/{rid}/shares
@@ -807,6 +808,19 @@ manifest and Dublin Core record give the tag). The MCP tool `set_language` does
 in the web app shows the language by name with its tag, which editors change there. The Language filter counts `en-GB`
 apart from `en`; a pipeline step's `languages` condition of `en` takes `en-GB` and `en-US` too.
 
+### Lining up a transcript with its audio
+
+A transcript imported without times (plain text, Word, PDF, Markdown, `Name: text` lines without times) has times
+estimated from the text, and `GET /resources/{rid}` says so: `times` is `estimated`. Attaching its audio keeps them.
+`POST /resources/{rid}/align` (the process permission) lines the transcript up with its audio in the background:
+202 with the job, which transcribes the audio as the transcribe step does (the same engine, activity ledger and
+budgets; a heavy step, so quiet processing hours apply), gives each line the times of its words, then runs analyze,
+embed and report again. The text and speakers stay as they are. Afterwards `times` is `aligned` and `aligned` is
+`{at, matched, engine}`: when, the share of the transcript's words heard (0 to 1) and the engine that heard them.
+Asking while it runs gives the same job; 400 when the resource has no audio, or its lines' times came with it or from
+transcription (`times` is null). Audited as `recording.align`; the MCP tool `align_transcript` does the same
+([Lining up a transcript with its audio](processing.md#aligning)).
+
 ### Documents and images
 
 A document (a PDF; a Word, PowerPoint or spreadsheet file, OpenDocument or RTF; text or Markdown; a saved web page; an

diff --git a/docs/architecture.md b/docs/architecture.md
@@ -82,8 +82,8 @@ jobs), drain it (it also hands its job back after the step it's on) and resume i
 record, so it holds across restarts. `GET /api/v1/events` streams job progress as server-sent events.
 
 **Quiet processing** (Settings › Workers, both off by default) keeps the heavy steps (`workers.heavy_steps`: by
-default transcribe, read, diarize, shots, ocr, faces, objects, nsfw and blur_faces, the ones that work through media
-on this machine) to when it suits a small machine at home:
+default transcribe, read, diarize, shots, ocr, faces, objects, nsfw, blur_faces and align, the ones that work through
+media on this machine) to when it suits a small machine at home:
 
 - `workers.heavy_hours`, a window like `01:00-06:00` in `workers.timezone` (UTC by default; it may pass midnight).
   Outside it no job starts a heavy step: a job that reaches one goes back on the queue, its log saying why, and waits

diff --git a/docs/database.md b/docs/database.md
@@ -133,6 +133,10 @@ The steps so far, in the order they run:
   merges carried them over (the `not_same` table) get a pair naming the person they were merged into, following merges
   of merges; a pair that would name one person twice is dropped. The old rows stay, so undoing such a merge brings
   them back, and a pair already there isn't made again, so it's safe to run twice.
+* `estimated-times`: transcripts imported before an asset said its lines' times were estimated (`times`,
+  [Lining up a transcript with its audio](processing.md#aligning)) say so now, where their lines are spaced as the
+  estimate spaces them (the first at 0, each 250 ms after the one before ends). Only assets without `times` are
+  looked at, so it's safe to run twice.
 
 To add a step:
 

diff --git a/docs/mcp.md b/docs/mcp.md
@@ -134,6 +134,7 @@ For an app that speaks OpenAI's API rather than MCP, the namespace can also answ
 | `get_face_blur` | a video's faces (track ids and names, to pick from) and its [copy with faces blurred](video.md#blurring-faces-for-sharing): the file and where to download it, whose faces it blurs and whether visitors play it; and the latest job making one, with how far it got, why it waits or why it failed. Members of its namespace |
 | `blur_faces` | make a [copy of a video with faces blurred](video.md#blurring-faces-for-sharing) for sharing: all faces, the unnamed ones or the tracks given; it runs in the background and is kept with the video's files, the original unchanged. Needs a write-scope token and the process permission |
 | `set_face_blur_for_visitors` | have visitors (public pages, embeds, share links, IIIF, feeds) play a video's copy with faces blurred instead of the original, or not. Needs a write-scope token and owner access |
+| `align_transcript` | [line an imported transcript up with its audio](processing.md#aligning) when its times are estimated from the text (`get_recording` says `times`): the audio is transcribed in the background and each line takes the times of its words, the text and speakers unchanged. Needs a write-scope token and editor access (the process permission) |
 | `namespace_integrations` | whether a namespace has [its own MCP server](#one-namespaces-mcp-server) and [chat completions](api.md#chat-completions-for-any-app) on, their addresses, and the tools offered there |
 | `set_namespace_integrations` | turn a namespace's own MCP server or chat completions on or off, and pick the read tools its chat completions may use. Owners, with a write-scope token |
 | `list_files` | a folder of [the graph as files](file-tree.md): your namespaces, then their collections, topics (nested by narrower topics) and entities, each holding its assets' files |
@@ -220,7 +221,8 @@ queues an episode, `create_feed` and `publish_to_feed`, which make and fill a [p
 `write_spec`, which queues a spec, `translate_transcript`, which queues a translation, `draft_reply` and `edit_reply`, which draft and change [email replies](email-replies.md) (nothing is sent), `send_reply`, which asks to send one (it waits in Approvals), `set_nsfw` and `mark_nsfw`, which change [NSFW classification](nsfw.md) (owners or admins, and
 editors), `set_language`, which sets an asset's language (editors), `enhance`, which enhances an image and reads it again (the process permission), `blur_faces` and
 `set_face_blur_for_visitors`, which make a video's copy with faces blurred and choose whether visitors play it (the
-process permission, and owners),
+process permission, and owners), `align_transcript`, which lines an imported transcript up with its audio (the process
+permission),
 `set_namespace_integrations`, which turns a namespace's own MCP server or chat completions on or off (owners), `set_picture_descriptions`, which changes what describes pictures (admins), `set_file_storage`, which changes where new files are kept (admins), `set_secret_reference`, which points a secret field at a password manager's item (admins), `install_plugin`, which
 installs a plugin from a marketplace as your own extension, `add_mcp_server`, which adds an
 MCP server (admins), `subscribe_feed` and `unsubscribe_feed`, which follow and stop following news feeds (admins), `set_mail_routing`, which changes where an email account's messages go (admins), `add_mesh_node`,

diff --git a/docs/processing.md b/docs/processing.md
@@ -226,7 +226,8 @@ on a text-only asset's page. The file is uploaded and becomes the asset's media;
 speakers stay. Then these steps run, after anything the asset's job still had to do: transcribe (which keeps the
 transcript and draws the waveform), diarize (speakers by voice, unless the transcript named them), shots, text on screen
 and faces (video only), analyze and report. The asset takes the file's fingerprint, so scans and uploads of the
-same file find it, unless another asset in the namespace has it already.
+same file find it, unless another asset in the namespace has it already. A transcript that came without times keeps
+its estimated ones until you [line it up with the audio](#aligning).
 
 Speakers are recognised from `Name: text`, `[12:30] Name: text`, `Name (12:30): text`, Otter/Zoom/Teams exports (a
 `Name  12:30` line, then what they said), `speaker|emotion|text` lines, and the speakers in SRT/VTT and JSON. In
@@ -238,6 +239,22 @@ first: front matter or the first heading becomes the title; imports, exports, JS
 imported as a transcript needs a text layer; import a scan as a document instead, and its pages are read by OCR. Named speakers are matched to the namespace's people by name; generic labels (SPEAKER_00, S1, CH0)
 become new people.
 
+### Lining up a transcript with its audio {#aligning}
+
+A transcript imported without times keeps its estimated times when its audio is attached, and its page says "Times
+estimated". **Line up with the audio** on that page (editors; the API's `POST /resources/{rid}/align`, or the MCP tool `align_transcript`) gives its lines real times, in the
+background: the audio is transcribed by the same engine the transcribe step uses (its fallbacks, your vocabulary,
+the activity ledger and budgets apply, and it is a heavy step, so quiet processing hours apply), the transcript's words
+are matched to the words heard (rare words first, then the words between them), and each line runs from its first
+matched word to its last, keeping those words' times so the player follows along word by word. A line none of whose
+words were heard shares the time between the lines either side by how many words each has. The text and the
+speakers stay exactly as they are; only the times change. Then analyze, embed and report run again, so chapters,
+search passages and the report follow the new times; a summary's key points keep their times until it's summarised
+again. The page then says "Times from the audio" and how many of the words were heard. When none were (the wrong file,
+or another language), the step fails, saying so, and the estimated times stay. Nothing is lined up unless someone
+asks: it costs a transcription. Forcing the transcribe step to run again replaces the imported lines with the
+engine's own.
+
 ## People and namespaces
 
 Person ids belong to one namespace. Per recording, `diarize.engine: auto` works like this:

diff --git a/docs/requirements.md b/docs/requirements.md
@@ -170,6 +170,7 @@ request, to another check that proves the same thing; loosening a requirement or
 | Connect with Google; Dropbox, OneDrive and Google Drive sign in through rclone on the server | [test_gmail_source.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_gmail_source.py) `test_connecting_the_account_in_the_browser`; [test_drive_google.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_drive_google.py) `test_drive_connects_with_google_for_rclone`; [test_rclone_signin.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_rclone_signin.py) `test_drive_signs_in_through_rclone_with_no_google_app`, `test_a_browser_on_the_server_reaches_rclone_itself`, `test_onedrive_finds_the_accounts_own_drive` |
 | Uploads of audio, PDFs, Office files, photos, JSON and archives | [test_uploads.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_uploads.py) `test_uploading_audio_in_chunks`, `test_json_uploaded_is_read_as_records`; [test_documents.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_documents.py) `test_a_pdf_becomes_a_document_with_its_pages_and_text`, `test_scans_and_images_are_read_by_ocr`; [test_conversion.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_conversion.py) `test_office_and_text_files_become_documents`; [test_anytopdf.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_anytopdf.py) `test_an_uploaded_zip_keeps_its_files`; [test_archives.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_archives.py) `test_reading_an_archive`; [journeys.mjs](https://github.com/adeelahmad/lense/blob/main/ci/clicks/journeys.mjs) `import-file`, `import-paste` |
 | Transcripts made elsewhere (text, Markdown, Word, PDF, SRT, WebVTT, JSON) imported with each line credited to the speaker the file names, WebVTT voice spans (`<v Name>`, `<v.loud Name>`) included | [test_importing.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/domain/test_importing.py) `test_formats_and_speakers_by_name`, `test_webvtt_voice_spans_name_each_line` |
+| A transcript imported without times says its times are estimated, and is lined up with its audio only when asked (API, MCP and one button): its words matched to the words the speech engine hears, each line taking their times and the rest placed between their neighbours, the text and speakers unchanged | [test_align.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_align.py) `test_lines_take_the_times_of_their_words_and_the_rest_fill_in_between`, `test_a_long_transcript_is_matched_as_well_as_word_by_word_and_quickly`, `test_imports_say_their_times_are_estimated_and_older_ones_are_recognised`, `test_lining_up_a_transcript_with_its_audio`, `test_agents_line_up_a_transcript_through_mcp`; [align.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/align.test.tsx) `is offered only for estimated times with audio, and not while a job is at it`, `lines it up in one click, then gets out of the way` |
 | Media browsers can't play (an AVI, HEVC, AMR) gets a copy they can, for the player only, off by default; files they play get none | [test_playable.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_playable.py) `test_a_camera_clip_gets_a_copy_every_browser_plays`, `test_a_file_browsers_play_gets_no_copy` |
 | Photos and videos dated by when they were taken, from the file itself (a photo's EXIF date and GPS fix as `place`, a video's creation time) when no name or source dates them; a date already set is kept and none is invented | [test_file_dates.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_file_dates.py) `test_a_photo_is_dated_and_placed_by_its_exif`, `test_a_date_already_set_is_kept`, `test_a_video_is_dated_by_its_creation_time`, `test_a_scanned_video_is_dated_by_its_creation_time` |
 | Assets placed by their file's GPS fix (a photo's EXIF, a video's ISO 6709 location) on the Library's map, listed by API (`located`, `bbox`) and MCP to members only, on tiles from a setting | [test_photo_map.py](https://github.com/adeelahmad/lense/blob/main/fastapi_backend/tests/api/test_photo_map.py) `test_located_assets_are_listed_inside_a_box`, `test_an_agent_finds_what_was_taken_in_a_place`, `test_a_video_is_placed_by_its_location`, `test_the_map_tiles_are_a_setting`; [library-map.test.tsx](https://github.com/adeelahmad/lense/blob/main/nextjs-frontend/__tests__/library-map.test.tsx) `placed assets are dots on the tiles from Settings, framed to show them all` |

diff --git a/fastapi_backend/app/api/mcp_tools.py b/fastapi_backend/app/api/mcp_tools.py
@@ -1030,6 +1030,7 @@ def get_recording(ctx: Context, recording_id: int) -> dict[str, Any]:
         "pages": raw.get("pages"),
         "language": raw.get("language"),
         "language_set": bool(raw.get("language_set")) or None,  # a person set it; transcription leaves it
+        "times": rec.times,  # an imported transcript's line times: estimated from the text, or aligned with its audio
         "status": rec.status,
         "tags": rec.tags or None,
         "your_role": rec.role,
@@ -3176,6 +3177,27 @@ def set_face_blur_for_visitors(ctx: Context, recording_id: int, on: bool) -> dic
     return _face_blur_out(ctx, recording_id)
 
 
+# ---------- an imported transcript lined up with its audio (docs/processing.md#aligning) ----------
+@tool(
+    "align_transcript",
+    "Line up a transcript with its audio",
+    "Give an imported transcript's lines real times from its audio: a transcript imported without times (plain text, "
+    "Word, PDF, Markdown) has times estimated from the text (get_recording says times: estimated), and attaching its "
+    "audio keeps them. This transcribes the audio in the background, matches the transcript's words to the words "
+    "heard, and gives each line the times of its words; the text and speakers stay as they are, and times becomes "
+    "aligned. It costs a transcription (a cloud speech engine's call goes in the activity ledger). Check on it with "
+    "runs. Needs a token with the write scope and the right to process the recording; refused when it has no audio, "
+    "or its lines' times came with it or from transcription.",
+    Arg("recording_id", "integer", "the asset", required=True),
+    writes=True,
+)
+def align_transcript(ctx: Context, recording_id: int) -> dict[str, Any]:
+    if not ctx.user.can_write:
+        raise ToolError("this token is read-only: ask for a token with the write scope to line up transcripts")
+    out = recording_routes.align_transcript(recording_id, ctx.acl, ctx.user, ctx.db)
+    return {"job": out.job, "status": "aligning", "recording_url": ctx.link(recording_id)}
+
+
 # ---------- a namespace's own ways in for other apps (integrations.py) ----------
 def _integrations(ctx: Context, namespace: str, sid: int) -> dict[str, Any]:
     return {