Repository navigation
feat(import): line up an imported transcript with its audio - #700
Merged
Merged
Conversation
A transcript imported without times now says its times are estimated
(`times`), and once its audio is attached a person or agent can line it up:
an align step transcribes the audio through the same engine path as the
transcribe step (ledger, budgets, quiet hours), matches the imported words to
the words heard with difflib, and gives each line their times; unmatched lines
sit between their neighbours. Text and speakers are unchanged. Then analyze,
embed and report run again. POST /resources/{rid}/align, the MCP tool
align_transcript and one button on the asset page. Off until asked; an
upgrade step marks older imports whose lines carry the estimate.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QFUNzy1QZy5HhKAyJxTAYJ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by Adeel · project thread
Description
Before: a transcript imported without times (plain text, Word, PDF, Markdown,
Name: textlines) got estimated times, and attaching its audio kept them. Clicking a line played the wrong moment.After: such an asset says
times: "estimated". Once it has audio, its page shows "Times estimated" with one button, Line up with the audio. A background job hears the audio through the same transcription path Lens already uses (same engine and fallbacks, vocabulary, activity ledger, budgets, quiet hours). It matches the imported words to the words heard withdifflib, rare words first and then the gaps, and gives each line its matched words' times. Lines that matched nothing are placed between their neighbours.The text and speakers stay exactly as imported. Analyze, embed and report then run again, so chapters, search passages and the report follow the new times. The asset then says
times: "aligned"with how many words were heard.Agents can do the same:
POST /resources/{rid}/align(process permission, 202 with the job, 400 without audio or when the times are real) and the MCP toolalign_transcript.get_recordingnow givestimes.Motivation and Context
Personal feature plan, Phase 1 "Bring existing transcripts": forced alignment was the gap left after the rest of the plan landed.
Types of changes
!in the title)Checklist:
make hookschecks it on commit).docs/accordingly.changelog.d/align-transcript.md, per build(release): changelog fragments in changelog.d/ instead of one Unreleased section #581.Done questions (AGENTS.md)
POST /resources/{rid}/alignand MCPalign_transcript.alignis one of the heavy steps in Settings › Workers › Quiet processing.recording.align; no new outward surface.estimated-timesupgrade step marks older text imports whose lines have the estimate's spacing; it only touches assets withouttimes, so it is safe to run twice (docs/database.md).make openapi).ingest.transcribe_onesplit into a reusableingest.hear(); jobs, quiet processing and the job banner reused.Local CI:
pytest tests/api/test_align.py tests/api/test_jobs.py tests/api/test_mcp.py tests/domain/test_importing.py tests/domain/test_migrations.py tests/domain/test_quiet.py(78 passed), ruff, mypy, jestalign.test.tsx(4 passed) and__tests__/recording*,tsc --noEmit,ci/check_requirements.py,mkdocs build --strictall clean; OpenAPI regenerated with no drift.Generated by Claude Code