Repository navigation
feat(transcribe): keep how sure transcription was of each word, and mark the unsure ones - #681
Merged
Merged
Conversation
…ark the unsure ones Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QFUNzy1QZy5HhKAyJxTAYJ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by Adeel · project thread
Description
Before: Whisper, Deepgram, AssemblyAI and ElevenLabs said how sure they were of each word, but Lens dropped it, so nothing pointed at the words worth checking.
After: Lens keeps each word's confidence with its timing. In the transcript, a word below the threshold gets a dotted gold underline and a tooltip ("Transcription wasn't sure of this word (42%)"), and correcting it takes the mark away. Settings → Transcription → "Mark words it was less sure of than" sets the threshold (
transcribe.confidence_highlight, 0.6 by default; 0 marks none).GET …/playergives each word's confidence as a fifth item inwand the threshold asunsure_below, and the MCP toolget_transcriptlists each line'sunsurewords. Transcripts made before, and those from SenseVoice and OpenAI's own API, which don't give a confidence, have no marks and work as before.How:
speech.pyreads each provider's word confidence,transcript.pyandingest.pystore it as the word's fifth field, andaudio_cleanup.remapkeeps it when times are remapped. The web app'smodel.tsandturn.tsxmark the unsure words.Motivation and Context
Item from the personal feature plan sheet (show low-confidence words).
Screenshots (if appropriate):
n/a
Steps to reproduce (if appropriate):
Transcribe a recording with Deepgram, then open its transcript: words it was less than 60% sure of are underlined, with the percentage on hover.
Types of changes
!in the title)Checklist:
make hookschecks it on commit).tests/domain/test_speech.py,tests/api/test_split_merge.py,tests/domain/test_audio_cleanup.py,tests/api/test_mcp.py,__tests__/recording-unsure-words.test.tsx,__tests__/recording-split-words.test.ts).docs/accordingly (docs/processing.md, Words transcription wasn't sure of;docs/api.md;docs/mcp.md).## UnreleasedinCHANGELOG.md. Written aschangelog.d/word-confidence.md, per build(release): changelog fragments in changelog.d/ instead of one Unreleased section #581.Done questions (AGENTS.md)
unsurewords inget_transcript, confidences in/player..envedit or manual step.transcribe.confidence_highlightin Settings → Transcription, default 0.6, not in the wizard.docs/requirements.mdname the checks.make openapi).Local CI (
ci/local-ci.sh): FastAPI embedded, FastAPI server, Next.js, pre-commit and PR title all passed.Generated by Claude Code