Repository navigation
Tags
Tags
feat(parakeet): add NVIDIA Nemotron streaming ASR models (EN + multil… …ingual 3.5) (OpenWhispr#1131) * feat: add Nemotron ASR model Add Nemotron Speech Streaming EN 0.6B through sherpa-onnx online websocket support. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(parakeet): bump sherpa-onnx to 1.13.4 for Nemotron streaming support The Nemotron model artifact in the registry (sherpa-onnx-nemotron-speech-streaming-en-0.6b-560ms-int8-2026-04-25) was re-exported alongside runtime updates in sherpa-onnx v1.13.0; the previously pinned v1.12.23 runtime produces degraded transcripts on Nemotron streaming models (verified empirically: 1.12.x mis-decodes words that 1.13.4 gets right on the model's own test wavs). Also correct the Nemotron registry size to the extracted model size (632MB, matching the convention used by the other Parakeet entries — this value feeds the pre-download disk-space check), and document the new model and the offline/online runtime split in CLAUDE.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdfxroZFwgsVyKMhum1UAo * feat(parakeet): add multilingual Nemotron 3.5 ASR Streaming 0.6B model Adds nemotron-3.5-asr-streaming-0.6b to the local model registry alongside the English Nemotron model. Cache-aware streaming FastConformer with per-stream automatic language detection; 15 transcription-ready languages. Uses the same online websocket runtime added for the English model — no code changes needed, only registry + i18n + docs. Verified against sherpa-onnx 1.13.4: the published int8 export (sherpa-onnx-nemotron-3.5-asr-streaming-0.6b-560ms-int8-2026-06-11) loads with the exact encoder/decoder/joiner/tokens flags OpenWhispr passes and decodes English, German, and Japanese test wavs with language auto-detection. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdfxroZFwgsVyKMhum1UAo * feat(parakeet): stream Nemotron live preview over a persistent connection The Live Transcription Preview previously transcribed buffered 1.5s chunks independently, which discards a cache-aware streaming model's context at every chunk boundary. When the selected NVIDIA model has runtime "online" (Nemotron), the preview now opens one persistent websocket stream to the online sherpa-onnx server for the whole recording: worklet PCM is converted to float32 and forwarded as it is captured, partial results replace the preview text live, and stopping the preview flushes the stream ("Done" / "Done!" handshake) for the final text. Audio captured while the streaming server is warming up is buffered and fed into the stream once it opens. Offline models keep the existing chunked path, and the online path falls back to it if the stream cannot start. Covered by test/helpers/parakeetOnlineStream.test.js using a mock websocket server that speaks the sherpa-onnx online protocol (partials, endpointed segments, finish handshake, abort). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdfxroZFwgsVyKMhum1UAo * refactor(parakeet): code-quality pass on Nemotron integration Applies the findings from a four-angle review (reuse, simplification, efficiency, altitude) of the Nemotron PR diff: - Single registry access point: new parakeetModelInfo.js owns getModelRuntime and REQUIRED_MODEL_FILES; parakeetWsServer no longer reads the model registry, and the getModelRuntime pass-throughs on both manager layers are gone. ipcHandlers now asks a capability question (supportsOnlineStreaming) instead of comparing runtime strings. - Runtime-only binary API: getWsBinaryPath/isAvailable take a runtime (the model-name-or-runtime union sniffing is gone), callers resolve model to runtime once, hasAnyWsBinary replaces the hand-rolled offline-OR-online checks, and per-runtime path memoization is restored. - One online-protocol implementation: _transcribeOnline is now a thin wrapper over createOnlineStream instead of a ~55-line duplicate of it. - Incremental result accumulation: createOnlineAccumulator parses each websocket message once (the previous per-message re-parse of the whole history was quadratic over a recording) and live preview updates are emitted only when the text actually changes. parseOnlineMessages keeps its API as a wrapper over the accumulator. - Wire format stays in the ws layer: the stream handle exposes sendPcm16 (int16 capture format, converted internally) so ipcHandlers no longer performs float32 conversion; abort now settles a pending finish(). - Preview session staleness is tracked by one generation counter instead of three heuristic condition sets. - pcm16ToFloat32 is the canonical converter (returns Float32Array); meetingEchoLeakDetector and liveSpeakerIdentifier drop their local copies. - Sidecar reaper matches parakeet against its two exact ws binary prefixes instead of the whole "sherpa-onnx-" vendor prefix, which also overlapped the diarization entry. - Download script reuses the shared cleanupFiles (extended to accept multiple keep prefixes) and copyBinary drops its redundant label and basename handling. Verified on Node 24 with full dependencies: 278 tests, 270 pass, 0 fail (8 skipped), tsc --noEmit clean, eslint clean, prettier clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdfxroZFwgsVyKMhum1UAo * fix(parakeet): repair sherpa-onnx 1.13.4 binary downloads - Windows release assets are now suffixed (-MD-Release); old name 404s - ad-hoc re-sign macOS binaries/dylibs: upstream ships libonnxruntime with an invalid arm64 signature that dyld SIGKILLs - rm before copy so a stale symlink can't redirect writes into an old inode - re-point the unversioned lib symlink and prune stale versioned libs - run download:sherpa-onnx in predev/prestart (marker-gated, fast no-op) * refactor(parakeet): condense comments --------- Co-authored-by: Joel Tony <github@jaytau.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com>
fix(windows): define WASAPI IIDs in-source to fix helper link (LNK2019) The previous uuid.lib attempt didn't help: no import library carries IID_IAudioClient, IID_IAudioCaptureClient, or IID_IActivateAudioInterfaceCompletionHandler — the SDK declares them via MIDL_INTERFACE for C++ __uuidof only, and <initguid.h> doesn't emit them in C mode. Define them with DEFINE_GUID (INITGUID is active) so the C references resolve at link time, matching how HELPER_IID_IAgileObject is already handled. Reverts the unneeded uuid.lib link.
feat(onboarding): use-case intent capture, meeting step, and finish s… …tep (OpenWhispr#928) * feat(onboarding): use-case intent capture, meeting step, and finish step Reworks the onboarding wizard around an id-keyed dynamic step machine: - New "Why are you here?" step: multi-select use-case cards (OptionCard) plus an optional note, persisted to settings and synced fire-and-forget to /api/onboarding-intent for signed-in users - Permission copy rewritten to explain outcomes; system audio badge flips to Recommended when the user came for meeting notes - New conditional meeting step introducing auto-detection and the existing meeting hotkey (shown when system audio is granted or meetings selected) - New finish step: OpenWhispr Cloud summary with Open Settings deep-link (Settings > Transcription) or Skip; local/BYOK variant for skip-auth users; registry-gated Corti partner screen for medical intent - All new strings translated across the 10 locale files * refactor(lint): fix react-refresh and hook-deps warnings, run formatter - Move pending-invitation token helpers out of AcceptInvitationModal into src/utils/pendingInvitationToken.ts so the component file only exports components (react-refresh/only-export-components) - Memoize the invitations fallback in ShareNoteDialog so its callback dependencies are stable (react-hooks/exhaustive-deps) - Prettier reflow in clipboard.js * fix(onboarding): add back-to-sign-in escape from email verification Without it, users who never receive the verification email are stuck on the polling screen with no way back to choose another email or sign-in method. * style(onboarding): match back-to-sign-in button to auth screen idiom Align the verification back button with AuthenticationStep's continue- without-account styling; prettier array reflow in zh locale files. * refactor(auth): use Button ghost variant for subtle text actions Replace bespoke raw buttons (back-to-sign-in, continue-without-account) with the design-system Button, gaining focus-visible rings and proper disabled styling. * feat(onboarding): broaden medical use case to healthcare, clarify multi-select - Rename use-case id medical -> healthcare; card now reads "Medical, clinical & therapy transcription / Dictate patient notes, session notes and reports" so therapy and psychology users self-identify - Add "Select all that apply" hint and switch OptionCard's selection indicator from radio circle to checkbox square so multi-select is obvious - Update Corti finish title to match; all 10 locales updated * fix(onboarding): allow proceeding with Corti credentials in setup step The Corti provider tab (OpenWhispr#929) now appears in the onboarding transcription picker, but canProceed only recognized openai/groq/mistral/custom — a user selecting Corti was blocked on Next unless an OpenAI key happened to be set. Gate on client id + secret, matching shouldUseStreaming's readiness rule. * feat(onboarding): preselect Corti provider when opening settings from finish step The Corti finish screen's Set up in Settings CTA now sets cloudTranscriptionProvider to corti so the settings picker opens on the Corti tab with its credential fields. Safe for signed-in users — the provider only takes effect in BYOK mode. * feat(onboarding): inline Corti setup on the finish step Replace the settings detour with credentials entered in place: honest copy (clinical-grade, HIPAA compliant), a visible $50 free credit badge next to the create-account link, Client ID/Secret fields reusing ApiKeyInput, and a US/EU data-region select (defaults prefilled from the store). Use Corti activates the provider in BYOK mode and completes onboarding; Not now falls through to the default finish. * refactor(onboarding): remove quit button from title bar The power/quit button only ever rendered in onboarding (TitleBar's sole consumer) and duplicates OS-level quit (Cmd+Q, dock, window controls). Remove it end to end: TitleBar quit UI + confirm dialog, app-quit IPC handler, appQuit preload bridge and type, and the titleBar i18n block in all 10 locales. * feat(onboarding): link Corti homepage with UTM and add setup steps Point the create-account link at corti.ai with the partnership UTM params instead of the console, and add a three-step guide (sign up, create API credentials in the console, paste below) since the homepage no longer lands users on signup. * feat(onboarding): refine use-case step copy and move progress into macOS title bar - Subtext constrained to readable width and rewritten to say plainly that the user's picks set up the app and steer what we build next - Select-all hint restyled to the existing uppercase section-label idiom - Note placeholder reduced to "Optional" — no canned examples - TitleBar gains a centered no-drag slot; on macOS the step progress renders there (reclaiming the progress strip's height), Windows/Linux keep the strip alongside their window controls * feat(onboarding): add upload-audio use case and skip button for optional steps - New "Uploading audio files" use-case option (id: upload) with FileAudio icon - Skip button beside Next on optional steps (use case, voice agent, meeting) - Add common.skip and the upload option copy across all 10 locales * feat(transcription): add Corti provider icon Register the Corti logo in PROVIDER_ICONS so the Corti tab and provider chips show a real icon; monochrome (currentColor) like xai. * refactor(meetings): consolidate meeting detection into the notification toggle The meeting-detection notification was the only consumer of detection, so a detector running with notifications off did nothing but burn CPU. Drive the audio detector from the notification toggle (notificationsEnabled && notifyMeetingDetection) and remove the redundant standalone Audio Detection setting, its UI section, and the now-dead calendar.detection translations. * fix(settings): don't grab the mic when opening settings MicrophoneSettings called getUserMedia on mount to read device labels, which activates a mic input session and interrupts other audio (pauses music on macOS) every time Settings opens. Enumerate devices first and only fall back to getUserMedia when labels are missing (permission not yet granted). * feat: add UTM tracking to Corti 'get a key' link Point the Corti transcription provider's 'Get a key' link at the corti.ai home page with referral UTM params, matching the canonical CORTI_SIGNUP_URL used in onboarding instead of console.corti.app. * chore: apply prettier formatting to pre-existing files Format 21 docs, build scripts, and test files that were not prettier-compliant on main. Pure formatting, no logic changes. Kept separate from the onboarding feature work. * chore(release): 1.7.3 Bump version to 1.7.3 and document the onboarding upgrade, Corti provider polish, simplified meeting-detection setting, and the Settings microphone fix in the changelog. * docs(changelog): capture full 1.7.3 scope Expand the 1.7.3 entry to cover everything merged since v1.7.2, not just the onboarding branch: Corti BYOK and xAI transcription providers, Snippets, Voice Agent hotkey, dictionary redesign, notification controls, Linux PipeWire/paste fixes, new AI models, and the broader fix batch. * fix(onboarding): keep step progress labels on one line The stepper renders inside TitleBar's absolutely-positioned `left-1/2` center slot, whose shrink-to-fit width is capped at the space from center to edge. That squeezed the row and wrapped the longest label ("Voice Agent") onto two lines. Adding `whitespace-nowrap` lets the wrapper resolve to the content's full width and center it cleanly, fixing the layout across all platforms (shared StepProgress component). * fix(auth): recover stale-bearer cloud requests via session cookie The main process authenticates cloud requests with a single saved bearer token that only refreshes on sign-in. After an email/password login (which, unlike OAuth, never force-refreshes it) that token can go stale, so main-process requests 401 while the renderer stays signed in via its session cookie. onboarding-intent was the visible casualty: it fires right after the use-case step and failed silently, leaving intent unsaved. Retry cloud-api-request once with the window's session cookie when a bearer request returns 401. Cookie-only on retry, since a tagging-along bearer overwrites the cookie server-side. * fix(onboarding): hide voice agent step for continue-without-account users Continue-without-account users have no LLM configured, so the dictation agent can't run — skip the voice agent onboarding step for them and show it only to signed-in account users. * fix(i18n): add Hyprland hotkey locale strings * fix(clipboard): prevent stale clipboard restores during paste * fix(dictation-agent): reach cloud agent from voice hotkey without a selected model The voice agent hotkey resolved to "skip" for signed-in OpenWhispr cloud users because agentReachable required a non-empty dictationAgentModel, which cloud mode never sets — pasting the raw transcript instead of running the agent. Mirror the cleanup cloud fallback: the dictation agent is now reachable in cloud (openwhispr) and self-hosted (lan) modes without an explicit model, matching ReasoningService.processText's empty-model allowance, and the agent route forces provider "openwhispr" in cloud mode so it no longer depends on the cleanup scope. Also pass voiceAgentRequested on the streaming-STT path so the hotkey works there too. Reachability logic is extracted into a pure, tested helper (resolveDictationAgentReachability) alongside resolveDictationRouteKind. * feat(onboarding): preview the real meeting notification in the meeting step Replace the abstract "meeting detected" explainer with a faithful preview of the actual notification, rendered from a shared MeetingNotificationCard that MeetingNotificationOverlay now also consumes so the two never drift. Add the onboarding.meeting.notification.{title,body,cta} keys across all 10 locales. * fix(note-formatting): use cloud provider when no model is selected Note formatting passed no provider to processText, so cloud users fell back to getModelProvider(""), which resolves from the cleanup scope — throwing "No reasoning model selected" when cleanup uses a BYOK/local provider. Pin the provider to openwhispr in cloud mode for both enhancement and title generation. * feat(onboarding): temporarily hide the meeting step for all users * fix(dev): diagnose missing Electron binary * feat(transcription): dedicated Audio Upload speech-to-text settings Give audio upload its own transcription context, mirroring how Note Recording was split from Dictation. Adds an Audio Upload tab under Settings -> Speech-to-Text with independent upload* settings; the upload page is now settings-only (inline picker removed). A one-time migration copies existing users' dictation transcription preference into the new context; new users default to OpenWhispr Cloud. - store: 9 upload* keys, selectResolvedUploadTranscription (falls back to base dictation values), migrateUploadTranscription (runs after provider migration) - settings: new UploadTranscriptionPanel (openwhispr/providers/local) - upload page: read-only consumer of resolved upload settings; no-provider CTA routes to the Audio Upload tab - i18n: tabs.upload across all 10 locales; remove orphaned upload setup strings - fix cross-window boolean sync for meeting/upload useLocalWhisper (BOOLEAN_SETTINGS) - extract shared useStartOnboarding hook; drop dead uploadSetupComplete write * fix(history): inline discarded toggle and stop list flash on refresh Move the "Show Discarded" toggle inline to the left of "Clear All" in the first day header so the pair reveals together on hover; keep it reachable above the empty state. Gate the loading spinner on the initial load only (no data yet) so toggling discarded keeps the current list mounted and swaps in new data instead of unmounting to a spinner and flashing. * feat(upload): cancel button during audio-file transcription Add a Cancel transcription action beneath the progress bar on the upload page (shown for every transcription). Cancelling returns to the upload screen and discards the result so nothing is saved; a run-id token guards the async path so a cancelled or superseded run never persists a note. * docs(changelog): document late-breaking 1.7.3 changes * fix(onboarding): apply Corti to all transcription scopes on finish Choosing Corti during onboarding only set the dictation cloud provider, so note recording and audio upload kept their previous provider and every Settings tab still showed OpenWhispr Cloud — each tab displays the InferenceMode field, which was never updated. setCloudTranscriptionForAllScopes now applies the chosen provider to dictation, note recording, and audio upload, and sets each scope's InferenceMode via a shared deriveTranscriptionMode helper (also reused by the provider-settings migration). * refactor(corti): rework streaming close handshake with ack tracking Replace the single closeResolve callback with a pendingAck/waitForAck mechanism that resolves on the matching server ack (flushed/ended) or socket close, times out cleanly, and closes the socket with code 1000. * feat(onboarding): rename activation step to Dictation and add voice agent test area - Rename the activation onboarding step to "Dictation" in the progress nav and "Dictation Setup" in the step header - Rename the voice agent step header to "Voice Agent Setup" - Add a test area under the voice agent hotkey with example spoken commands and a textarea to try it - Add and translate the new strings across all 10 locales * fix(meeting): send true 24kHz sample rate to cloud streaming providers Meeting capture runs at 24kHz end-to-end, but Corti/Deepgram/AssemblyAI were told 16kHz, so they read the PCM ~1.5x too slow and degraded accuracy. Thread the real rate through the meeting connect path and teach cortiStreaming to honor options.sampleRate (matching Deepgram/AssemblyAI). OpenAI already declares 24kHz; dictation (16kHz) and the local downsample path are untouched. * fix(dictation-agent): default to OpenWhispr Cloud for signed-in users The dictation agent scope defaulted to "providers" (BYOK) with no model, unlike the cleanup and chat-agent scopes which default to OpenWhispr Cloud. As a result, a signed-in user who never explicitly configured the dicta 8000 tion agent had it unreachable, so the voice agent hotkey silently typed the raw transcript instead of running the command. Default dictationAgentMode and dictationAgentCloudMode to "openwhispr" so the voice agent works out of the box on Cloud. Users who explicitly chose BYOK, local, or self-hosted are unaffected (persisted values are read first). * perf(corti): pre-warm dictation socket to cut startup latency and first-word loss Corti dictation paid the full token-mint + socket-open + CONFIG_ACCEPTED cost at hotkey time (~1s), and audio captured before the socket existed was dropped. Give CortiStreaming a real warmup() that opens the socket and finishes the config handshake ahead of recording, keeps it alive with WS pings, and lets connect() promote the warm socket for an instant start — mirroring Deepgram/AssemblyAI. Falls back to a cold connect when no warm socket exists, so behavior is never worse than before. Meeting mode (fresh instance, no warmup) is unaffected.
v1.6.10 Speaker diarization controls, Integrations view, dedicated Meetings settings, cloud-sync tombstones for folders and transcriptions, and a settings reorganization that collapses AI Models into Speech-to-Text and Language Models. Includes post-release fixes: - Cloud-api error surfacing (cloud request errors now include message/code/stack) - macOS app startup packaging restoration - Batch transcription tombstone sync (single request per 100 ids instead of per-row loop) See CHANGELOG.md for full details.