Repository navigation
MIC SUPPORT, UI ENHANCEMENTS - #1
Conversation
…ent can directly input the api keys from the interface itself rather than messing up .env
fix(settings): auto-create .env from template on startup if missing - Add ensure_env_file helper to copy .envexample to .env when missing - Invoke ensure_env_file in read_env_file and save_settings handlers - Resolves configuration loading errors on fresh git clones
📝 WalkthroughWalkthroughAdds Sarvam STT/TTS support, TTS helpers, a casual-chat responder, frontend settings to persist Sarvam and Groq API keys, backend .env ensure/read/write logic, voice UI controls and styles, and Bun tests; README and branch metadata updated. ChangesSarvam Voice Features & API Key Management
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (4)
src-tauri/src/lib.rs (1)
124-174:⚠️ Potential issue | 🟠 Major | 🏗️ Heavy liftPrevent empty
SARVAM_API_KEY/GROQ_API_KEYfrom overwriting stored.envvalues insave_settings.
save_settingsunconditionally rewritesSARVAM_API_KEY=andGROQ_API_KEY=insrc-tauri/src/lib.rs, so passing""wipes existing credentials. Infrontend/src/App.tsxandfrontend/src/CommandBar.tsx, both key states start asuseState('')and are only filled later via an asyncgetSettings()call—so if the user changes provider or shortcut before that load completes,updateProvider/updateShortcutcan callsaveSettings(...)with empty key values and permanently clear the stored keys.🛡️ Possible guard
- } else if trimmed.starts_with("SARVAM_API_KEY=") { - *line = format!("SARVAM_API_KEY={}", sarvam_api_key); - sarvam_api_key_found = true; + } else if trimmed.starts_with("SARVAM_API_KEY=") { + if !sarvam_api_key.is_empty() { + *line = format!("SARVAM_API_KEY={}", sarvam_api_key); + } + sarvam_api_key_found = true;(apply the same for
GROQ_API_KEY)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src-tauri/src/lib.rs` around lines 124 - 174, The save_settings function currently overwrites SARVAM_API_KEY and GROQ_API_KEY with empty strings; change its logic so when iterating lines in save_settings: for SARVAM_API_KEY and GROQ_API_KEY, if the incoming sarvam_api_key/groq_api_key is non-empty then replace the line and set the *_found flag, but if the incoming value is empty then leave the existing line untouched and still set the *_found flag so we don't append an empty new entry; likewise, when appending missing keys at the end only push SARVAM_API_KEY/GROQ_API_KEY lines if the corresponding sarvam_api_key/groq_api_key argument is non-empty; reference save_settings, sarvam_api_key, groq_api_key, sarvam_api_key_found and groq_api_key_found for where to apply these checks.frontend/src/CommandBar.tsx (2)
613-653:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winAdd accessible names to the new icon-only buttons.
titleis not a reliable accessible name. The mic and speak buttons needaria-label, and the toggle states should exposearia-pressed, otherwise assistive tech users won't get a dependable description of these controls.♿ Minimal accessibility fix
<button type="button" className={`command-mic-btn ${isRecording ? 'recording' : ''} ${isTranscribing ? 'loading' : ''}`} + aria-label={isRecording ? 'Stop recording' : 'Record voice command'} + aria-pressed={isRecording} => { e.stopPropagation(); toggleRecording(); }} @@ <button type="button" className={`command-speak-btn ${isSpeaking ? 'speaking' : ''}`} + aria-label={isSpeaking ? 'Stop speaking response' : 'Speak response'} + aria-pressed={isSpeaking} => { e.stopPropagation(); speakResponse(); }}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@frontend/src/CommandBar.tsx` around lines 613 - 653, The mic and speak icon-only buttons in the CommandBar component lack reliable accessible names and state exposure; add explicit aria-labels and aria-pressed attributes: for the mic button (class command-mic-btn) add aria-label that reflects the current action (e.g., "Stop recording" when isRecording true, otherwise "Record voice command") and set aria-pressed={isRecording}; for the speak button (class command-speak-btn) add aria-label reflecting its action ("Stop speaking" vs "Speak response") and set aria-pressed={isSpeaking}; ensure these attributes update in the same handlers (toggleRecording, speakResponse) and remain present even when disabled so assistive tech can read the control and state.
316-350:⚠️ Potential issue | 🟠 Major | ⚡ Quick winSerialize or debounce full settings saves here too.
This component has the same last-write-wins problem as
App: each async save rewrites the entire settings object using sibling values captured from the current render, and the new key inputs trigger that on every keystroke. Overlapping saves can therefore restore stale provider/shortcut/API key values. Route these updates through one debounced or serialized persistence path instead of four independentsaveSettings(...)calls.Also applies to: 567-587
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@frontend/src/CommandBar.tsx` around lines 316 - 350, The four handlers updateProvider, updateShortcut, updateSarvamApiKey, and updateGroqApiKey currently call saveSettings independently on every change, causing last-write-wins races and stale sibling values; replace these direct calls with a single serialized persistence path (e.g., one debounced saveSettings that reads the latest provider/shortcut/sarvamApiKey/groqApiKey from state or a queued/serialized updater) so keystroke-level updates only trigger one debounced save and overlapping async saves are sequenced; update both these handlers and the similar block at 567-587 to use that debounced/serialized wrapper around saveSettings and ensure the wrapper reads current state atomically (or uses functional state updates) before calling saveSettings.frontend/src/App.tsx (1)
98-132:⚠️ Potential issue | 🟠 Major | ⚡ Quick winSerialize or debounce full settings saves.
These handlers persist a complete settings payload on every change, but each call is built from sibling state captured in the current render. If two saves overlap, an older request can finish last and roll back a newer provider/shortcut/API key value with stale data. Update local state immediately, then persist through one debounced or serialized save path that always writes the latest snapshot.
💡 One way to avoid the stale overwrite
+ const [didLoadSettings, setDidLoadSettings] = useState(false); + useEffect(() => { getSettings() .then((settings) => { setProvider(settings.provider); setShortcut(settings.shortcut); setSarvamApiKey(settings.sarvam_api_key || ''); setGroqApiKey(settings.groq_api_key || ''); + setDidLoadSettings(true); }) .catch((err) => console.error('Failed to load settings:', err)); }, []); + + useEffect(() => { + if (!didLoadSettings) return; + const timer = window.setTimeout(() => { + void saveSettings(provider, shortcut, sarvamApiKey, groqApiKey).catch((err) => { + console.error('Failed to save settings:', err); + }); + }, 250); + + return () => window.clearTimeout(timer); + }, [didLoadSettings, provider, shortcut, sarvamApiKey, groqApiKey]);Also applies to: 367-387
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@frontend/src/App.tsx` around lines 98 - 132, The four handlers (updateProvider, updateShortcut, updateSarvamApiKey, updateGroqApiKey) race because each immediately calls saveSettings with sibling state captured from the render; serialize or debounce saves instead: update local state immediately in each handler, then route persistence through a single function (e.g., persistSettings) that either (a) debounces writes and reads the latest snapshot from state refs/useState getters before calling saveSettings, or (b) serializes saves by awaiting an in-flight promise so each save reads current state when it runs. Refactor updateProvider/updateShortcut/updateSarvamApiKey/updateGroqApiKey to call only setXxx(...) and then trigger persistSettings (not direct saveSettings), and implement persistSettings to always call saveSettings(provider, shortcut, sarvamApiKey, groqApiKey) using the latest values (via refs or reading state) with debounce or an activeSave promise to prevent stale-overwrite races.
🧹 Nitpick comments (1)
frontend/src/styles.css (1)
1459-1473: ⚡ Quick winFix keyframe naming convention to resolve stylelint errors.
Stylelint is flagging these keyframe names as errors because they don't follow the kebab-case convention. While the animations are functionally correct, the linting errors could block CI/CD if stylelint is configured to fail on errors.
Note: The existing codebase also uses camelCase for multi-word animations (
slowPulse,riseIn), so there's already inconsistency. Consider either:
- Updating only these new animations to kebab-case for a quick fix
- Updating all animations to kebab-case for full consistency
🔧 Proposed fix for new animations (quick solution)
-.command-mic-btn.recording { +.command-mic-btn.recording { background: rgba(255, 83, 112, 0.25); border-color: rgba(255, 83, 112, 0.6); color: `#ffb7c5`; - animation: micPulse 1.5s infinite; + animation: mic-pulse 1.5s infinite; } .mic-record-indicator { width: 10px; height: 10px; border-radius: 50%; background: `#ff5370`; display: inline-block; - animation: beaconPulse 1.2s infinite alternate; + animation: beacon-pulse 1.2s infinite alternate; } .command-speak-btn.speaking { background: rgba(167, 255, 122, 0.2); border-color: rgba(167, 255, 122, 0.5); color: `#bdff94`; - animation: speakWobble 0.3s infinite alternate; + animation: speak-wobble 0.3s infinite alternate; } -@keyframes micPulse { +@keyframes mic-pulse { 0% { box-shadow: 0 0 0 0 rgba(255, 83, 112, 0.4); } 70% { box-shadow: 0 0 0 8px rgba(255, 83, 112, 0); } 100% { box-shadow: 0 0 0 0 rgba(255, 83, 112, 0); } } -@keyframes beaconPulse { +@keyframes beacon-pulse { from { transform: scale(0.9); opacity: 0.8; } to { transform: scale(1.15); opacity: 1; } } -@keyframes speakWobble { +@keyframes speak-wobble { from { transform: scale(1.0); } to { transform: scale(1.08); } }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@frontend/src/styles.css` around lines 1459 - 1473, Stylelint rejects the current keyframe names because they are camelCase; rename the `@keyframes` identifiers micPulse, beaconPulse, and speakWobble to kebab-case (e.g., mic-pulse, beacon-pulse, speak-wobble) and update every reference to those animations (any animation or animation-name usages in CSS/JS/JSX that target micPulse, beaconPulse, speakWobble) to the new kebab-case names so definitions and usages match.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@branch_meta.txt`:
- Around line 2-3: Update the objective text to fix typos: change "sarwam" to
"Sarvam", replace "most porlly" with "most probably" (or "mostly" if preferred),
and change "filling me in .env" to "filling them in .env" so the sentence reads
clearly about adding Sarvam STT/TTs support and implementing a simple UI to let
clients enter API keys instead of manually filling them in .env.
In `@frontend/src/CommandBar.tsx`:
- Around line 74-96: The cleanup currently only pauses currentAudioRef and
doesn't release the microphone MediaStream; modify CommandBar to store the
getUserMedia() stream in a ref (e.g., mediaStreamRef) so you can stop its tracks
on every exit path: stop all tracks in the catch block where new
MediaRecorder(...) could throw, in the MediaRecorder.onstop handler, and in the
existing unmount useEffect; update startRecording to assign the stream to
mediaStreamRef, and before constructing MediaRecorder use
MediaRecorder.isTypeSupported(...) to gate/fallback the mimeType instead of
hardcoding 'audio/webm'; keep existing refs (mediaRecorderRef, audioChunksRef,
currentAudioRef) and update stopSpeaking to also stop/clear mediaStreamRef when
appropriate.
In `@src-tauri/src/lib.rs`:
- Around line 103-104: The in-memory default for the keyboard shortcut is set to
"Enter" (let mut shortcut = "Enter".to_string()) but the env defaults (and
ensure_env_file) use "Space"; update the default shortcut initialization to
"Space" so get_settings' fallback aligns with BLINKY_SHORTCUT=Space written by
ensure_env_file (referencing the let mut shortcut variable and ensure_env_file
behavior) to avoid divergent persisted vs in-memory defaults.
---
Outside diff comments:
In `@frontend/src/App.tsx`:
- Around line 98-132: The four handlers (updateProvider, updateShortcut,
updateSarvamApiKey, updateGroqApiKey) race because each immediately calls
saveSettings with sibling state captured from the render; serialize or debounce
saves instead: update local state immediately in each handler, then route
persistence through a single function (e.g., persistSettings) that either (a)
debounces writes and reads the latest snapshot from state refs/useState getters
before calling saveSettings, or (b) serializes saves by awaiting an in-flight
promise so each save reads current state when it runs. Refactor
updateProvider/updateShortcut/updateSarvamApiKey/updateGroqApiKey to call only
setXxx(...) and then trigger persistSettings (not direct saveSettings), and
implement persistSettings to always call saveSettings(provider, shortcut,
sarvamApiKey, groqApiKey) using the latest values (via refs or reading state)
with debounce or an activeSave promise to prevent stale-overwrite races.
In `@frontend/src/CommandBar.tsx`:
- Around line 613-653: The mic and speak icon-only buttons in the CommandBar
component lack reliable accessible names and state exposure; add explicit
aria-labels and aria-pressed attributes: for the mic button (class
command-mic-btn) add aria-label that reflects the current action (e.g., "Stop
recording" when isRecording true, otherwise "Record voice command") and set
aria-pressed={isRecording}; for the speak button (class command-speak-btn) add
aria-label reflecting its action ("Stop speaking" vs "Speak response") and set
aria-pressed={isSpeaking}; ensure these attributes update in the same handlers
(toggleRecording, speakResponse) and remain present even when disabled so
assistive tech can read the control and state.
- Around line 316-350: The four handlers updateProvider, updateShortcut,
updateSarvamApiKey, and updateGroqApiKey currently call saveSettings
independently on every change, causing last-write-wins races and stale sibling
values; replace these direct calls with a single serialized persistence path
(e.g., one debounced saveSettings that reads the latest
provider/shortcut/sarvamApiKey/groqApiKey from state or a queued/serialized
updater) so keystroke-level updates only trigger one debounced save and
overlapping async saves are sequenced; update both these handlers and the
similar block at 567-587 to use that debounced/serialized wrapper around
saveSettings and ensure the wrapper reads current state atomically (or uses
functional state updates) before calling saveSettings.
In `@src-tauri/src/lib.rs`:
- Around line 124-174: The save_settings function currently overwrites
SARVAM_API_KEY and GROQ_API_KEY with empty strings; change its logic so when
iterating lines in save_settings: for SARVAM_API_KEY and GROQ_API_KEY, if the
incoming sarvam_api_key/groq_api_key is non-empty then replace the line and set
the *_found flag, but if the incoming value is empty then leave the existing
line untouched and still set the *_found flag so we don't append an empty new
entry; likewise, when appending missing keys at the end only push
SARVAM_API_KEY/GROQ_API_KEY lines if the corresponding
sarvam_api_key/groq_api_key argument is non-empty; reference save_settings,
sarvam_api_key, groq_api_key, sarvam_api_key_found and groq_api_key_found for
where to apply these checks.
---
Nitpick comments:
In `@frontend/src/styles.css`:
- Around line 1459-1473: Stylelint rejects the current keyframe names because
they are camelCase; rename the `@keyframes` identifiers micPulse, beaconPulse, and
speakWobble to kebab-case (e.g., mic-pulse, beacon-pulse, speak-wobble) and
update every reference to those animations (any animation or animation-name
usages in CSS/JS/JSX that target micPulse, beaconPulse, speakWobble) to the new
kebab-case names so definitions and usages match.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 95fc54af-8154-412b-a93f-c81676ed7df7
⛔ Files ignored due to path filters (1)
package-lock.jsonis excluded by!**/package-lock.json
📒 Files selected for processing (9)
.envexamplebranch_meta.txtfrontend/src/App.tsxfrontend/src/CommandBar.tsxfrontend/src/lib/tauri.tsfrontend/src/styles.cssfrontend/src/vite-env.d.tsfrontend/tsconfig.jsonsrc-tauri/src/lib.rs
| To add sarwam stt and tts support... | ||
| and most porlly implement a simple UI design to get api keys from client instead of manually filling me in .env.... No newline at end of file |
There was a problem hiding this comment.
Fix spelling and grammar errors in objective documentation.
The documentation contains several typos that should be corrected:
- "sarwam" should be "Sarvam" (the proper name of the AI service)
- "most porlly" appears to be "most probably" or "mostly"
- "filling me in .env" should be "filling them in .env"
📝 Proposed fix
-To add sarwam stt and tts support...
-and most porlly implement a simple UI design to get api keys from client instead of manually filling me in .env....
+To add Sarvam STT and TTS support...
+and most probably implement a simple UI design to get API keys from client instead of manually filling them in .env....📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| To add sarwam stt and tts support... | |
| and most porlly implement a simple UI design to get api keys from client instead of manually filling me in .env.... | |
| # OBJECTIVE | |
| To add Sarvam STT and TTS support... | |
| and most probably implement a simple UI design to get API keys from client instead of manually filling them in .env.... |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@branch_meta.txt` around lines 2 - 3, Update the objective text to fix typos:
change "sarwam" to "Sarvam", replace "most porlly" with "most probably" (or
"mostly" if preferred), and change "filling me in .env" to "filling them in
.env" so the sentence reads clearly about adding Sarvam STT/TTs support and
implementing a simple UI to let clients enter API keys instead of manually
filling them in .env.
| const mediaRecorderRef = useRef<MediaRecorder | null>(null); | ||
| const audioChunksRef = useRef<Blob[]>([]); | ||
| const [isRecording, setIsRecording] = useState(false); | ||
| const [isTranscribing, setIsTranscribing] = useState(false); | ||
| const [isSpeaking, setIsSpeaking] = useState(false); | ||
| const currentAudioRef = useRef<HTMLAudioElement | null>(null); | ||
|
|
||
| const stopSpeaking = () => { | ||
| if (currentAudioRef.current) { | ||
| currentAudioRef.current.pause(); | ||
| currentAudioRef.current = null; | ||
| } | ||
| setIsSpeaking(false); | ||
| }; | ||
|
|
||
| // Cleanup audio on unmount | ||
| useEffect(() => { | ||
| return () => { | ||
| if (currentAudioRef.current) { | ||
| currentAudioRef.current.pause(); | ||
| } | ||
| }; | ||
| }, []); |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE="frontend/src/CommandBar.tsx"
# Show relevant sections with line numbers
sed -n '1,140p' "$FILE" | nl -ba | sed -n '1,140p'
echo "----"
sed -n '140,260p' "$FILE" | nl -ba | sed -n '1,160p'
echo "----"
sed -n '560,720p' "$FILE" | nl -ba | sed -n '1,220p'
echo "----"
# Quick targeted searches for key patterns
rg -n "getUserMedia|MediaRecorder|mimeType|onstop|stop\\(\\)|track\\.stop|mediaStreamRef|aria-label|title=\"(Mic|Speak)" "$FILE" -SRepository: KingSahil/Blinky
Length of output: 103
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE="frontend/src/CommandBar.tsx"
echo "=== Section: top/refs + cleanup (approx 1-140) ==="
sed -n '1,140p' "$FILE" | nl -ba
echo "=== Section: startRecording/MediaRecorder (approx 140-260) ==="
sed -n '140,260p' "$FILE" | nl -ba
echo "=== Section: icon-only mic/speak buttons (approx 560-720) ==="
sed -n '560,720p' "$FILE" | nl -ba
echo "=== Targeted pattern search ==="
rg -n "getUserMedia|MediaRecorder|mimeType|onstop|stop\\(\\)|track\\.stop|mediaStreamRef|aria-label|role=|title=|IconButton" "$FILE" -SRepository: KingSahil/Blinky
Length of output: 172
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE="frontend/src/CommandBar.tsx"
echo "=== Section: top/refs + cleanup (approx 1-140) ==="
sed -n '1,140p' "$FILE" | cat -n
echo "----"
echo "=== Section: startRecording/MediaRecorder (approx 140-280) ==="
sed -n '140,280p' "$FILE" | cat -n
echo "----"
echo "=== Section: icon-only mic/speak buttons (approx 520-720) ==="
sed -n '520,720p' "$FILE" | cat -n
echo "----"
echo "=== Targeted pattern search ==="
rg -n "getUserMedia|MediaRecorder|mimeType|onstop|stop\\(\\)|track\\.stop|mediaStreamRef|aria-label|title=\"(Mic|Speak)|aria-labelledby|role=|IconButton" "$FILE" -SRepository: KingSahil/Blinky
Length of output: 19243
🌐 Web query:
Tauri WebView2 WKWebView MediaRecorder audio/webm mimeType support
💡 Result:
Tauri applications rely on the underlying system's webview engine, which dictates MediaRecorder support: Windows uses WebView2 (Chromium-based) and macOS/iOS use WKWebView (WebKit-based) [1][2]. Because of this, MediaRecorder compatibility for audio/webm depends on the host OS rather than Tauri itself [3]. WebView2 (Windows): WebView2, being Chromium-based, generally supports audio/webm and video/webm [1][4]. You can verify support using MediaRecorder.isTypeSupported('audio/webm') [5][3]. WKWebView (macOS/iOS): WKWebView has historically had limited support for the WebM container [6]. While recent WebKit development has introduced work to add experimental or limited WebM support, it is not consistently available or enabled by default across all versions [7][8]. Traditionally, WKWebView relies on MP4 (via AVFoundation) for media recording [9]. Recommendation: 1. Feature Detection: Always use MediaRecorder.isTypeSupported to check for compatibility at runtime [9][5][3]. 2. Cross-Platform Strategy: For consistent audio recording across Tauri platforms, do not rely solely on the browser's MediaRecorder [10]. Official Tauri audio plugins (like tauri-plugin-audio-recorder) typically use native Rust backends (e.g., cpal, AVAudioRecorder, MediaRecorder API) to ensure reliable output formats like WAV (desktop) or M4A/AAC (mobile), bypassing webview limitations [10]. 3. Fallback: If you must use MediaRecorder, implement a fallback mechanism that attempts to record in 'audio/webm' but reverts to 'video/mp4' (which is widely supported on iOS/macOS) if the former is not supported [11][3].
Citations:
- 1: https://dev.to/moboudra/i-was-wrong-about-electron-1e9g
- 2: https://dev.to/sudo_blaze/electron-is-worst-development-platform-1p6n
- 3: https://stackoverflow.com/questions/76124559/javascript-mediarecorder-video-format-compatibility-between-different-browsers
- 4: https://dev.to/faisal_rehman_5a8de221423/show-hn-nextgen-dlm-a-lightweight-open-source-download-manager-built-in-rust-tauri-memory-4if2
- 5: https://stackoverflow.com/questions/42048368/where-is-a-comprehensive-list-of-supported-media-types-when-recording-with-the-m
- 6: https://crates.io/crates/tauri-plugin-media-toolkit
- 7: [Cocoa] Add webm support to MediaRecorder WebKit/WebKit#34905
- 8: Support video/webm;codecs=h264,pcm in MediaRecorder WebKit/WebKit#39443
- 9: https://webkit.org/blog/11353/mediarecorder-api/
- 10: https://github.com/brenogonzaga/tauri-plugin-audio-recorder
- 11: https://stackoverflow.com/questions/67874713/mediarecorder-ios-14-6-mimetype-not-supported
🌐 Web query:
MediaRecorder constructor throws Unsupported mimeType audio/webm getUserMedia stream not stopped
💡 Result:
If you call new MediaRecorder(stream, { mimeType: 'audio/webm' }) and the constructor throws “Unsupported mimeType audio/webm…”, the root cause is almost always that the current browser/user agent cannot record that MIME type for the provided stream/track configuration. The MediaRecorder spec/platform behavior is to throw NotSupportedError at construction time when the UA can’t record the requested format [1]. What to do (robust, up-to-date pattern) 1) Don’t hardcode audio/webm. Probe support first: js const mimeType = 'audio/webm'; const supported = MediaRecorder.isTypeSupported(mimeType); MediaRecorder.isTypeSupported(mimeType) returns true only if the UA is capable of recording Blob objects for that MIME type [2][3]. 2) If audio/webm isn’t supported, fall back to another MIME type that is supported by the UA, e.g. audio/mp4 on some browsers (common example: iOS requires mp4). A real-world case shows iOS throwing “NotSupportedError: mimeType is not supported” for audio/webm and using video/mp4 as a fallback [4]. 3) Also ensure you pass the correct stream/tracks: MediaRecorder can throw NotSupportedError when it cannot record one or more tracks with the current configuration. In particular, using “video/webm;codecs=vp8” for an audio track can fail because the configured codecs/container don’t match what the track can be recorded with; the underlying rule is that if the UA cannot record each track using the constrained configuration, it aborts with NotSupportedError [5]. 4) Stop the getUserMedia stream yourself only after you’re done recording/when the UI stops. In general, the “Unsupported mimeType …” failure happens during MediaRecorder construction, before start—so there is nothing “recording” that would auto-stop. Use track.stop to release the microphone. Example end-to-end flow js async function startMicRecording() { const stream = await navigator.mediaDevices.getUserMedia({ audio: true }); // Candidate list in priority order const candidates = [ 'audio/webm;codecs=opus', 'audio/webm', 'audio/mp4', 'video/mp4' // fallback used in the iOS workaround reports ]; const mimeType = candidates.find(t => MediaRecorder.isTypeSupported(t)); if (!mimeType) { stream.getTracks().forEach(t => t.stop()); throw new Error('No supported MediaRecorder mimeType found'); } let recorder; try { recorder = new MediaRecorder(stream, { mimeType }); } catch (e) { // Ensure mic is released if MediaRecorder construction failed stream.getTracks().forEach(t => t.stop()); throw e; } recorder.start(); return recorder; } Why this also explains “stream not stopped” complaints When MediaRecorder construction fails due to unsupported mimeType, the code after the constructor often never runs, so cleanup code (like stream.getTracks.forEach(t => t.stop)) may never execute. That makes it look like “the stream wasn’t stopped”, even though the correct fix is: always put stream-stop cleanup in a catch/finally path around MediaRecorder creation. Key takeaways - “Unsupported mimeType audio/webm” from the MediaRecorder constructor means the UA can’t record that MIME type; it throws NotSupportedError immediately at construction [1]. - Use MediaRecorder.isTypeSupported before constructing [2][3]. - Provide fallbacks (e.g., audio/mp4 or video/mp4 on iOS) and ensure you stop tracks if construction/start fails [4]. - Wrap MediaRecorder creation in try/catch and stop the getUserMedia tracks in the error path so the microphone is released.
Citations:
- 1: https://chromium.googlesource.com/external/github.com/web-platform-tests/wpt/+/refs/heads/gecko/1791332/mediacapture-record/MediaRecorder-mimetype.html
- 2: https://developer.mozilla.org/en-US/docs/Web/API/MediaRecorder/isTypeSupported_static
- 3: https://wiki.developer.mozilla.org/en-US/docs/Web/API/MediaRecorder/isTypeSupported_static
- 4: https://stackoverflow.com/questions/67874713/mediarecorder-ios-14-6-mimetype-not-supported
- 5: mimeType ambiguity: "video/webm;codecs=vp8" means? w3c/mediacapture-record#194
🌐 Web query:
Tauri webview engine WebView2 WKWebView windows mac linux media recorder support
💡 Result:
Tauri applications do not bundle a browser engine, instead utilizing the operating system's native webview component [1][2][3]. Because these underlying engines differ by platform, support for web APIs like MediaRecorder can vary significantly [4][5][3]. Platform Webview Engines: - Windows: Uses WebView2 (Chromium-based), which is kept up-to-date by Microsoft and generally offers the most consistent Chromium-like behavior [6][2]. - macOS: Uses WKWebView (WebKit-based), the same engine used by Safari [2]. - Linux: Uses WebKitGTK (WebKit-based), which varies depending on the specific distribution's package repositories and installed libraries [6][2]. MediaRecorder Support and Challenges: While the MediaRecorder API is generally available in these webviews, developers often encounter platform-specific limitations: 1. Codec and Format Inconsistencies: The available output formats (e.g., WebM vs. MP4) depend on what the native webview and the underlying system-level media frameworks (like GStreamer on Linux or AVFoundation on macOS) support [5]. For instance, on Linux, some developers have reported that while WebM recording is supported, playback of these files within the same Tauri application can fail due to codec detection issues, even when required system plugins are installed [5]. 2. Platform Quirks: Because Tauri relies on the system's webview, inconsistencies are "real" and often reflect the underlying engine's limitations rather than Tauri itself [7][3]. 3. Alternative Solutions: Due to these native webview limitations, it is common practice in the Tauri ecosystem to use native Rust-based plugins for high-reliability tasks like audio or video recording [8][9]. Plugins such as tauri-plugin-audio-recorder bypass the webview's limitations by performing recording operations directly via the Rust backend using system-native APIs (e.g., cpal on desktop), providing more consistent cross-platform results [8][9]. If your application requires reliable, cross-platform media recording, leveraging a native Rust plugin is recommended over relying solely on the frontend MediaRecorder API [8][5].
Citations:
- 1: https://danielraffel.me/2026/02/14/i-built-a-webdriver-for-wkwebview-tauri-apps-on-macos/
- 2: https://dev.to/shrsv/exploring-system-webviews-in-tauri-native-rendering-for-efficient-cross-platform-apps-9hl
- 3: https://dev.to/sudo_blaze/electron-is-worst-development-platform-1p6n
- 4: https://dev.to/moboudra/i-was-wrong-about-electron-1e9g
- 5: [Linux] WebM videos recorded by MediaRecorder cannot be played back - MEDIA_ERR_SRC_NOT_SUPPORTED error code 4 tauri-apps/tauri#14656
- 6: https://v2.tauri.app/reference/webview-versions/
- 7: https://www.reddit.com/r/rust/comments/1ct98mp/does_tauri_solve_web_renderer_inconsistencies/
- 8: https://github.com/brenogonzaga/tauri-plugin-audio-recorder
- 9: https://crates.io/crates/tauri-plugin-audio-recorder/0.1.0
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n "saveSettings\\(" frontend/src -S
rg -n "getSettings\\(|sarvamApiKey|groqApiKey|provider|shortcut" frontend/src/App.tsx frontend/src/CommandBar.tsx -S || true
ls -la frontend/srcRepository: KingSahil/Blinky
Length of output: 6443
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE_A="frontend/src/App.tsx"
FILE_C="frontend/src/CommandBar.tsx"
echo "=== App.tsx settings load (lines ~60-160) ==="
sed -n '1,180p' "$FILE_A" | cat -n
echo "=== App.tsx settings save handlers (lines ~90-160 maybe) ==="
sed -n '80,160p' "$FILE_A" | cat -n
echo "=== CommandBar.tsx settings load (lines ~250-310) ==="
sed -n '250,340p' "$FILE_C" | cat -n
echo "=== CommandBar.tsx settings save handlers (lines ~300-370) ==="
sed -n '300,390p' "$FILE_C" | cat -nRepository: KingSahil/Blinky
Length of output: 17405
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE_A="frontend/src/App.tsx"
FILE_C="frontend/src/CommandBar.tsx"
echo "=== App.tsx settings load (lines ~1-180) ==="
sed -n '1,180p' "$FILE_A" | cat -n
echo "=== App.tsx settings save handlers (lines ~80-160) ==="
sed -n '80,160p' "$FILE_A" | cat -n
echo "=== CommandBar.tsx getSettings load (lines ~250-320) ==="
sed -n '250,320p' "$FILE_C" | cat -n
echo "=== CommandBar.tsx saveSettings handlers (lines ~300-370) ==="
sed -n '300,370p' "$FILE_C" | cat -nRepository: KingSahil/Blinky
Length of output: 15993
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE="frontend/src/lib/tauri.ts"
echo "=== tauri.ts saveSettings implementation ==="
sed -n '1,120p' "$FILE" | cat -n
sed -n '120,260p' "$FILE" | cat -n
echo "=== Check CommandBar settings input handlers (onChange triggers) ==="
rg -n "updateSarvamApiKey\\(|updateGroqApiKey\\(" frontend/src/CommandBar.tsx -nRepository: KingSahil/Blinky
Length of output: 1913
Release the microphone stream on every exit path.
frontend/src/CommandBar.tsx unmount cleanup only pauses currentAudioRef and never stops the microphone MediaStream tracks created by getUserMedia(). Also, startRecording hardcodes mimeType: 'audio/webm'; on Tauri’s OS webview engines (WebView2 on Windows vs WKWebView on macOS/iOS), audio/webm support varies and new MediaRecorder(stream, { mimeType }) may throw. In that throw path, the getUserMedia() stream is never released because it isn’t stopped in the catch (and stream is scoped to the try). Keep the MediaStream in a ref and stop tracks in catch, onstop, and the unmount cleanup, and gate/fallback audio/webm via MediaRecorder.isTypeSupported(...).
Also applies to: startRecording (getUserMedia / new MediaRecorder / recorder.start).
🔧 Suggested cleanup pattern
const mediaRecorderRef = useRef<MediaRecorder | null>(null);
+ const mediaStreamRef = useRef<MediaStream | null>(null);
const audioChunksRef = useRef<Blob[]>([]);
@@
useEffect(() => {
return () => {
+ if (mediaRecorderRef.current?.state !== 'inactive') {
+ mediaRecorderRef.current.stop();
+ }
+ mediaStreamRef.current?.getTracks().forEach((track) => track.stop());
+ mediaStreamRef.current = null;
if (currentAudioRef.current) {
currentAudioRef.current.pause();
}
};
}, []);
@@
const startRecording = async () => {
if (!sarvamApiKey) {
setStatus('Please set your Sarvam AI API Key in settings first.');
return;
}
audioChunksRef.current = [];
+ let stream: MediaStream | null = null;
try {
- const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
+ stream = await navigator.mediaDevices.getUserMedia({ audio: true });
+ mediaStreamRef.current = stream;
const options = { mimeType: 'audio/webm' };
const recorder = new MediaRecorder(stream, options);
@@
recorder. () => {
- stream.getTracks().forEach(track => track.stop());
+ mediaStreamRef.current?.getTracks().forEach((track) => track.stop());
+ mediaStreamRef.current = null;
const audioBlob = new Blob(audioChunksRef.current, { type: 'audio/webm' });
await handleAudioTranscription(audioBlob);
};
@@
} catch (err) {
+ stream?.getTracks().forEach((track) => track.stop());
+ mediaStreamRef.current = null;
console.error('Error starting audio recording:', err);
setStatus('Microphone access failed or was denied.');
}
};🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@frontend/src/CommandBar.tsx` around lines 74 - 96, The cleanup currently only
pauses currentAudioRef and doesn't release the microphone MediaStream; modify
CommandBar to store the getUserMedia() stream in a ref (e.g., mediaStreamRef) so
you can stop its tracks on every exit path: stop all tracks in the catch block
where new MediaRecorder(...) could throw, in the MediaRecorder.onstop handler,
and in the existing unmount useEffect; update startRecording to assign the
stream to mediaStreamRef, and before constructing MediaRecorder use
MediaRecorder.isTypeSupported(...) to gate/fallback the mimeType instead of
hardcoding 'audio/webm'; keep existing refs (mediaRecorderRef, audioChunksRef,
currentAudioRef) and update stopSpeaking to also stop/clear mediaStreamRef when
appropriate.
| let mut provider = "groq".to_string(); | ||
| let mut shortcut = "Enter".to_string(); |
There was a problem hiding this comment.
Default shortcut here is "Enter", but the env defaults elsewhere use "Space".
get_settings falls back to "Enter" while both .envexample and ensure_env_file (Line 236) write BLINKY_SHORTCUT=Space. On a fresh clone the persisted default and the in-memory default diverge, which can surface as a confusing shortcut state in the settings UI. Align the fallback with the env default.
🔧 Proposed fix
let mut provider = "groq".to_string();
- let mut shortcut = "Enter".to_string();
+ let mut shortcut = "Space".to_string();📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| let mut provider = "groq".to_string(); | |
| let mut shortcut = "Enter".to_string(); | |
| let mut provider = "groq".to_string(); | |
| let mut shortcut = "Space".to_string(); |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src-tauri/src/lib.rs` around lines 103 - 104, The in-memory default for the
keyboard shortcut is set to "Enter" (let mut shortcut = "Enter".to_string()) but
the env defaults (and ensure_env_file) use "Space"; update the default shortcut
initialization to "Space" so get_settings' fallback aligns with
BLINKY_SHORTCUT=Space written by ensure_env_file (referencing the let mut
shortcut variable and ensure_env_file behavior) to avoid divergent persisted vs
in-memory defaults.
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
frontend/src/CommandBar.tsx (2)
656-668:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winShow the speak button for summary-only responses too.
speakResponse()already handles any non-defaultstatus, but this render path hides the button unlesssteps.length > 0. Casual-chat replies and summary-only tutor results can't be replayed manually as a result.Suggested change
- {steps.length > 0 && sarvamApiKey && ( + {!isRunning && status !== defaultStatus && sarvamApiKey && ( <button🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@frontend/src/CommandBar.tsx` around lines 656 - 668, The speak button is hidden when steps.length === 0, preventing replay for casual-chat and summary-only results; update the conditional render around the button (currently using steps, sarvamApiKey, isSpeaking, speakResponse) to show whenever sarvamApiKey is present instead of requiring steps.length > 0, keeping the existing onClick that calls speakResponse() and the isSpeaking-based class/title behavior so summary-only and casual replies can be manually replayed.
330-364:⚠️ Potential issue | 🟠 Major | ⚡ Quick winSerialize or debounce settings writes before persisting them.
Every change here calls
saveSettings(...)immediately with a full settings snapshot. Because those writes are async, an older request can complete after a newer one and revert the saved provider or API keys in.env.Save on blur/close, or funnel all changes through one debounced/serialized persistence path.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@frontend/src/CommandBar.tsx` around lines 330 - 364, The handlers updateProvider, updateShortcut, updateSarvamApiKey, and updateGroqApiKey currently call saveSettings immediately which can cause race conditions; replace the immediate saves with a single debounced or serialized persistence path: stop calling saveSettings directly in each setter, create a debouncedSaveSettings function (or a small queue/lock around saveSettings) and invoke that from these updateX functions, and additionally trigger a final save on blur/close (or on an explicit "apply" action) to ensure the latest state is persisted; keep references to saveSettings, updateProvider, updateShortcut, updateSarvamApiKey, and updateGroqApiKey when implementing the debounce/serialization.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@frontend/src/lib/casualChat.ts`:
- Around line 29-35: The identity check misses normalized contractions like
"whats your name": update the IDENTITY_QUESTIONS Set used in casualChat (and the
other identity set referenced later) to include normalized forms produced by
normalizeQuestion(), e.g., add "whats your name" (and any other contracted forms
normalizeQuestion() generates for the existing entries such as "whos your
name"/"whos you" as appropriate); ensure you update both occurrences (the
IDENTITY_QUESTIONS constant and the second identity set around lines 64-70) so
normalizeQuestion() results will match these keys.
In `@README.md`:
- Line 13: The badge image tag in README.md lacks alt text which hurts
accessibility; update the <img> element (the Bun badge) to include a descriptive
alt attribute (e.g., alt="Bun version 1.3.14 badge" or similar) so screen
readers can convey the badge meaning.
---
Outside diff comments:
In `@frontend/src/CommandBar.tsx`:
- Around line 656-668: The speak button is hidden when steps.length === 0,
preventing replay for casual-chat and summary-only results; update the
conditional render around the button (currently using steps, sarvamApiKey,
isSpeaking, speakResponse) to show whenever sarvamApiKey is present instead of
requiring steps.length > 0, keeping the existing onClick that calls
speakResponse() and the isSpeaking-based class/title behavior so summary-only
and casual replies can be manually replayed.
- Around line 330-364: The handlers updateProvider, updateShortcut,
updateSarvamApiKey, and updateGroqApiKey currently call saveSettings immediately
which can cause race conditions; replace the immediate saves with a single
debounced or serialized persistence path: stop calling saveSettings directly in
each setter, create a debouncedSaveSettings function (or a small queue/lock
around saveSettings) and invoke that from these updateX functions, and
additionally trigger a final save on blur/close (or on an explicit "apply"
action) to ensure the latest state is persisted; keep references to
saveSettings, updateProvider, updateShortcut, updateSarvamApiKey, and
updateGroqApiKey when implementing the debounce/serialization.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: e20db0f1-e9a4-418c-90cc-2a6cb3c28029
📒 Files selected for processing (6)
README.mdfrontend/src/CommandBar.tsxfrontend/src/lib/casualChat.tsfrontend/src/lib/tts.tsfrontend/tests/casualChat.test.tsfrontend/tests/tts.test.ts
✅ Files skipped from review due to trivial changes (1)
- frontend/tests/casualChat.test.ts
| const IDENTITY_QUESTIONS = new Set([ | ||
| 'who are you', | ||
| 'what are you', | ||
| 'what is your name', | ||
| 'tell me about yourself', | ||
| 'introduce yourself', | ||
| ]); |
There was a problem hiding this comment.
Handle "what's your name" after normalization.
normalizeQuestion() converts that prompt to whats your name, but the identity set only includes what is your name. That common query will fall through to the full tutor flow.
Suggested change
const IDENTITY_QUESTIONS = new Set([
'who are you',
'what are you',
'what is your name',
+ 'whats your name',
'tell me about yourself',
'introduce yourself',
]);Also applies to: 64-70
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@frontend/src/lib/casualChat.ts` around lines 29 - 35, The identity check
misses normalized contractions like "whats your name": update the
IDENTITY_QUESTIONS Set used in casualChat (and the other identity set referenced
later) to include normalized forms produced by normalizeQuestion(), e.g., add
"whats your name" (and any other contracted forms normalizeQuestion() generates
for the existing entries such as "whos your name"/"whos you" as appropriate);
ensure you update both occurrences (the IDENTITY_QUESTIONS constant and the
second identity set around lines 64-70) so normalizeQuestion() results will
match these keys.
|
|
||
| <img src="https://img.shields.io/badge/Tauri-2.x-orange?style=for-the-badge"> | ||
| <img src="https://img.shields.io/badge/React-TypeScript-61dafb?style=for-the-badge"> | ||
| <img src="https://img.shields.io/badge/Bun-1.3.14-f9f1e1?style=for-the-badge"> |
There was a problem hiding this comment.
Add alt text to the badge image for accessibility.
The Bun badge image is missing alt text, which impacts screen reader users. Consider adding descriptive alt text.
♿ Proposed fix
-<img src="https://img.shields.io/badge/Bun-1.3.14-f9f1e1?style=for-the-badge">
+<img src="https://img.shields.io/badge/Bun-1.3.14-f9f1e1?style=for-the-badge" alt="Bun 1.3.14">📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| <img src="https://img.shields.io/badge/Bun-1.3.14-f9f1e1?style=for-the-badge"> | |
| <img src="https://img.shields.io/badge/Bun-1.3.14-f9f1e1?style=for-the-badge" alt="Bun 1.3.14"> |
🧰 Tools
🪛 markdownlint-cli2 (0.22.1)
[warning] 13-13: Images should have alternate text (alt text)
(MD045, no-alt-text)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@README.md` at line 13, The badge image tag in README.md lacks alt text which
hurts accessibility; update the <img> element (the Bun badge) to include a
descriptive alt attribute (e.g., alt="Bun version 1.3.14 badge" or similar) so
screen readers can convey the badge meaning.
Added expo app, fixed some bugs Expo app 52 SDK version required Expo app: 1. Power features such as sleep, power off and restart added. 2. Query can be given and be searched on the web by the LLM using playwright automation 2a. If pre-defined toolkit found -> uses that and provides answer to the user. 2b. If unsuccessful -> uses old toolkits. If still unsuccessful -> creates new automation toolkit from scratch by the LLM provider.
MIC SUPPORT, UI ENHANCEMENTS
Added expo app, fixed some bugs Expo app 52 SDK version required Expo app: 1. Power features such as sleep, power off and restart added. 2. Query can be given and be searched on the web by the LLM using playwright automation 2a. If pre-defined toolkit found -> uses that and provides answer to the user. 2b. If unsuccessful -> uses old toolkits. If still unsuccessful -> creates new automation toolkit from scratch by the LLM provider.
…e overall branch objectives.
HACKATHON PROJECT
DEVELOPMENTS DONE:
Added microphone and speaker icons to record query audio (STT) and play back screen instructions (TTS) using Sarvam AI.
Added Groq and Sarvam API key password fields directly in the Settings menu. (client is assumed to be a lazy developer)
Updated the backend to save and load these keys automatically into the .env file.
Added an automatic check that creates the .env file from the .envexample template if it's missing on startup.
Fixed compiler errors in tsconfig.json and main.tsx so the application builds cleanly.
NEXT STEPS:
Summary by CodeRabbit
New Features
Style
Documentation
Tests