MyWhispr

MyWhispr is a local hold-to-record dictation daemon for GNOME Wayland, with native Windows and macOS (Apple Silicon) ports. Hold the configured trigger key, speak, release, and the final transcript is inserted into the focused app. Audio is recorded locally through PipeWire and transcribed by local ASR backends by default. External transcription APIs are available as explicit opt-in model cards.

The local control panel runs at:

http://127.0.0.1:16666/

Screenshots

The control panel separates everyday dictation from model management, settings, and system diagnostics. Screenshots below use sample transcripts and demo configuration; they contain no personal dictation history.

Dictation — live preview, the latest transcript, and language- 10BC8 filtered history with copy and retranscription controls.

Models — a responsive catalogue with active-model status and expandable language and model details.

Settings — audio input, output preferences, keyboard shortcuts, and custom vocabulary in one place.

What It Does

Hold-to-record dictation with release-to-insert behavior.
Configurable trigger key and combo shortcuts from the web UI.
Modifier-aware trigger: holding Shift/Ctrl/Alt/Super with the trigger passes the real keystroke through (e.g. Shift+grave types ~) instead of dictating.
Live transcript preview while recording.
Optional streaming into the focused app with guarded backspace/rewrite logic.
Local history for recent dictations, including in-memory WAV playback.
Manual retranscription of retained history through another configured model.
PipeWire input-device selection.
Custom vocabulary and hallucination phrase filtering.
GNOME Shell top-bar indicator.
Local-only HTTP UI bound to 127.0.0.1 by default.

Platform Support

Platform	Status	Notes
Ubuntu / GNOME / Wayland	Supported target	Main development and test environment. Uses PipeWire, evdev, GNOME shortcuts, `ydotool`, and `wl-copy`.
Other Linux Wayland desktops	Possible, not guaranteed	Core pieces may work if PipeWire, `/dev/input`, `ydotool`, and `wl-copy` are available. GNOME shortcut and top-bar integration are GNOME-specific.
Linux X11	Not supported	The app is designed around Wayland-era input and clipboard tools.
macOS (Apple Silicon)	Supported (beta)	Native port: `sounddevice` recording, a Quartz event tap for the hold-to-record key, synthetic paste output, a menu bar item (`rumps`), LaunchAgents, and local ASR through Metal: Parakeet TDT 0.6B v3 via MLX (default) and whisper.cpp GGML models via Homebrew `whisper-server`. Verified on an M2 / macOS 26: both ASR paths transcribe (English + Ukrainian), the app-bundle launcher is the TCC-responsible process. Intel Macs: whisper.cpp only (no MLX). See "macOS" below.
Windows	Supported (desktop daemon)	Native port: WASAPI recording via `sounddevice`, global hold-to-record hotkey via a low-level keyboard hook, `SendInput`/Win32 clipboard output, `winsound` cues, a system tray icon with state colors, and CUDA GPU transcription through the same worker venv. See "Windows setup" below.
WSL / WSLg	Not supported	Global hotkeys, audio capture, and focused-app input are host-desktop problems, not normal WSL process capabilities.

The app currently works best as a desktop daemon on GNOME Wayland. The code is being kept portable where practical, but the shipping workflow is Linux-first.

How Text Gets Inserted

MyWhispr separates transcription from output transport:

Text is pasted through the clipboard first for fast bulk insertion.
If clipboard paste fails, short printable ASCII text can fall back to the synthetic-input backend.
On Linux Wayland, output uses ydotool plus wl-copy --paste-once.
On Windows, the output backend maps the same operations to SendInput and the Win32 clipboard.

On Wayland the previous clipboard contents are snapshotted and restored around each paste. A paste chord is a real keystroke, so the app's clipboard read can still be in flight long after the chord was delivered. clipboard_paste_grace_seconds (default 2.5) is how long the dictation keeps being offered before the snapshot goes back — restoring inside that window is what makes a slow app paste the previously copied text instead of the dictation. Lower it only if you would rather have a missed paste than a late one. (clipboard_paste_consume_timeout_seconds no longer affects the Wayland path: there is no observable "the app consumed it" signal — see below.)

No chord is sent until the dictation is really on the clipboard. Spawning wl-copy does not mean it owns the selection yet. wl-copy forks once it has taken the selection, so the spawned process exiting is exactly the moment the clipboard is ours — measured on this desktop (wl-clipboard 2.2.1): ~32ms idle, but up to ~140ms under load, where 4/20 trials took longer than the 120ms this code used to wait blindly. A chord sent before that pastes whatever was on the clipboard before, i.e. the previously copied text, and still logs as a successful paste. MyWhispr therefore waits for that exit (paste._CLIPBOARD_OWNERSHIP_TIMEOUT_SECONDS, 1.5s) and, if it never comes, drops the dictation rather than sending a keystroke it cannot back (clipboard ownership not confirmed … in the log).

Whether the app actually pasted is not observable. The forked child that serves the data is not ours to wait on, and on GNOME the clipboard manager reads the offer within ~20-35ms whether or not anything pasted. So the dictation always stays on the clipboard for at least paste._MIN_HOLD_AFTER_CHORD_SECONDS (0.6) after the last chord, even on the success path; restoring earlier is what made a busy app paste the previously copied text on an otherwise clean dictation. For the same reason the fallback chord (shift+insert) fires only when a keystroke could not be delivered, never on a "the app ignored it" guess.

The grace window belongs to the clipboard, not to the paste call. Releasing the trigger key cancels the streaming session (cancel_streaming_join_timeout) and can land inside a grace wait, so the restore is handed to a detached release that keeps offering the dictation for the rest of the window before putting the snapshot back. While a release is still pending the clipboard holds MyWhispr's own text, so the next paste reuses that pending snapshot instead of re-reading the clipboard — otherwise the dictation itself would be handed back to you as your clipboard contents.

This is intentionally content-agnostic. MyWhispr does not special-case phrases or leading words to decide whether insertion should work.

Short printable ASCII dictation (up to direct_type_max_chars, 240 by default) is sent as one direct key stream by default. This bypasses clipboard transport but is not proof that duplicated insertion originates in Codex. Unicode, multiline, and longer text still use the guarded clipboard path above. Set prefer_clipboard_paste to true to restore clipboard-first routing.

On this workstation streaming.app_output_enabled=false: preview remains available, but editor insertion waits for the final transcript. Interim backspace rewrites previously left partial duplicate phrases. Synthetic ydotool events must be ignored at the entry to Daemon._on_key, including release and combo handling, so output cannot change recording language or stop recording. Local config.json owns live settings; mywhispr/config.py and config.example.json supply defaults/examples, not effective overrides.

Requirements

For the supported Linux/GNOME/Wayland setup:

Python 3.10+.
PipeWire tools: pw-record and pw-play.
Wayland clipboard tool: wl-copy from wl-clipboard.
Synthetic input tool: ydotool and a working ydotoold service/socket.
Python packages used by the daemon: aiohttp, evdev, and pyudev.
Read access to the selected /dev/input/event* keyboard devices.
A local whisper.cpp server binary for whisper.cpp models.
Optional GPU ASR Python environments for Hugging Face / NeMo models.

On Ubuntu-style systems:

sudo apt install pipewire-bin wl-clipboard ydotool python3-evdev python3-pyudev python3-aiohttp

Install

Windows

Clone the repo, then run the installer from PowerShell:

git clone https://github.com/ibmua/MyWhispr.git
cd MyWhispr
PowerShell -ExecutionPolicy Bypass -File .\scripts\install-windows.ps1

The Windows installer is idempotent. It creates .venv, .venv-gpu-asr, and an isolated .venv-qwen-asr runtime, installs runtime dependencies, downloads the whisper.cpp CUDA server build (for the quantized GGML Whisper models), writes a working config.json, downloads the default Parakeet model into the Hugging Face cache, and then stops. Qwen runs in its own virtualenv because its package pins an older Transformers release than Parakeet needs. The installer does not create startup entries or launch a hidden background process unless you ask it to.

Start MyWhispr when setup is done:

.\bin\mywhisprd.cmd

Then open the control panel:

http://127.0.0.1:16666/

Optional installer switches:

# Set up Python environments and config without downloading model weights.
PowerShell -ExecutionPolicy Bypass -File .\scripts\install-windows.ps1 -SkipModelDownload

# Skip the optional isolated Qwen runtime.
PowerShell -ExecutionPolicy Bypass -File .\scripts\install-windows.ps1 -SkipQwenBackend

# Launch MyWhispr after setup.
PowerShell -ExecutionPolicy Bypass -File .\scripts\install-windows.ps1 -Start

# Create Start Menu and login startup shortcuts.
PowerShell -ExecutionPolicy Bypass -File .\scripts\install-windows.ps1 -CreateStartMenuShortcut -CreateStartupShortcut

# Skip the ~460 MB whisper.cpp CUDA server download (GGML models stay disabled).
PowerShell -ExecutionPolicy Bypass -File .\scripts\install-windows.ps1 -SkipWhisperServer

Older -NoStart and -NoStartupShortcut flags are still accepted for scripts that already use them, but starting and startup shortcuts are now opt-in.

Whisper.cpp GGML models (including the quantized Whisper Large v3 Q5) download with one click from the web UI once whisper_server_binary points to a whisper-server.exe; the installer sets this up automatically. If the binary is missing the cards stay visible but disabled, avoiding the first-run trap where a local model file exists but no server executable can run it.

Antivirus note: the prebuilt whisper-server.exe is unsigned, and some antivirus products (notably Avast/AVG with a generic IDP.Generic verdict from "behavior analysis") silently freeze its network threads or quarantine it. The symptom is a GGML model that loads but never becomes ready. Add an exclusion for the MyWhispr folder in your antivirus settings; the Python-based GPU models (Parakeet, Qwen, etc.) are unaffected.

macOS

Requirements: macOS 13+ (MLX needs Apple Silicon and macOS 14+), Homebrew, and the Xcode Command Line Tools (xcode-select --install, for clang).

git clone https://github.com/ibmua/MyWhispr.git
cd MyWhispr
scripts/install-macos.sh                    # set up, then stop
scripts/install-macos.sh --install-agents   # ...and run at login (daemon + menu bar)

The installer is idempotent. It installs Homebrew whisper.cpp (Metal whisper-server), creates .venv (daemon: aiohttp, sounddevice, numpy, pyobjc, rumps) and .venv-mlx-asr (mlx + parakeet-mlx), writes a macOS config.json only if none exists, downloads the default model (parakeet-tdt-0.6b-v3-mlx, ~2.5 GB, into the Hugging Face cache), and builds ~/Applications/MyWhispr.app. Options:

scripts/install-macos.sh --skip-model-download   # no model weights
scripts/install-macos.sh --with-ggml             # also Whisper Large v3 Turbo Q5 (574 MB) into models/
scripts/install-macos.sh --reset-config          # back up config.json and write a fresh macOS one
scripts/install-macos.sh --install-agents --no-menubar
scripts/install-macos.sh --uninstall-agents      # unload + remove both LaunchAgents

Permissions (one time). macOS asks for three privacy permissions and attributes them to the app that is responsible for the process. The daemon therefore runs through MyWhispr.app, a tiny ad-hoc signed launcher (bundle id local.mywhispr.daemon) that spawns .venv/bin/python -m mywhispr as a child without disclaiming responsibility, so the grants belong to "MyWhispr" and survive Homebrew Python upgrades. Enable MyWhispr under System Settings → Privacy & Security → Microphone, Accessibility, and Input Monitoring. To trigger the prompts (or re-check attribution):

open -a ~/Applications/MyWhispr.app --args --request-permissions
open -W -a ~/Applications/MyWhispr.app --args --diagnose; cat ~/Library/Logs/MyWhispr/permissions.json

The launcher is only rebuilt when scripts/macos/launcher.c or scripts/macos/Info.plist change; a rebuild changes its code signature and macOS then wants the permissions granted again (remove the old MyWhispr entry first). Running .venv/bin/python -m mywhispr from a terminal also works, but then the permissions attach to your terminal app instead.

Services and logs.

launchctl print gui/$(id -u)/local.mywhispr.daemon | head -20
launchctl kickstart -k gui/$(id -u)/local.mywhispr.daemon      # restart
tail -f ~/Library/Logs/MyWhispr/daemon.log                      # also menubar.log

The menu bar item (bin/mywhispr-menubar, agent local.mywhispr.menubar) shows 🎙️ idle, 🔴 recording, ⏳ transcribing (with queue depth), ⚠️ error, 🔒 missing permission, and 💤 daemon down, plus the current model, recent transcripts to copy, Start Dictation (hands-free) / Stop Recording (⌘D), Open Control Panel, Restart Daemon, Open Logs, shortcuts to the Privacy panes, and Quit Menu Bar. Hands-free dictation inserts into whichever app was frontmost; it is the same as bin/mywhisprctl start --force followed by bin/mywhisprctl stop (a forced start skips the physical-key check and does not grab the keyboard for combo keys).

Trigger key on Mac keyboards. Keys are mapped by physical position, so the default trigger (evdev 41) is the key left of 1: ` on ANSI keyboards, § on ISO keyboards (the web UI and System panel show the right keycap; the ` key beside Left Shift keeps typing). Right Option, Right Command, Right Control, F13-F15 and Fn/Globe (evdev 464, config only) also work as triggers. The System panel lists the event-tap and permission state.

Verified end to end on an M2 / macOS 26 (built-in mic, launchd, MyWhispr.app permissions): English short text typed directly, Ukrainian pasted via the clipboard with Cmd+V, and the previous clipboard restored after the grace window. Live streaming into apps is off by default in the generated config.

Models on macOS. CUDA-only catalogue entries are hidden. Measured on an M2 MacBook Pro (16 GB), 6.5 s English clip:

Model	Backend	Load	Latency (warm)	Memory
Parakeet TDT 0.6B v3 (MLX), default	`mlx_parakeet` in `.venv-mlx-asr`	~1-3 s	~0.2 s	~0.9 GB RSS, ~1.3 GB MLX active (2.0 GB peak)
Whisper Large v3 Turbo Q5	whisper.cpp Metal (`/opt/homebrew/bin/whisper-server`)	~0.6 s	~1.4-1.8 s	~0.75 GB RSS

Parakeet v3 detects the language itself (25 European languages including English and Ukrainian). Other GGML models (large-q5, turbo, small, ...) download with one click in the web UI.

Linux / GNOME

git clone https://github.com/ibmua/MyWhispr.git
cd MyWhispr
cp config.example.json config.json
./scripts/install.sh

Open the local UI:

xdg-open http://127.0.0.1:16666/

Windows Notes

The installer sets "gpu_asr_python": "./.venv-gpu-asr/Scripts/python.exe" and uses parakeet-tdt-0.6b-v3 as the first-run default. Qwen model entries point at ./.venv-qwen-asr/bin/python, which resolves to the Windows Scripts\python.exe path at runtime. Workers use offline model loading, so the installer downloads Parakeet during setup by default and the UI downloader caches additional model weights before loading them. The tray icon shows daemon state; the trigger key from triggers (grave by default) is captured by a global keyboard hook, so it does not type into the focused app.

Daily Use

The default workflow is:

Focus any text field, terminal, editor, or chat box.
Hold the configured trigger key.
Speak.
Release the trigger key.
MyWhispr transcribes locally and inserts the final text into the focused app.

Combo shortcuts can switch language or mode while the trigger is held. For example, a setup can use one combo key for English, another for Ukrainian, and another for non-streaming dictation.

You do not have to wait for the previous take to finish: as soon as the trigger is released the next press starts a new recording immediately, while the previous recording is transcribed and inserted in the background. Takes queue up and land in the order they were spoken; the web UI header shows how many are still in flight.

Trigger Key And Modifiers

By default MyWhispr only starts dictation on a bare press of the trigger key. If you hold a modifier at the same time, the real keystroke is passed through untouched, so the trigger stays usable for normal typing:

grave alone → starts dictation (hold to record, release to insert)
Shift + grave → types ~ (tilde)
Ctrl / Alt / AltGr / Super + grave → your usual shortcut

This is standard behavior and needs no setup. It works because two things line up: the installed GNOME custom keybinding is bound to the exact bare key (so it never fires for modifier combos), and the daemon's evdev fallback skips a start when any of Shift, Ctrl, Alt, or Super is held.

To turn it off (always start on the raw keycode, ignoring modifier state), set ignore_when_modifier_held to false on the trigger in config.json:

"triggers": {
  "grave": {
    "binding": "grave",
    "stop_on_release_codes": [41],
    "ignore_when_modifier_held": false
  }
}

Restart the daemon after editing config: systemctl --user restart mywhisprd.

Commands

systemctl --user status mywhisprd --no-pager
journalctl --user -u mywhisprd --no-pager -n 120
./bin/mywhisprctl status
./bin/mywhisprctl start grave
./bin/mywhisprctl stop
./bin/mywhisprctl start --force      # hands-free: no trigger key held; end with "stop"
./bin/mywhisprctl stop --no-paste    # keep the take in history only; nothing is inserted

start --force plus stop --no-paste is also the way to script a test dictation without typing into whatever app happens to be focused.

Models

Whisper.cpp GGML models can be stored in models/:

./scripts/download-model.sh large-v3-q5_0

Configured Hugging Face / NeMo GPU ASR models can be downloaded into the local HF cache:

./scripts/download-gpu-asr-model.sh parakeet-tdt-0.6b-v3
./scripts/download-gpu-asr-model.sh qwen3-asr-0.6b
./scripts/download-gpu-asr-model.sh canary-1b-v2

The included model catalog covers whisper.cpp models plus GPU ASR backends for Qwen, Parakeet, Canary, Cohere, Granite, and Seamless M4T. The web UI can also add external API models. The default API template is remote-large-q5, a LAN whisper.cpp Large Q5 server at http://192.168.50.100:18178/inference; OpenAI transcription models are also available if an API key is configured. Model weights, virtualenvs, local config, and transcript scratch files are intentionally ignored by git.

The Settings panel can also expose this MyWhispr instance as an authenticated LAN transcription API. When enabled, it serves OpenAI-style multipart transcriptions and shows copyable client JSON that another MyWhispr instance can paste into its API model configuration.

Architecture

MyWhispr is split into small local components:

mywhispr/daemon.py: state machine, recording lifecycle, history, config, and final paste orchestration.
mywhispr/recorder.py: PipeWire recording.
mywhispr/transcriber.py: final ASR requests and text cleanup.
mywhispr/streaming.py: live preview and optional app-output streaming.
mywhispr/paste.py: platform output backend for typing, paste, copy, and backspace.
mywhispr/web.py and webui/: local control-panel API and React UI. webui/app.jsx owns four workspace views (Dictation, Models, Settings, System) and API state; hidden panels stay mounted to preserve drafts and filters. webui/components/ owns each panel. webui/app.css owns shared design tokens and responsive layouts (final Workspace design section); webui/index.html owns asset cache versions and the webui/favicon.svg reference. These files are served directly from disk: reload the browser after UI-only edits; daemon changes still require a process restart. Model language/path details use native disclosure controls; selecting a model retains the existing API.
extensions/mywhispr@local/: GNOME Shell top-bar indicator.

The detailed implementation plan and design constraints live in REWRITE_DESIGN.md.

Privacy And Safety

The daemon does not send audio to cloud APIs unless an external API model is explicitly selected.
The default history is RAM-only and disappears when the daemon restarts.
Temporary recordings live under /run/user/$UID, which is tmpfs on typical Linux desktops.
The web UI binds to 127.0.0.1 by default.
The shared transcription API is disabled by default and requires an API key before it can start.
Transcript text is not logged by default.
Config and custom words live in config.json; treat that file as user-confidential.

Known Limits

The web UI is desktop-oriented. It is not currently designed as a mobile UI.
GNOME integration is first-class; other desktops may need launcher or shortcut work.
Windows has an output backend scaffold, not a full supported release.
Non-ASCII dictation uses the paste fallback by default because direct typing is limited to short printable ASCII text on the current Linux path.
macOS: MyWhispr sends no output while the screen is locked (the take stays in history); triggers do not fire while a password field has secure input.
macOS: Fn/Globe works as a trigger (evdev 464) but can only be set in config.json; browsers cannot capture it for the web UI.
macOS: opening MyWhispr.app while the LaunchAgent daemon runs just opens the control panel. The launcher's --python mode (used by test harnesses) runs scripts with MyWhispr's privacy grants — equivalent to editing the user-writable venv, but worth knowing. A pending launcher fix (blocking SIGTERM around posix_spawn) is held back because rebuilding the launcher changes its signature and macOS then asks for all permissions again.

Name		Name	Last commit message	Last commit date
assets		assets
bin		bin
docs/screenshots		docs/screenshots
extensions/mywhispr@local		extensions/mywhispr@local
launchd		launchd
mywhispr		mywhispr
scripts		scripts
systemd		systemd
tests		tests
webui		webui
.gitignore		.gitignore
AGENTS.md		AGENTS.md
LICENSE		LICENSE
README.md		README.md
REWRITE_DESIGN.md		REWRITE_DESIGN.md
config.example.json		config.example.json

Navigation Menu

Repository navigation

About

Repository files navigation

MyWhispr

Screenshots

What It Does

Platform Support

How Text Gets Inserted

Requirements

Install

Windows

macOS

Linux / GNOME

Windows Notes

Daily Use

Trigger Key And Modifiers

Commands

Models

Architecture

Privacy And Safety

Known Limits

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

MyWhispr

Screenshots

What It Does

Platform Support

How Text Gets Inserted

Requirements

Install

Windows

macOS

Linux / GNOME

Windows Notes

Daily Use

Trigger Key And Modifiers

Commands

Models

Architecture

Privacy And Safety

Known Limits

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages