8000 GitHub - zce/toas: Talk Once, Act Smart. Push-to-talk voice input for GNOME Shell (Fedora / Wayland): record, transcribe, refine, and insert into the focused application. · GitHub
[go: up one dir, main page]

Skip to content
zcePublic

About

Talk Once, Act Smart. Push-to-talk voice input for GNOME Shell (Fedora / Wayland): record, transcribe, refine, and insert into the focused application.

Resources

Stars

3 stars

Watchers

0 watching

Forks

Repository files navigation

toas

Talk Once, Act Smart.

Push-to-talk voice input for Fedora, GNOME, and Wayland.
Say it once. Keep moving.

Homepage · Source · MIT License

GNOME Shell 49 and 50 Wayland native MIT license

Cloud intelligence, local lightness: a small desktop-native voice layer for the moments when typing is slower than thinking.

What It Does

Demo

toas turns one held shortcut into text in the application that is already in focus:

record -> transcribe -> optional refine -> focused application

The extension lives in the GNOME top bar. Hold the default Ctrl+Shift+Space, speak, and release the modifiers. The standard overlay shows a waveform and processing-stage labels. An optional experimental overlay uses a dot-matrix waveform and processing indicator instead, without opening another application.

Cloud Intelligence, Local Lightness

toas keeps the intelligence in the cloud and the desktop layer deliberately small. It captures audio locally, sends the completed recording through your configured transcription provider, and returns text to the application in focus. Each provider owns its own request protocol, while the local extension stays focused on recording, orchestration, history, and output.

There are no model weights to download, no local inference runtime to maintain, and no always-on model service consuming your machine's resources.

Processing begins after you finish a recording: one transcription request, plus one optional Refine request when enabled. Choose from the providers and models supported by toas based on the speed, quality, policy, and cost that fit your workflow.

Privacy, Clearly Bounded

On your desktop: our local promise is concrete. There is no background listening, telemetry, local model runtime, or always-on inference service. Audio is captured only during an explicit recording and uploaded only after you stop.

toas never automatically inspects your desktop, editor, clipboard, files, or project. The only reference material it sends is the Context text you explicitly configure, and only to providers that support it.

Voice input text and retained recordings are kept locally for history and retry, bounded by your retention settings and removable from the top-bar menu.

For a zero-retention tier, turn on Private mode from the top-bar menu. New voice inputs are transcribed, refined, and inserted as usual, but nothing is written to history and each recording is deleted as soon as processing finishes — including recordings from failed attempts, which are not retained for retry while Private mode is on.

Private mode stays enabled until you turn it off. Existing history remains available.

Private mode changes only what toas keeps on your disk. It does not change what is uploaded.

After upload: your chosen provider's policy applies. Completed audio goes to your configured transcription provider. Optional Refine sends transcript text to its configured provider. Retention, training, logging, and data residency are determined by those services, not by the local extension.

Why toas

One gesture, no context switching

Push-to-talk keeps the interaction predictable. Left-click the top-bar microphone to toggle recording, right-click it for history and settings, or choose another shortcut from Preferences.

Clean words, unchanged meaning

The optional Refine stage can remove filler, fix punctuation, and repair broken speech according to the instructions you configure, without changing the transcription path itself.

If Refine fails, the On refine failure setting decides whether toas falls back to the transcription result or fails the voice input.

Output that respects your target

Text is committed directly when the target application supports it, including multiline text outside standalone terminals. Otherwise, toas uses the clipboard to preserve whitespace and formatting, with terminal-aware paste shortcuts.

If the original target window is unavailable or focus changes during processing, the result stays on the clipboard instead of being inserted into another window.

Compatibility

Component Target
Distribution Fedora
Desktop shell GNOME Shell 49 / 50
Display server Wayland
Audio capture pw-record from PipeWire
Transcription Qwen (recommended), Doubao, MiMo
Optional Refine MiMo, OpenAI, or OpenAI-compatible

Install

One-line install

The online installer downloads the current main snapshot from GitHub and runs the same installer used by a local checkout:

curl -fsSL https://toas.cc/install.sh | bash

Remove the extension with:

curl -fsSL https://toas.cc/uninstall.sh | bash

The online installer needs curl and tar. The extension installer checks for glib-compile-schemas, gnome-extensions, and pw-record before installing.

Local install

sudo dnf install pipewire-utils glib2
./install.sh

If GNOME does not immediately discover the extension, log out and back in, then run:

gnome-extensions enable toas@zce.me

Open Preferences with:

gnome-extensions prefs toas@zce.me

First Run

On first enable, a one-time notice explains the shortcut and discloses that audio is uploaded to your configured transcription provider while voice input text and some recordings may be kept on disk.

Recording is blocked until the selected transcription provider has the required API key.

Preferences includes connection tests for both transcription and Refine so the configured provider can be verified before normal use.

Configure

Open GNOME Preferences for toas.

The default transcription configuration is the speed-first recommendation:

provider = qwen
model    = qwen-audio-3.1-asr-flash

The Model field accepts any model ID, while each Provider uses one fixed API contract. For Qwen, that contract is synchronous DashScope Audio-ASR HTTP (the input_audio envelope). Models using other protocols are not supported by that Provider. An endpoint override is available for advanced configurations.

Doubao with volc.bigasr.auc_turbo (BigASR Flash) and MiMo with mimo-v2.5-asr are also supported for transcription.

Looking for a starting point? See Recipes for practical Context and Refine Instructions you can copy, combine, and adapt.

Context

Context is free text you write once — domain terms, background, names, or anything else that may help processing.

Providers that support Context receive the text verbatim. Providers without Context support never receive it.

Refine

Refine is optional and currently runs as a separate text-processing step after transcription.

Supported Refine providers are:

  • MiMo
  • OpenAI
  • OpenAI-compatible endpoints

Configure the provider, model, API key, and your own Refine instructions in Preferences.

The On refine failure setting controls what happens when Refine fails:

  • Use transcription — continue with the transcription result.
  • Fail voice input — treat the entire voice input as failed.

API keys

Keys entered in Preferences are stored as plain text in GNOME settings.

Each provider also declares environment-variable fallbacks for its credentials and advanced connection settings. Leave a key empty in Preferences and provide the corresponding environment variable before the GNOME session starts if you prefer not to store it in dconf.

Secret Service integration is not implemented.

Use It

  1. Enable toas and configure a transcription provider API key.
  2. Hold Ctrl+Shift+Space while speaking.
  3. Release the modifiers to stop and process the recording.
  4. Let toas insert the result, or use the clipboard when automatic insertion is disabled.

A shortcut without modifiers behaves as a start/stop toggle because GNOME Shell does not expose a matching shortcut-release callback.

To change the shortcut, click the shortcut button in Preferences. Escape cancels and Backspace disables it.

Failure Handling

Recording, processing, and insertion failures show the error in the overlay and send a desktop notification with a next action.

If Refine fails, behavior follows the configured On refine failure policy.

The standard overlay's close button or the experimental overlay's capsule cancels a live voice input.

If the original target window is unavailable or focus changes during processing, the result stays on the clipboard with a notice.

Failed processing retains its recording when possible so it can be retried from history without recording again. Private mode and Saved recordings = 0 make retry-from-audio unavailable for affected voice inputs.

Recording

pw-record captures mono signed 16-bit PCM in 100 ms chunks. Those chunks drive the live waveform, then the completed recording is wrapped in a standard WAV container before processing.

The Audio quality preference controls the sample rate for new recordings:

Preset Sample rate Approximate cap
Minimum 8 kHz 26 minutes
Low 12 kHz 17 minutes
Standard 16 kHz 13 minutes
High 24 kHz 9 minutes
Maximum 48 kHz 4 minutes

Standard is the default.

Audio is never uploaded during capture. Very short recordings are ignored to avoid accidental activations. The threshold is configurable in Preferences and defaults to 600 ms.

Recordings are capped at 24 MB of PCM data so an accidentally open recording cannot exhaust GNOME Shell memory during upload. A recording that reaches the cap stops and processes what was captured.

History

Recent voice inputs are listed directly in the top-bar menu. Each row has a copy action; failed voice inputs also offer retry while their recording is still retained.

Retry reprocesses the stored recording and appends a linked attempt without pasting.

The Private mode switch above the list suspends retention for new voice inputs. Existing history stays visible and Clear History keeps working while it is on.

History is stored under:

${XDG_STATE_HOME:-~/.local/state}/toas/
  history.jsonl
  recordings/*.wav

History entries defaults to 30 and limits saved voice inputs. Retry attempts stay attached to their original voice input and do not consume additional history slots. Saved recordings defaults to 20 and limits retained WAV files. Set Saved recordings to 0 to keep text history without retaining completed audio. Recording duration and sample-rate metadata remain even when the WAV file is no longer retained.

The top-bar Clear History action asks for confirmation and is disabled while recording.

Technical details

Processing architecture

Voice processing runs through a small runtime-agnostic kernel with pluggable providers.

A voice input resolves to an ephemeral plan of one or two physical steps:

  1. audio-to-text transcription
  2. an optional separate text Refine step

Each provider owns its protocol mapping. The GNOME host owns HTTP execution, persistence, recording, and output.

History stores the final text together with structured audio, transcribe, and optional refine details. Processing stages retain their own text, provider/model, latency, available usage, and request/response IDs — never raw HTTP bodies or credentials.

Qwen transcription

The default model is qwen-audio-3.1-asr-flash, using the synchronous DashScope Audio-ASR HTTP protocol. The Model field is editable; compatible models such as qwen-audio-3.0-asr-flash and fun-asr-flash-2026-06-15 can be entered without a TOAS release.

The default Endpoint is the Beijing DashScope URL: https://dashscope.aliyuncs.com/api/v1/services/aigc/multimodal-generation/generation. Workspace-specific domains can be entered in Advanced settings when a Workspace ID is available. A saved blank Endpoint resets to the environment override or Provider default; non-empty custom endpoints are preserved.

The request uses input.messages with an input_audio part. When configured, Context is sent as an input_text message preceding the recording. Model IDs do not change the endpoint, request envelope, or response decoder. In particular, the older qwen3-asr-flash protocol is no longer supported by this Provider. For an existing configuration using it, select an Audio-ASR-compatible model.

Doubao transcription

Doubao uses the recording-file Flash HTTP API with the default Model volc.bigasr.auc_turbo. The editable Model value is transmitted as the provider's X-Api-Resource-Id header; the request body's model_name remains bigmodel. The newer standard 2.0 recording-file service requires submit/query and is not compatible with the Flash protocol. Doubao supports Context through the native corpus field and is available for transcription only. Its normal silence business code (20000003) counts as no recognized speech in the Connection Test.

MiMo transcription and Refine

MiMo uses the same Chat Completions API for both roles, with independent editable model fields and defaults:

Transcription: mimo-v2.5-asr
Refine:        mimo-v2.6-flash

The primary role supplies audio; the Refine role supplies text. The selected model ID does not determine the request shape. Choose a compatible model for each role, as the service may reject an unsupported ID or modality.

MiMo defaults to the pay-as-you-go API base URL https://api.xiaomimimo.com/v1. If using Token Plan, set the advanced Endpoint to https://token-plan-cn.xiaomimimo.com/v1 explicitly. Existing saved Endpoint overrides are preserved.

The Refine stage supports configurable Instructions and Context. Toas disables thinking by default for MiMo and supported native OpenAI models. Generic OpenAI-compatible endpoints keep Provider default because their reasoning extensions vary; choose Off under Advanced when the endpoint supports reasoning_effort: "none".

OpenAI and OpenAI-compatible Refine

OpenAI and OpenAI-compatible providers are text-only in toas and are used for Refine, not transcription.

The official OpenAI provider defaults to:

gpt-4o-mini

The OpenAI-compatible provider accepts a user-configured service base URL and model ID using the Chat Completions wire contract. Its optional Off setting uses the OpenAI-style reasoning_effort: "none" extension; endpoints with a different or unsupported control should keep Provider default.

Refine behavior

Refine currently makes one non-streaming text request after transcription. This is the product execution path, not a Provider capability mode; Providers describe the inputs and capabilities they actually support.

Its instructions are configurable free text. Providers that support Context also receive your configured Context text.

Refine never changes which provider or model performs the original audio transcription.

Output behavior

When direct text input is unavailable, the final output is copied to the GNOME clipboard and pasted through a compositor-side Clutter virtual keyboard.

Standalone terminals use Ctrl+Shift+V; other applications, including IDE-embedded terminals, use Shift+Insert.

The extension never synthesizes Enter. Embedded line breaks remain part of clipboard-pasted text.

When Restore clipboard is enabled, the previous text clipboard value is restored after an automatic clipboard-based paste. Direct insertion and copy-only delivery do not restore it. Rich or image clipboard content cannot be restored through St.Clipboard.

When Insert automatically is disabled, the result stays on the clipboard as the deliverable.

Debug

Watch extension logs:

journalctl --user -f -o cat /usr/bin/gnome-shell

Check extension state:

gnome-extensions info toas@zce.me

On Wayland, GNOME Shell's ES-module cache can make disable/enable insufficient after JavaScript changes. Log out and back in when updated code does not appear to load.

License

Released under the MIT License.

About

Talk Once, Act Smart. Push-to-talk voice input for GNOME Shell (Fedora / Wayland): record, transcribe, refine, and insert into the focused application.

Resources

Stars

3 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages

2BB8
0