A small toolkit I built on top of Supertone's Supertonic to turn the TTS engine into a local WebSocket server with a feature-rich browser client. Everything in this section lives in
tool/and is not part of the upstream project.
- 🚀 One-click / one-command launchers — Windows
.batplus Ubuntu/macOS.sh, both auto-install deps and boot the server. - ⚡ Accelerated when available — auto-detects CUDA → DirectML → CoreML → CPU with CPU fallback if an accelerator fails.
- 🧠 Batch inference — multiple requests within a 100 ms window are merged into a single GPU call, so 6 lines render as fast as 1.
- 🎭 Multi-line dialogue queue — assign a different voice/language per line for true conversational TTS.
- 📥 Bulk import — load dialogue from
.xlsx,.csv,.txt, or markdown tables. - 📦 Batch ZIP download — convert N lines, get N
.wavfiles in one zip. - 🔌 Open WebSocket API — drop-in client examples for Browser, Chrome MV3 extension, Node.js, Python.
- 🔄 Upstream sync helper —
sync_upstream.batkeeps the fork in lock-step withsupe 8000 rtone-inc/supertonic.
| File / Folder | Description |
|---|---|
ws_tts_server.py |
WebSocket TTS server. Auto-detects providers (CUDA / DirectML / CoreML / CPU), batches concurrent requests, retries inference failures, and handles client disconnect gracefully. |
tts_web.html |
Standalone browser client. Multi-line dialogue queue, per-line voice/lang, file import, sequential auto-play, batch ZIP export. No build step. |
start_tts_server.bat |
One-click Windows launcher. Runs uv sync, then starts the server with banner + usage hints. |
start_tts_server.sh |
Ubuntu/macOS launcher. Uses uv sync when available, falls back to python3 -m venv + pip, then starts the server. |
sync_upstream.bat |
Pulls latest upstream/main, merges into local main, and pushes to origin. Detects merge conflicts. |
WEBSOCKET_API.md |
Full protocol spec + integration guides (vanilla JS, Chrome MV3 extension with offscreen audio, Node, Python). |
samples/ |
Ready-to-import dialogue samples (.csv, .md table, .txt with F1: text prefix format). |
1. Set up the upstream project first — only needed once, see Upstream Setup below.
2. Start the server.
Windows:
cd tool
start_tts_server.batUbuntu/macOS:
cd tool
sh start_tts_server.shFrom the repo root, you can also run the Unix launcher directly:
chmod +x tool/start_tts_server.sh
./tool/start_tts_server.shThe server listens on ws://127.0.0.1:8765. First launch warms up the selected provider; subsequent inferences are faster.
3. Open the web UI:
Double-click tool/tts_web.html — it connects automatically.
The launchers and dependencies are selected by OS and CPU architecture so the same repo can run on more machines without hand-editing dependency files.
| Platform | Launcher | Default runtime path | Acceleration |
|---|---|---|---|
| Windows x64 | tool/start_tts_server.bat |
uv sync → py/.venv/Scripts/python.exe |
CUDA, DirectML, CPU fallback |
| Windows ARM64 | tool/start_tts_server.bat |
uv sync → py/.venv/Scripts/python.exe |
CPU fallback |
| Ubuntu/Linux x64 | tool/start_tts_server.sh |
uv sync or python3 -m venv fallback |
CUDA, CPU fallback |
| Ubuntu/Linux ARM64 | tool/start_tts_server.sh |
uv sync or python3 -m venv fallback |
CPU fallback |
| macOS Intel / Apple Silicon | tool/start_tts_server.sh |
uv sync or python3 -m venv fallback |
CPU fallback; CoreML only if your ONNX Runtime build exposes it |
Recommended prerequisites:
- Python 3.10+
uvfor the most reliable install pathgit-lfsfor downloading model assets
The Ubuntu/macOS launcher can fall back to python3 -m venv and pip if uv is not installed. The Windows launcher requires uv.
A self-contained HTML page (no build, no server). Open with any modern browser.
- Each row has its own Voice and Language selector — perfect for two-speaker dialogues, narration with character voices, or multi-language announcements.
- Visual highlight on the currently playing row.
Enteradds a new row below;Backspaceon an empty row deletes it.
| Action | How |
|---|---|
| Paste multi-line | Click 📋 Paste multi-line. Each pasted line becomes a row. Supports F1: hello / M2|vi: xin chào shorthand to set voice + lang inline. |
| Import file | Click 📥 Import. Supports .xlsx (via SheetJS), .csv, .md markdown tables, and plain .txt. |
| Voice assignment modes | Use default / Alternate F1↔M1 (auto-dialogue) / From column (read from imported file). |
- Convert All (Ctrl+Enter) — synthesizes the whole queue, plays back sequentially with the chosen playback speed (
preservesPitch = true, so no chipmunk effect). - Per-line audio player with individual
⬇download button (01_F1_en_Hello.wav). - ⬇ Download All (zip) — bundles everything into
tts_batch_YYYY-MM-DD.zip. - ⏹ Stop — halts mid-batch and cancels playback.
- Live indicator dot (green = connected, red = disconnected).
- Auto-reconnect on close with 2 s backoff.
At startup, the server probes ONNX Runtime providers in this order:
- CUDA (NVIDIA on Windows/Linux x86_64) — picks up bundled CUDA/NVIDIA libraries from the virtualenv when present.
- DirectML (Windows DX12 GPUs) — sets
enable_mem_pattern=Falseand sequential execution to dodge the DirectML command-queue overflow bug. - CoreML (macOS, only if the installed ONNX Runtime exposes it).
- CPU fallback.
If an accelerator is detected but fails during initialization or warmup, auto mode falls back to CPU instead of crashing the launcher.
Pass --cpu to skip accelerator detection entirely, or --provider cpu|cuda|directml|coreml|auto to choose a preferred provider.
- Incoming requests are pushed into an
asyncio.Queue. - A worker waits 100 ms to collect a batch, groups by voice (style), then runs one ONNX inference for the whole group.
- A global
asyncio.Lockserializes batches → DirectML stays happy. - Retry up to 3× on transient inference errors before surfacing an error to the client.
- Warmup on boot with
total_step=4thentotal_step=8to avoid DirectML's first-run timeout. - Disconnect-safe: if a client drops mid-inference, the server discards its result and keeps serving others.
- Echo
textin audio_meta so clients can pair binary frames with their original requests when many are in flight.
cd tool
uv run --project ../py python ws_tts_server.py [--port 8765] [--cpu] [--provider auto|cpu|cuda|directml|coreml]| Flag | Meaning |
|---|---|
--port N |
Override the WebSocket port (default 8765, env WS_PORT). |
--provider NAME |
Prefer auto, cpu, cuda, directml, or coreml (default auto, env TTS_PROVIDER). |
--cpu |
Shortcut for --provider cpu. |
Examples:
# Force CPU on any OS
sh start_tts_server.sh --cpu
# Use a custom port
sh start_tts_server.sh --port 9000
# Prefer CUDA on Linux
TTS_PROVIDER=cuda sh start_tts_server.shREM Force CPU on Windows
start_tts_server.bat --cpu
REM Use a custom port
start_tts_server.bat --port 9000Connect: ws://127.0.0.1:8765
Handshake (server → client, on connect):
{ "status": "connected", "voices": ["F1", ..., "M5"], "languages": ["en", "vi", "ko", ...] }Request (client → server):
{ "text": "Hello world", "lang": "en", "voice": "M1", "speed": 1.05 }Response: for each request, the server sends two paired frames:
- JSON
audio_meta—{ type, text, duration, latency_ms, size } - Binary frame — a complete WAV file (16-bit PCM mono).
Options:
voice:M1–M5(male),F1–F5(female) — case-insensitive, falls back toM1.speed:0.25 – 4.0(apply on client viaaudio.playbackRate;preservesPitch = true).lang:en,vi,ko,ja,fr,de,es,pt,it, … (31 languages).- Inline markers in
text:<laugh>,<breath>,<sigh>, plus standard punctuation for natural pauses.
Full spec with Chrome MV3 / Node / Python examples → tool/WEBSOCKET_API.md.
The server reuses the upstream Python runtime in py/ and the ONNX assets in assets/. You need these once:
# 1. Download ONNX models + preset voices (requires git-lfs)
git lfs install
git clone https://huggingface.co/Supertone/supertonic-3 assets
# 2. Install Python deps for the upstream runtime
cd py
uv sync
cd ..After that, start_tts_server.bat on Windows or start_tts_server.sh on Ubuntu/macOS handles the rest.
| Symptom | Fix |
|---|---|
Missing ONNX assets |
Run the asset setup commands above from the repo root. |
uv is required but was not found on Windows |
Install uv, then reopen the terminal and run tool/start_tts_server.bat again. |
Ubuntu/macOS says python3 is required |
Install Python 3.10+ for your OS, or install uv. |
| CUDA/DirectML/CoreML fails at startup | Run with --cpu; auto mode should also fall back to CPU when possible. |
| Port already in use | Run with --port 9000, or set WS_PORT before starting the launcher. |
| macOS is slow | This is expected on CPU fallback. Use shorter batches or fewer concurrent lines. |
If you fork this repo, tool/sync_upstream.bat automates the maintenance loop:
cd tool
sync_upstream.batIt runs git fetch upstream → git merge upstream/main → git push origin main, and stops cleanly if it detects a conflict so you can resolve it manually.
Supertonic by Supertone Inc. is a lightning-fast, on-device TTS system powered by ONNX Runtime. It runs fully offline, supports 31 languages, and ships with ready-to-use examples in Python, Node.js, Browser, Java, C++, C#, Go, Swift, Rust, iOS, and Flutter (see the corresponding folders in this repo).
- Models & demo: Hugging Face — Supertonic 3 · Interactive Demo
- Python SDK:
pip install supertonic— docs at supertone-inc.github.io/supertonic-py - Upstream README & citations: see the original repository for architecture details, benchmarks, paper citations, and per-language examples.
- Sample code (including
tool/): MIT — seeLICENSE. - ONNX model weights: OpenRAIL-M — see the model license on Hugging Face.
Upstream © 2026 Supertone Inc. Additions in tool/ © their respective author.