docker-orpheus is a Dockerized version of Orpheus TTS, an open-source text-to-speech system built on the Llama-3b backbone. Orpheus demonstrates the emergent capabilities of using LLMs for speech synthesis.
Check out the Orpheus blog post
- Human-Like Speech: Natural intonation, emotion, and rhythm that is superior to SOTA closed source models
- Zero-Shot Voice Cloning: Clone voices without prior fine-tuning
- Guided Emotion and Intonation: Control speech and emotion characteristics with simple tags
- Low Latency: ~200ms streaming latency for realtime applications, reducible to ~100ms with input streaming
Before running the Docker container, you need to set up your Hugging Face token to download the Orpheus TTS model:
- Create an account on Hugging Face if you don't have one
- Go to your Hugging Face tokens page
- Create a new token with read access
- Add your token to the
.envfile:
# .env file
PORT=7325
HUGGINGFACE_TOKEN=your_huggingface_token_hereAfter setting up your Hugging Face token, run the Docker container:
docker-compose upThis will:
- Download the Orpheus TTS models from Hugging Face:
- Finetuned model:
canopylabs/orpheus-tts-0.1-finetune-prod - Pretrained model:
canopylabs/orpheus-3b-0.1-pretrained
- Finetuned model:
- Start two services:
- Web UI: Available at
http://localhost:7325 - API: Available at
http://localhost:7324
- Web UI: Available at
The Web UI provides a user-friendly interface for the Orpheus TTS system. It allows you to:
- Enter text to convert to speech
- Select from available voices
- Adjust advanced parameters like temperature and repetition penalty
- Add emotive tags to enhance speech generation
The Web UI communicates with the API service to generate speech, providing a seamless experience for users.
You can add the following emotive tags to your text:
<laugh>- Add laughter<chuckle>- Add a chuckle<sigh>- Add a sigh<cough>- Add a cough<sniffle>- Add a sniffle<groan>- Add a groan<yawn>- Add a yawn<gasp>- Add a gasp
Example: "I just heard the funniest joke it was about a chicken crossing the road."
The API provides a streaming text-to-speech endpoint that can be used to generate audio from text in real-time.
Converts text to speech and streams the audio as a WAV file.
Parameters:
prompt(required): The text to convert to speechvoice(optional, default: "tara"): The voice to use for speech synthesis. Available voices: "tara", "leah", "jess", "leo", "dan", "mia", "zac", "zoe"model(optional, default: "finetune-prod"): The model to use. Available models: "finetune-prod", "pretrained"
Example:
http://localhost:7324/api/tts?prompt=Hello%20world&voice=tara&model=finetune-prod
Lists all available voices.
Example:
http://localhost:7324/api/voices
Lists all available models.
Example:
http://localhost:7324/api/models
Health check endpoint.
Example:
http://localhost:7324/api/health
The API documentation is available at:
http://localhost:7324/api/docs
-
The
finetune-prodmodels: for the primary model, your text prompt is formatted as{name}: I went to the .... The options for name in order of conversational realism (subjective benchmarks) are "tara", "leah", "jess", "leo", "dan", "mia", "zac", "zoe". Our python package does this formatting for you, and the notebook also prepends the appropriate string. You can additionally add the following emotive tags:<laugh>,<chuckle>,<sigh>,<cough>,<sniffle>,<groan>,<yawn>,<gasp>. -
The pretrained model: you can either generate speech just conditioned on text, or generate speech conditioned on one or more existing text-speech pairs in the prompt. Since this model hasn't been explicitly trained on the zero-shot voice cloning objective, the more text-speech pairs you pass in the prompt, the more reliably it will generate in the correct voice.
Additionally, use regular LLM generation args like temperature, top_p, etc. as you expect for a regular LLM. repetition_penalty>=1.1is required for stable generations. Increasing repetition_penalty and temperature makes the model speak faster.
This project provides a zero-shot voice cloning TTS model intended for academic research, educational purposes, and legitimate applications, such as personalized speech synthesis, assistive technologies, and linguistic research.
Please note:
- Do not use this model for unauthorized voice cloning, impersonation, fraud, scams, deepfakes, or any illegal activities.
- Ensure compliance with local laws and regulations when using this model and uphold ethical standards.
- The developers assume no liability for any misuse of this model.
We advocate for the responsible development and use of AI and encourage the community to uphold safety and ethical principles in AI research and applications. If you have any concerns regarding ethics or misuse, please contact us.