Stars
A Conversational Speech Generation Model
Unified automatic quality assessment for speech, music, and sound.
LLaSA: Scaling Train-time and Inference-time Compute for LLaMA-based Speech Synthesis
Large Concept Models: Language modeling in a sentence representation space
Official PyTorch implementation of "Paralinguistics-Aware Speech-Empowered LLMs for Natural Conversation" (NeurIPS 2024)
Awesome-LLM: a curated list of Large Language Model
SCOREQ: Speech COntrastive REgression for Quality Assessment (NeurIPS 2024)
Official code for "F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching"
SALMONN family: A suite of advanced multi-modal LLMs
An Open-Sourced LLM-empowered Foundation TTS System
High-quality Text-to-Audio Generation with Efficient Diffusion Transformer
Encode and decode audio samples to/from compressed latent representations!
Official implementation of the paper "BigCodec: Pushing the Limits of Low-Bitrate Neural Speech Codec"
Moshi is a speech-text foundation model and full-duplex spoken dialogue framework. It uses Mimi, a state-of-the-art streaming neural audio codec.
The open source code for SimpleSpeech series
LLaMA-Omni is a low-latency and high-quality end-to-end speech interaction model built upon Llama-3.1-8B-Instruct, aiming to achieve speech capabilities at the GPT-4o level.
SSR-Speech: Towards Stable, Safe and Robust Zero-shot Speech Editing and Synthesis
✨✨[NeurIPS 2025] VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction
Evaluation Protocol for Large-Scale Zero-Shot TTS Literature
AAAI 2025: Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Model
The hub for audio AI research: papers, open models, benchmarks & datasets across audio LLMs, speech recognition, TTS, music & audio generation.
The official repo of Qwen2-Audio chat & pretrained large audio language model proposed by Alibaba Cloud.
[ICLR & NeurIPS 2025] Repository for Show-o series, One Single Transformer to Unify Multimodal Understanding and Generation.
[ACMMM'2024] Generative Expressive Conversational Speech Synthesis
Official PyTorch implementation of BigVGAN (ICLR 2023)