Stars
SGLang is a high-performance serving framework for large language models and multimodal models.
Deprecated and no longer maintained. Langfuse is the recommended replacement: see the README.
[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.
State of the art inference performance with modern accelerators on Kubernetes
Verify Precision of all Kimi K2 API Vendor
[ICLR2025 Spotlight] SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models
[ICLR 2025 Oral] Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models
SANA & Sol: Efficient Generative Models and Full-Stack Training & Inference Engines for Image, Video, and World Modeling
Leaked system prompts of ChatGPT, Claude, Gemini and more. The hidden system prompt of every popular chatbot, in full, every version since May 2025. Also Anthropic Claude Code, OpenAI Codex, Google…
A Datacenter Scale Distributed Inference Serving Framework
My learning notes for ML SYS.
Beautiful iOS-like sheet transitions for React Native
The true native bottom sheet experience 💩
Modeling, training, eval, and inference code for OLMo
simple, expressive API for tailwindcss + react-native
xDiT: A Scalable Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism
Next.js Firebase Authentication for Edge and Node.js runtimes. Compatible with latest Next.js features.
Very low latency speech to text, intent recognition, and text to speech, for building voice agents and interfaces
🎨 The exhaustive Pattern Matching library for TypeScript, with smart type inference.
A benchmark for role-playing language models
The ultimate training toolkit for finetuning diffusion models
The repository provides code for running inference with the Meta Segment Anything Model 2 (SAM 2), links for downloading the trained model checkpoints, and example notebooks that show how to use th…
Quickly make and deploy full-stack apps with database, auth, styling, storage etc. figured out for you. Add all primitives you want.