Stars
DeepInfer: High-performance, large-scale inference framework for DeepSeek models
lunar.dev: Agent native MCP Gateway for governance and security
Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serving systems.
A framework for few-shot evaluation of language models.
TokenSpeed is a speed-of-light LLM inference engine.
Vera: a programming language designed for LLMs to write
Offline optimization of your disaggregated Dynamo graph
A PyTorch-native inference engine with cache, parallelism, quantization and cpu offload for 100+ DiTs.
Visualize CPython's specializing, adaptive interpreter. 🔥
Our first fully AI generated deep learning system
FlagTree is a unified compiler supporting multiple AI chip backends for custom Deep Learning operations, which is forked from triton-lang/triton.
Autonomous GPU Kernel Generation & Optimization via Deep Agents
magic-trace collects and displays high-resolution traces of what a process is doing
🤖 MLE-Agent: Your intelligent companion for seamless AI engineering and research. 🔍 Integrate with arxiv and paper with code to provide better code/research plans 🧰 OpenAI, Anthropic, Gemini, Ollam…
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
A Python-embedded DSL that makes it easy to write fast, scalable ML kernels with minimal boilerplate.
Wave: Python Domain-Specific Language for High Performance Machine Learning
jax-triton contains integrations between JAX and OpenAI Triton
Mirage Persistent Kernel: Compiling LLMs into a MegaKernel
Development repository for the Triton-Linalg conversion
Efficient Triton Kernels for LLM Training