Highlights
- Pro
Lists (5)
Sort Name ascending (A-Z)
Starred repositories
Disaggregated Prefill-Decode LLM serving engine on mini-sglang with a proved KV page state machine (PdLedger)
Formal verification workflow and skill stack for coding agents (TLA+, Z3, Apalache, TLAPS)
DeepInfer: High-performance, large-scale inference framework for DeepSeek models
An open, visual atlas of looped transformers, recurrent computation, adaptive depth, and latent reasoning.
Kernel Design Agents (KDA) is a agent-centric workflow to write high-performance CUDA Kernels.
SoL-Pi: Scaling Auto-Research Loops for Efficient Agent Harnesses
DeepSelect: TopK kernels for DeepSeek Sparse Attention (DSA) and Samplers
A lightweight library for xPU kernel JIT compilation
Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse
FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.
Dynamic Memory Management for Serving LLMs without PagedAttention
Archived: b12x development has moved to FlashInfer. See README for migration and PR-porting guidance.
RTX 6000 Pro Wiki — Running Large LLMs (Qwen3.5-397B, Kimi-K2.5, GLM-5) on PCIe GPUs without NVLink
Closed-loop fault localization and checkpoint recovery for LLM Pre-training
DeepSeek Harness: Everything is a Plugin.
AgentENV (AENV) is a distributed platform for running agent environments at scale.
MoonEP: A Perfectly Balanced Expert Parallelism Library via Dynamic Redundant Experts
FlashRT is a high-performance realtime inference engine for small-batch, latency-sensitive AI workloads. The flagship integration is production VLA control for Pi0, Pi0.5, GROOT N1.6, and Pi0-FAST.…
Production-ready MoE load balancing via real-time expert replication
JetSpec++/JetSpec: Causal Parallel (Tree) Drafting with Self Refinement
DeepSpec: a full-stack codebase for training and evaluating speculative decoding algorithms
Block-GTQ: RoPE-aware bit allocation for KV-cache quantization