Stars
TokenSpeed is a speed-of-light LLM inference engine.
AirLLM 2.8T LLM inference with single 4GB GPU
A Python-embedded DSL that makes it easy to write fast, scalable ML kernels with minimal boilerplate.
High-efficiency floating-point neural network inference operators for mobile, server, and Web
ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
Speech-to-text, text-to-speech, speaker diarization, speech enhancement, source separation, and VAD using next-gen Kaldi with onnxruntime without Internet connection. Support embedded systems, Andr…
Open standard for machine learning interoperability
Universal LLM Deployment Engine with ML Compilation
A hierarchical inference engine for frontier MoE models: hot experts on the GPU (CUDA, Vulkan), warm BA06 ones in RAM, cold ones streamed from NVMe. Written in C, runs on the hardware you already own.
DSPy: The framework for programming—not prompting—language models
High-performance In-browser LLM Inference Engine
WebAssembly Micro Runtime (WAMR)
NVIDIA curated collection of educational resources related to general purpose GPU programming.
CORE-ET Hackathon. AIFoundry x HuggingFace x OpenHW
The Torch-MLIR project aims to provide first class support from the PyTorch ecosystem to the MLIR ecosystem.
Multipass orchestrates virtual Ubuntu instances
This is an online course where you can learn and master the skill of low-level performance analysis and tuning.
The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.
a language for fast, portable data-parallel computation
A high-throughput and memory-efficient inference and serving engine for LLMs
A curated resource list for learning AI performance engineering, from GPU fundamentals to production inference.