Stars
Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes
verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework
🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.
NVIDIA curated collection of educational resources related to general purpose GPU programming.
An agentic context evolution approach for RTL coding
Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost…
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface. The fastest local inference engine in the world.
PersonaPlex on Apple Silicon: an MLX port of NVIDIA’s full-duplex speech-to-speech model with realtime local/web modes and offline WAV inference.
CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation
End-to-end benchmark for AI-generated GPU kernels, drawn from real production traces — turn a PyTorch reference into a DSL kernel (Triton, Gluon, FlyDSL, CuteDSL) and grade it on compilation, numer…
An end-to-end agent project for GPU kernel implementation, analysis, profiling, and iterative optimization. It helps an agent turn PyTorch logic or an existing kernel into a high-performance GPU ke…
TokenSpeed is a speed-of-light LLM inference engine.
Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing,…
AlpaSim is an open-source autonomous vehicle simulation platform designed for development and testing of end-to-end AV policies
A safetensors extension to efficiently store sparse quantized tensors on disk
Flash Vision-Language-Action Inference for Autonomous Driving
CLI proxy that reduces LLM token consumption by 60-90% on common dev commands. Single Rust binary, zero dependencies
Open Source AI Platform - AI Chat with advanced features that works with every LLM
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
DSPy: The framework for programming—not prompting—language models
ARIS ⚔️ (Auto-Research-In-Sleep) — Lightweight Markdown-only skills for autonomous ML research: cross-model review loops, idea discovery, and experiment automation. No framework, no lock-in — works…
Offline optimization of your disaggregated Dynamo graph
A Datacenter Scale Distributed Inference Serving Framework