Lists (2)
Stars
A Datacenter Scale Distributed Inference Serving Framework
Kubernetes enhancements for Network Topology Aware Gang Scheduling & Autoscaling
Making SLOs with Prometheus manageable, accessible, and easy to use for everyone!
Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
Evolving the Prometheus exposition format into a standard.
Intelligent LLM routing plugin for DeepSeek Harness: route groups, 4 routing strategies, intent rule engine, cache-affinity Stays mode, auto failover and sample settings UI.
Privacy-aware local and cloud model collaboration for Codex, with local redaction and human review.
Run Slurm on Kubernetes. A Slinky project.
verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework
Get your documents ready for gen AI
Development repository for the Triton language and compiler
A modern model graph visualizer and debugger
On-device AI across mobile, embedded and edge for PyTorch
An open-source database of AI models.
An app that brings language models directly to your phone.
An Open-source, self-hosted AI model hub with Hugging Face compatibility, accelerating vLLM/SGLang performance.
DeepEP: an efficient expert-parallel communication library
Monarch CRD and operator for native kubernetes support
procfs provides functions to retrieve system, kernel and process metrics from the pseudo-filesystem proc.
Next Generation Agentic Proxy for AI Agents and MCP servers
Production-ready MCP server for Apache DolphinScheduler (小海豚) — lets AI agents build, schedule, run and recover workflows