8000 vllm · GitHub Topics · GitHub

#

vllm

Here are 4,019 public repositories matching this topic...

modelscope / FunASR

Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.

Updated Nov 6, 2026
Python

meta-llama / llama-cookbook

Welcome to the Llama Cookbook! This is your go to guide for Building with Llama: Getting started with Inference, Fine-Tuning, RAG. We also show you how to solve end to end problems using Llama model family and using them on various provider services

python machine-learning ai pytorch llama finetuning llm langchain vllm llama2

Updated May 19, 2026
Jupyter Notebook

budtmo / docker-android

Android in docker solution with noVNC supported, video recording, mcp server and AI-agent

Updated Nov 5, 2026
Python

Orchestra-Research / AI-Research-SKILLs

Comprehensive open-source library of AI research and engineering skills for any AI model. Package the skills and your claude code/codex/gemini agent will be an AI research agent with full horsepower. Maintained by Orchestra Research.

ai skills gemini codex claude ai-research machine-leanring megatron huggingface gpt-5 vllm grpo claude-code claude-skills

Updated Oct 11, 2026
TeX

halfrost / Halfrost-Field

✍🏻 Source Code Deep Dives, System Design & Engineering Blogs | Halfrost-Field 冰霜之地：源码解析、系统设计与工程实践笔记

Updated Sep 30, 2026
Go

LMCache / LMCache

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

fast amd cuda inference pytorch speed rocm kv-cache llm vllm

Updated Nov 6, 2026
Python

OpenRLHF / OpenRLHF

An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO & REINFORCE++ & VLM & TIS & vLLM & Ray & Async RL)

reinforcement-learning raylib transformers proximal-policy-optimization large-language-models reinforcement-learning-from-human-feedback vllm visual-language-models

Updated Nov 3, 2026
Python

xorbitsai / inference

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

Updated Nov 6, 2026
Python

ai-dynamo / dynamo

A Datacenter Scale Distributed Inference Serving Framework

kubernetes rust routing-engine omni diffusion vllm llm-inference tensorrt-llm sglang disaggregated-serving

Updated Nov 6, 2026
Rust

Mooncake

kvcache-ai / Mooncake

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

reinforcement-learning inference rdma disaggregation llm vllm sglang kvcache trt-llm tokenspeed

Updated Nov 6, 2026
C++

vllm-project / semantic-router

An open, programmable decision layer for models and compute.

kubernetes inference pytorch transformer guardrails mixture-of-models llm vllm ai-gateway semantic-router llmrouter

Updated Nov 6, 2026
Go

mostlygeek / llama-swap

Reliable model swapping for any local AI server. Chosen by thousands of users. Batteries included. If we don't have it, you probably don't need it.

golang openai llama openai-api llamacpp vllm localllm localllama

Updated Nov 6, 2026
Go

kserve / kserve

Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes

Updated Nov 6, 2026
Go

gpustack / gpustack

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

cuda inference openai llama maas rocm ascend llm llm-serving vllm genai llm-inference qwen deepseek sglang distributed-inference high-performance-inference mindie

Updated Nov 6, 2026
Python

UltraRAG

OpenBMB / UltraRAG

A Low-Code MCP Framework for Building Complex and Innovative RAG Pipelines

flask demo ui mcp openai easy gpt embedding vlm multimodal rag sentence-transformers huggingface-transformers llm vllm qwen deepseek

Updated Nov 4, 2026
Python

skyzh / tiny-llm

learn LLM inference system on Apple Silicon for systems engineers: build a tiny vLLM + Qwen

python course serving llm large-language-model vllm qwen

Updated Nov 6, 2026
Python

Awesome-LLM-Inference

xlite-dev / Awesome-LLM-Inference

📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.🎉

mla vllm llm-inference awesome-llm flash-attention tensorrt-llm paged-attention deepseek flash-attention-3 deepseek-v3 minimax-01 deepseek-r1 flash-mla qwen3

Updated Oct 20, 2026
Python

katanaml / sparrow

Structured data extraction, instruction calling and agentic workflows with ML, LLM and Vision LLM

computer-vision machinelearning huggingface-transformers documentai llm vllm agentic-ai

Updated Nov 5, 2026
Python

lemony-ai / cascadeflow

Cascading runtime for AI agents. Optimize cost, latency, quality, and policy decisions inside the agent loop.

Updated Sep 25, 2026
Python

PaddlePaddle / FastDeploy

High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle

inference openai serving ernie llm llm-serving vllm ernie-45 ernie-45-vl

Updated Sep 24, 2026
Python

Add this topic to your repo

To associate your repository with the vllm topic, visit your repo's landing page and select "manage topics."

0