Vllm - DEV Community

Skip to content

DEV Community

👋 Sign in for the ability to sort posts by relevant, latest, or top.

xbill for Google Developer Experts

Nov 11

Six Serving Engines on One Tesla T4: Gemma 4 E2B on a Turing GPU

#gemma #vllm #googlecloud #nvidia

13 min read

Daniel Sam Pete Thiyagu

Nov 10

PagedAttention and the KV Cache, Plainly: Why Memory, Not FLOPs, Caps Your Concurrency

#vllm #llm #googlecloud #machinelearning

11 min read

Nov 11

Why vLLM and SGLang might not be the best option to run AI agents

#ai #agents #vllm #rust

4 min read

Daniel Sam Pete Thiyagu

Nov 9

vLLM 0.33, Plainly: CUDA 13, Model Runner V2, and What Breaks on Upgrade

#vllm #llm #googlecloud #machinelearning

9 min read

Nov 10

llm-compressor for vLLM: The FP8/AWQ Deployment Tradeoff

#quantization #vllm #fp8 #awq

9 min read

Nov 9

Day-0 Model Support Is a Fork: What 26 NVIDIA Dynamo Snapshot Builds Say About Pinning Your LLM Serving Stack

#ai #opensource #llm #vllm

15 min read

Nov 6

verl GRPO + vLLM Rollouts: GPU Memory Settings for Single-Node Deployment

#rlposttraining #grpo #vllm #gpumemory

12 min read

Nov 5

vLLM 0.33 Adds a Lossy Speculative Verifier: Why Single-Run Accuracy Tables Cannot Tell You It Is Safe

#ai #machinelearning #llm #vllm

15 min read

ram.mehta.1899@gmail.com

Nov 4

Managing Agent GPU Kernels: vLLM's Kimi K3 Fast-Track RFC

#vllm #gpukernels #cutedsl #codereview

1 min read

Nov 8

vLLM 0.33 Ships an ABI-Stable Wheel, but PyPI Still Pins torch==2.14.0: What You Can Actually Swap

#ai #python #llm #vllm

13 min read

Nov 3

vLLM Fast Start Explained: What Preload and Snapshots Save, What They Cannot, and the Fingerprint Gap

#ai #machinelearning #llm #vllm

14 min read

Nov 2

The Local vLLM Speculative Decoding Blueprint: How I Quadrupled DeepSeek-R1 Inference to 48 Tokens/Sec on Consumer GPUs with...

#aitoolsproductivity #aitools #deepseekr1 #vllm

14 min read

Oct 31

Your LLM Serving Engine Ships Every 13 Days: Reading vLLM's Breaking-Change Lists as Upgrade Risk Data

#ai #machinelearning #vllm #llm

13 min read

Huilin Ye

Oct 30

FP8 Cut Our TPOT by 25%. At High Concurrency, It Also Changed the Answers.

#llm #vllm #quantization #benchmarking

3 min read

Oct 30

Architecting On-Premise Mistral Large 4 for Enterprise SLAs: A Systems Blueprint

#architecture #ai #vllm #infrastructure

4 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.