Stars
Early-stage Rust drop-in alternative frontend for vLLM
Tools for Python coroutines and advanced scheduling for `asyncio`
Moving to vllm-project/vllm-jax. TPU inference for vLLM, with unified JAX and PyTorch support.
State of the art inference performance with modern accelerators on Kubernetes
A high-throughput and memory-efficient inference and serving engine for LLMs
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
Fake XRandR configurations for multi-head setups with crappy video drivers, like fakexinerama but with xrandr
Netty project - an event-driven asynchronous network application framework
The Java gRPC implementation. HTTP/2 based RPC