kvcache.ai
KVCache.AI is an open source orgnization between MADSys and top industry collaborators, focusing on efficient Agent/LLM serving.
Pinned Loading
Repositories
Showing 10 of 22 repositories
- ktransformers Public
A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations
- firecracker Public
- kvcache-blog Public
- accelerate Public Forked from huggingface/accelerate
🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support
- transformers Public Forked from huggingface/transformers
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
- vllm Public Forked from vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
- AFD-Ledger Public
AFD-Ledger is an analytical toolkit for provisioning and evaluating attention–FFN disaggregated LLM inference deployments.
Most used topics
Loading…