NeurIPS 2026 | Paper
Official implementation of MoEZip, a routing-aware KV cache compression method for sparse Mixture-of-Experts (MoE) language models.
MoEZip addresses a key limitation of existing KV cache compression methods: evicting KV entries can alter expert routing decisions and degrade generation quality in MoE models.
To address this, MoEZip estimates routing sensitivity using Fisher information and combines it with attention-based importance scores to guide KV cache eviction.
At 10% KV cache retention, MoEZip outperforms the state-of-the-art baseline by:
- LongBench: +24 points (2.3× higher score)
- RULER: +20 points (4.9× higher score)
MoEZip builds on NVIDIA KVPress.
Install KVPress:
pip install kvpressInstall any additional dependencies required by this repository.
Run the provided example:
bash run_example.shIf you find MoEZip useful in your research, please cite our paper:
@inproceedings{kim2026moezip,
title = {MoEZip: Routing-Aware KV Cache Compression for Sparse Mixture-of-Experts LLMs},
author = {Kim, Minsang and Baek, Seungjun},
booktitle = {Advances in Neural Information Processing Systems},
year = {2026},
url = {https://openreview.net/pdf?id=R8hTibgKBX}
}This implementation builds on NVIDIA KVPress.