I am Zheng Cai, nickname zigzagcai, an AI Infra engineer, working on (M)LLM pre/post-train/serving and Automatic kernel/framework optimization.
I love the general idea of open-source, learn from the open-source community and try my best to contribute back.
Selected code/knowledge I have ever shared:
- Developed sequence parallelism for DSA(DeepSeek Sparse Attention)-like models in vLLM, which delivered 25.42% TTFT redution and 32.59% throughput improvement for GLM-5.3-Flash under 1M long context length. vllm-project/vllm#54951
- Developed CUDA kernel for variable length Mamba training and delivered 2~4x end-to-end training speedup. state-spaces/mamba#244
- Figure out and fix deadlock for Colossal.AI training framework under very large scale (>2k gpus) training scenarios. hpcaitech/ColossalAI#5625
- Implement FSDP+EP+Ulysses_SP for 671B full-size DeepSeek V3 model. https://github.com/zigzagcai/DeepSeekV3
- Figure out and fix accuracy issue with NCCL all-reduce under real world LLM training. https://zhuanlan.zhihu.com/p/701623664
- My best practice on how to deal with PyTorch dataloder memory leakage caused by Python copy-on-write mechanism. pytorch/pytorch#13246 (comment)
- Support NoGIL experimental feature for NumPy-1.18.5. https://github.com/colesbury/numpy/commit/0d6ef2770268711ee6417792ba0da35fcb264bf5