π’ Pretraining infra engineer at Baidu BMU, and owner of the optimizer infra for ERNIE.
βοΈ Focus on optimizers: bringing new optimizers (e.g. Muon-style orthogonalized updates) into distributed training, and making them fast β sharded optimizer states, fused kernels, checkpoint save / load / resharding.
π Also work on distributed performance in general: parallel strategies, communication overlap, and squeezing more throughput out of every GPU.
π Enjoy digging into where the time and memory actually go β from the math of an update rule down to the communication pattern.
π Life motto: δΊε·²θ³ζ€οΌε
ει₯ε§ β it is what it is, let's eat first.
- β‘
paddle.optimizer.Muonβ the Muon optimizer in PaddlePaddle: NewtonβSchulz orthogonalized momentum updates for 2D weight matrices, with AdamW handling the rest (source). - π§©
MuonShardingOptimizerβ sharding-stage-1 support for Muon in Paddle Fleet: whole matrices are assigned to owner ranks so orthogonalization stays local, while the remaining parameters are sharded element-wise.
- π’ Baidu, BMU Pretraining Infra β Optimizer Infra Owner for ERNIE Β· 2025 β Present
- π The Hong Kong University of Science and Technology (Guangzhou) β MPhil, Data Science and Analytics (DSA) Thrust Β· 2023 β 2025
- π China University of Petroleum, Beijing β B.Eng. in Computer Science and Technology Β· 2019 β 2023
