Repository navigation
Tags
Tags
v0.1.1 — Q4_K routed-MoE prefill fix (issue antirez#123) Merges PR #18: per-pair Q4_K kernels for routed-MoE prefill (n_tokens > 1) on multi-GPU CUDA. Fixes BOS-loop / garbage output on mixed-quant GGUFs (e.g. DeepSeek-V4-Flash-Layers37-42Q4KExperts-...-imatrix-fixed.gguf) where Q4_K routed experts were being mis-decoded as IQ2_XXS. Verified correct on rmhubbert's 8x RTX 3090 setup.
mgpu v0.1.0 Multi-GPU performance branch: prefill tensorcore, decode split-KV/flash-decode attention, routed-MoE gate/up decode launch geometry, q8->f16 cache reserve.