Repository navigation
Tags
Tags
[GrugMoE] Allow explicit piecewise graphs around ShortConv (#108) * Keep ShortConv outside explicitly requested piecewise graphs * Cover explicit graph opt-in and preserve unsupported mode checks * Keep graph-mode rejection tests independent of error wording --------- Co-authored-by: Romain Yon <1596570+yonromai@users.noreply.github.com>
[determinism] Preserve IEEE FP32 dot inputs (#117) Use explicit IEEE input precision in batch-invariant matmul kernels and cover FP32 precision and exact batch/per 8000 mutation/CUDA-graph behavior.
Fix the GrugMoE online FP8 integration test (#113) * Run the GrugMoE online FP8 test on CUDA * Preserve the GrugMoE test import order * [grug] Check online FP8 output against quantized math Build the reference from the original BF16 weights with per-expert FP8 scales and dynamic activation rounding. Compare BF16 outputs using the dtype arithmetic tolerances so expected quantization loss does not fail the kernel check. * [grug] Simplify the FP8 numerical reference Hide FP8 payload and scale bookkeeping behind a round trip to FP32. Preserve one scale per expert for weights and one scale per tensor for activations. Compare against the reference at one BF16 epsilon with no absolute-error allowance. * [grug] Match BF16 activation rounding in the FP8 reference Round the SiLU result to BF16 before multiplication, as the CUDA activation does. The FP32 intermediate can move the next FP8 activation into a different rounding bin; the seven-token H100 case exposed the mismatch. * [grug] Test FP8 loading against a direct vLLM layer Compare Grug's checkpoint mapping and FP8 setup with a vLLM expert layer initialized directly from the same BF16 weights. Both paths use the same quantizer and kernels, so require exact output agreement and remove the separate FP8 numerical reference. * [grug] Materialize the direct FP8 reference weights Use vLLM's existing materialization helper before filling the direct reference's BF16 expert weights. Online FP8 creates those parameters on the meta device until loading. --------- Co-authored-by: Romain Yon <1596570+yonromai@users.noreply.github.com>
Keep Grug reasoning and tool calls separate in streaming (#94) * Keep Grug reasoning scoped to the current assistant turn Ignore think markers from earlier turns when selecting the initial parsing phase. Hold unmarked streaming text until a closing think marker, a tool call, or stream completion so reasoning cannot leak into answer content and marker-free tool calls reach Hermes. * Name Grug streaming test results and marker IDs --------- Co-authored-by: Ben Feuer <benfeuer@Bens-MacBook-Pro.local>
Retain compiler provenance in the main-only 0.30 release Carry the qualified Torch 2.14.1 and CUDA 13.2.86 inputs through the current build, hardware qualification, and publication workflow. Update its metadata CLI regression fixture and retain the prior recipe rationale. Model source, native source, dependency constraints, and compiler component URLs remain unchanged from the qualified 47b pair.
[release] Include the GPU manifest in Iris validation jobs (#91) Co-authored-by: Romain Yon <1596570+yonromai@users.noreply.github.com>
Retain compiler provenance in the main-only 0.30 release Carry the qualified Torch 2.14.1 and CUDA 13.2.86 inputs through the current build, hardware qualification, and publication workflow. Update its metadata CLI regression fixture and retain the prior recipe rationale. Model source, native source, dependency constraints, and compiler component URLs remain unchanged from the qualified 47b pair.
[release] Fit TPU qualification to worker disk capacity Request 64 GiB of disk for the TPU qualification task. Its previous 100 GiB request is 107.37 GB, exceeding the 90.6 GB of disk reported by registered v6e-4 workers and preventing placement on that worker shape. The new request is 68.72 GB.
PreviousNext