Hand-rolled BGE-M3 inference engine: multilingual embeddings at 1800 req/s on a $400 GPU. Zero-dependency C + CUDA, beats PyTorch sdpa at every length.
multilingual cuda grpc embeddings avx2 from-scratch inference-engine int8-quantization flash-attention bge-m3
-
Updated
Jul 30, 2026 - C