I put LLM and AI-agent systems into production on serverless and rented GPUs, for a fraction of what managed APIs cost.
Upwork Top Rated · 100% Job Success
Scholar · Hugging Face · ORCID
My main open-source focus: keeping per-request state correct in vLLM-Omni’s multi-stage serving. Voice references, LoRA adapters and engine args should never leak across requests or stages. I find these silent failures by measuring, then turn them into benchmarks and CI checks.
- Found #8235: CosyVoice3 returned 11.7 to 25% wrong voices at concurrency 8 (fixed in #8224).
- Merged #8456: the benchmark now takes multiple reference voices, so mixed-voice batches can expose cross-request leaks.
- CI check: #8555 adds a nightly cross-request voice check for voice-cloning TTS, built on #8456. The missing piece is verifying who is actually speaking, which my speaker-embedding verifier (
cv3_verify) does. - Cited in the Q4 roadmap discussion as the basis for that nightly check.
- Merged #6783: engine args leaked between pipeline stages. One-line fix, 145 lines of tests.
- Osmantic/ODS: 14 PRs, including health checks for non-HTTP services
- casys-kaist/LLMServingSim: my profiling run exposed a KV-cache bug; after the fix, validation error fell from 87% to under 1%
- aimclub/OSA (2 PRs) and neka-nat/freecad-mcp
- Serverless GPU endpoints on RunPod, Modal, Vast.ai
- RAG pipelines with LLM-as-judge evals
- Multi-agent systems and hot-swappable LoRA adapters
- Rescuing deployments that almost work: OOM, driver mismatch, models that barely fit
Extends LatentMAS (Princeton, UIUC, Stanford; ICML 2026 Spotlight). Listed as community extension #5 in the official Gen-Verse repo.
Agents pass hidden states instead of text, with role LoRA adapters switched on one base model. In my GSM8K run (26 items): 83.7% fewer output tokens and 5.5x faster than the text-based pipeline, no significant accuracy difference. I also fixed a semantic router that was quietly deciding 199 of 200 routes: 51.3% to 96.2% accuracy on 450 labelled items. Limits are in the README: PEFT switching, not true S-LoRA, adapters not trained yet. Demo.
Hardened setup with 28 documented host failure modes. Not merged, kept as reference.
- gpu-broker-mcp: MCP server for agents to reserve and dispatch GPUs
- RAG + LLM-as-Judge: retrieval with citations and evals
- Wan 2.2 Multi-GPU: per-GPU VRAM capping
- FLUX.2 klein Worker: serverless, near-zero idle cost
- wacrm: WhatsApp CRM for immigration consultancies
GPU: RunPod, Modal, Vast.ai, vLLM, Docker, FSDP, 4-bit quantization
LLM: PyTorch, PEFT/LoRA, Unsloth, Axolotl, LangChain, MCP, RAG
Ops: Python, FastAPI, Bash, GitHub Actions, pytest, SQL
B.Sc. EEE, University of Rajshahi. 5 papers, 77 citations, h-index 5 on solar cells and medical imaging, in Solar Energy, Heliyon (first author), Nano Select, Sustainable Energy Tech. & Assessments, and IJACSA. Full list
I'm most interested in health, energy, education, and scientific tooling.
Send the model, latency target, and budget. You'll get a straight answer, including when you don't need me.
