Madhusudan N

At the upcoming GTC, you can learn how system-level innovations are accelerating LLM inference. A good example is this new AWS blog on P-EAGLE, introducing GPU parallel speculative decoding in vLLM to generate and verify multiple tokens in parallel. Why this matters? Inference performance is increasingly a systems problem. Techniques like this improve tokens/sec per GPU, helping reduce latency and cost for production GenAI workloads. Good read for anyone working on LLM serving and inference optimization. Huang Xin Florian Saupe Jaime Campos Salas Benjamin Chislett Zhenghang (Max) Xu Faradawn Yang Omri Almog Jiahong Liu AWS blog: https://lnkd.in/g-GjMgvX⁠�

43

5 commenti

I've said this before: cut corners smartly. Progress beats perfect planning. I still believe that. But here's what that philosophy cost us. Back in 2016, for WorkIndia, we built our data architecture around star schemas. Fluid, scalable, handled complexity beautifully. When we needed to move fast, we leaned on Django's defaults. Django's many-to-many relationships, when defined without an explicit bridge table, auto-generate one. Clean on the application side. Nobody loses sleep over it. We didn't. Then came the need for a full Data Information System. Syncing everything into a data lake via CDC pipelines. That auto-generated bridge table? No timestamp column. Timestamps are everything in CDC. If a pipeline breaks and MySQL's binlog gets purged, you cannot resume. You cannot patch. You're rebuilding from scratch. A default setting from 2016 became a crisis years later. Weeks lost. Team stretched. So what's the real lesson? Cut corners. But write down every corner you cut. Because shortcuts without a paper trail don't disappear. They just wait!

40

1 commento

What if you could bring GPU-level LLM inference to CPUs? I was recently invited to Kompact.ai’s global launch event in Bengaluru. Great to see such ambitious work coming out of India’s AI ecosystem. 🚀 Kompact AI by Ziroh Labs is building scalable LLM inference on CPUs—without compromising model quality, while making high-performance inference more accessible and efficient. GPU power. CPU scale. No compromise. Thank you to the Kompact.ai team for the invitation and for putting together a great launch event. 🚀

653

6 commenti

India needs to manufacture its own RAM. Not assemble. Not import and brand. Manufacture. Why this matters: - RAM is a core dependency for every phone, laptop, server, and data center India is almost entirely import-dependent for memory chips - Any supply disruption directly impacts cost, availability, and scale - AI, cloud, and compute growth will only amplify this risk This isn’t about nationalism. It’s about control over critical digital infrastructure. You can’t build a serious digital economy on borrowed memory. P.S. Yes, it is possible to manufacture RAM in India — but not overnight. It requires: • Massive capital investment • Advanced fabrication plants (fabs) • Long-term policy consistency • Global technology partnerships It’s hard, expensive, and slow — but strategic, and eventually unavoidable. Semiconductors built the last decade. Memory will define the next one.

7

4 commenti

Keerti Melkote Anyscale is a unified AI compute platform that makes it easy to run artificial intelligence (AI) and machine learning (ML) models at scale . The company was founded in 2019 by the same founders who created Ray, a popular open source distributed computing framework . in July 2026, Nscale, a leading full-stack AI cloud platform, acquired Anyscale . details about the scale are given below:🛠️ What does a scale do?Typically, tasks such as training AI models, processing data, and using them in live applications (Inference) require massive amounts of computer power (GPUs/CPUs). AnyScale provides the following facilities: scaling: Developers can run code written on their laptop directly on thousands of cloud servers without any infrastructure changes. Automatically increases or decreases the capacity of servers based on workload, reducing cloud costs. Ray: Any Scale takes full responsibility (Fully Managed Infrastructure) for the open source 'Ray' framework, without the need for developers to manage it themselves

3