Anna Berenberg

The 2025 OCP Global Summit kicked off yesterday. The keynotes from all the hyperscalers and GPU vendors point to the same reality - scaling AI isn’t just about re-architecting one part of the stack. It’s about rethinking power, cooling, networking, and trust as one coherent, open system. Some interesting takeaways... ✅ Inference workloads are exploding - exceeding 80% CAGR, significantly outpacing training workloads. Token generation has surged ~50× in just two years, fueled by model complexity, longer context windows, and compute intensity. Sustaining that momentum means revisiting how the data centers deliver power/dissipate heat, and how we architect every layer of the system. ✅ Traditional AC distribution is giving way to high-voltage DC (HVDC) designs such as Google and Microsoft’s Mt. Diablo architecture and Nvidia’s Kyber rack. These systems operate at ±400 V to 800 V DC. Several speakers talked about actively managing power spikes using predictive telemetry - powered by ML - to smooth load transients and stabilize the grid. In effect, the data center is evolving from a static load into an adaptive, interactive participant in the electrical ecosystem. ✅ Cooling has become equally central. Across the keynotes, the message was clear. Liquid cooling is now foundational. Nvidia, Google, Meta, and Microsoft highlighted rack-level liquid-cooling initiatives and standardized coolant-distribution interfaces. Cooling is now a co-engineered subsystem - scaling alongside power and compute lifecycles. 💡 Nvidia pushes deeper into networking. Their announcement of Spectrum-X deployments by Meta, Microsoft, and Oracle suggests Nvidia is positioning itself as a serious contender to Broadcom in scale-out Ethernet, including the Data Center Interconnect (DCI) domain, with it's spectrum-XGS. This is impressive. I always thought it is hard to win DCI market (Nvidia calls it "scale-across") without deep buffer switches for absorbing congestion. It would be interesting to learn about their distance-based load balancing for congestion control in scale-across networks. ❓ AMD joins the Ethernet for Scale-Up Networking (ESUN) work stream. This move raises intriguing questions: Is it linked to their recent collaboration agreement with OpenAI? How will they juggle both UALink and ESUN support simultaneously? 🤔 Plenty more to learn in the next few days, I guess - new technologies, new collaborations, and new directions for open infrastructure. Btw, if you’re there, don’t forget to stop by Astera Labs's booth (B33)!

253

9 Comments

I have some personal news I'd like to share. I started an AI company with my dear friend and college roommate, Wayne Pan. It took over two decades, but everything has fallen into place, personally and professionally. At Cisco I led edge compute in industrial IoT.  At NEOM I helped shape nation-building technology strategies.  At SambaNova I scaled AI infrastructure globally and built deep partnerships across the Americas and the Middle East. That journey taught me something simple: AI is no longer just an important piece of technology. It is national infrastructure. And that realization is what led me here, to building haimaker. And before I give you the pitch, I wanted to say a few things. → Get your children involved in AI. Teach them how to use the tools. Let them use it to express their creative side. Or maybe show off their tech-y side. And maybe teaching is the wrong word, because in many respects, we’re all still learning. AI is a superhuman ability. Let’s get our kids involved in using these tools. → We are living through one of the greatest technological innovations of our lifetime. It’s a blessing to be able to witness the impact that AI has had on my life and career. I’ve never felt the vibe like this before. It’s something bigger than all of us. AI brings out the little kid in me a lot of times. It’s fun to tinker and create new things with the endless tools that are available. → We are so early. So, so, so early. When I think of AI. I think of what’s possible. I find myself discovering new things every day that I can’t believe are possible because of AI. I don’t think we are anywhere near our limit of discoveries. Don’t ever feel that you’re too late to get involved with AI as a career path. → AI isn’t a feature. It’s critical national infrastructure. Just like airports, hospitals, and telco networks, AI is essential to the growth and security of every nation. And it should be treated as such. Back to haimaker. haimaker is a Sovereign AI Compute Orchestration Platform. haimaker enables nations and telcos to launch sovereign AI Compute as a Service, while giving enterprises and developers the ability to run workloads in compliant regions. This is just the beginning. We have a lot of great news coming very soon and we can’t wait to share what we’ve been working on.

465

114 Comments

What happens when Amazon (Zoox) builds a Robotaxi?  They skip the hype and just ship it. While we continue to wait for Tesla’s much-anticipated Robotaxi, Amazon-backed Zoox quietly launched its autonomous vehicle service to the public in Las Vegas this week. Free and very limited rides for now, but here are 2 reasons why I like Zoox: 1. Designed from scratch Zoox didn’t retrofit an existing car. They reimagined the vehicle altogether. It has no steering wheel. No pedals. No front or back. Just a fully symmetrical electric vehicle, purpose-built for autonomy. 2. Passenger-first interior With face-to-face seating and no driver distractions, the design is built entirely around the rider experience. It’s not trying to replicate driving. It’s trying to make us forget about it. Tesla may have dominated headlines, but Zoox is executing quietly and methodically. And in the world of autonomous vehicles, that might just be the better strategy. It's good to see an innovative company that doesn't shout. It ships. Video Credits: Zoox Channel on Youtube #AutonomousVehicles #AI #Zoox #Amazon #Robotaxi #FutureOfMobility #ProductDesign #Innovation #TechLeadership

678

23 Comments

I've found that most devs have rarely looked at GPU compute dies. There's a tremendous amount of innovation that happens at the silicon level that can help in understanding optimization paths when considering different platforms. This blog, authored by Kyle Aubrey and Nick Stam, takes a deep dive into the Blackwell Ultra GPU architecture including the high-speed interconnects, SMs, and other critical components responsible for driving critical workload performance at scale. Blog link: https://lnkd.in/eUXNFb_m #nvidia #gpus #blackwell #hardware #ai

263

7 Comments

Excited to announce Ray on Anyscale is now available on CoreWeave to further deliver on our multi-cloud support. This gives users access to CoreWeave’s purpose-built AI infrastructure, optimized for distributed AI at scale with low latency, fast startup times, and high performance GPUs. It's a powerful combination for teams building and scaling production AI workloads. Read more from CoreWeave: https://lnkd.in/dN9AjA3s

83

1 Comment