About
Over 20 years of experience as an engineering executive, builder, and technology leader…
Activity
8K followers
-
Eyal Carmi reposted thisEyal Carmi reposted thisPeople started by asking AI questions. Now they're delegating tasks, building solutions, and automating work that runs in the background. We see those patterns in trillions of productivity signals across Microsoft Copilot. Office set the shape of productivity for the PC era. Copilot is doing that for the AI era, expanding both what people can do and who gets to do it. Today we’re introducing the new Copilot, an evolution of the Copilot app that connects the tools people rely on with the next generation of capabilities they’ll need to build, customize, and scale AI across work. Copilot has three new capabilities: Office in Copilot brings the full value of Word, Excel, and PowerPoint right where the conversation’s happening. Code in Copilot empowers every knowledge worker to turn ideas into solutions using natural language. And the new Autopilots tab in Copilot where you can create a personal autopilot that is persistent and proactive. Learn more: https://msft.it/6044a956o
-
Eyal Carmi reposted thisEyal Carmi reposted thisThe New Bottleneck The bottleneck is no longer intelligence. Over the last six months, intelligence has accelerated dramatically. But smarter models are not enough for everyone to benefit from novel intelligence. New form factors are required. GPT-3 needed Chat. Opus 4.5 needed Code and Cowork. Today, with Astra and Fable, we’re in a familiar position. Intelligence is advancing faster than it can reach people in useful forms. The model overhang is back. The new bottleneck is diffusion. Increased intelligence needs to translate to human progress. Microsoft’s mission is to empower every person and every organization to achieve more. If we fail to create new ways to utilize increased intelligence, this mission is at risk. The New Copilot Our next step is the new Copilot. Home, Code, and Autopilot. Home is your new starting point, unifying Chat and Cowork into a single experience. Ask a quick question or delegate a whole project. No need to decide where to start. Home also features Today, our new proactive view that spans Outlook, Teams, meetings, and tasks. Today doesn’t just flag things you missed; it moves work forward. The draft is already written, and the schedule change is already proposed. We’re also bringing Office into Copilot. For the first time, you don’t have to choose between AI editing and human collaboration. We’ve brought the full power of Word, Excel, and PowerPoint into Copilot, so you can create, edit, and collaborate with colleagues and Copilot at the same time. Next, Code is coming to Copilot. Models can write code but environments, hosting, and sharing get in the way. With Code, just describe an app, tracker or workflow in plain English. Code runs sandboxed on your machine or safely in your tenant in the cloud. Now, hosting and sharing an app is as easy as a Word document. Finally, Autopilot. Your digital teammate. Give it a name, an appearance, a role, and a goal. Your Autopilot has it’s own identity, memory, computer, and workspace. Autopilot is enterprise grade, with real permissions, audit, and governance, all from Agent365. It can monitor work, coordinate with your team, and keep projects running over weeks -- just tag it in a document, forward it an email, or message it on Teams! Home. Code. Autopilot. Intelligence is here. Put it to work.
-
Eyal Carmi reposted thisEyal Carmi reposted thisWe’re building Copilot as a new OS for work that spans every model, every form factor, and every task. Today, we’re announcing our biggest update to Copilot to date, bringing four things together: • Autopilot: proactive and long-running agent built for the enterprise • Code: build apps with Copilot, hosted inside your company’s tenant • Home: Chat + Cowork together • Office: now fully embedded in Copilot (and Copilot embedded in Office, of course!) Plus, you can invoke Copilot in Teams, and we’re introducing Today, a proactive experience that surfaces the most important information from across M365 without needing to ask for it. The way we work is changing and so are our workflows. This update brings AI into that flow, from answering a question, to building an app, to getting work done on your behalf. Read more about what we are announcing: https://lnkd.in/gRuNDGb7
-
Eyal Carmi reposted thisEyal Carmi reposted thisAI is helping mathematicians make real progress on Navier-Stokes. A problem that has resisted humans for generations. Super cool. But that’s not how a billion people are going to decide if AI matters. They’re going to decide it matters when it gives them time back. When Sunday night isn’t spent doing expense reports. When a five-step approval becomes one step. When a doctor catches a disease earlier. When a kid anywhere in the world gets access to a great teacher. When healthcare gets better and more affordable. Kevin Scott put it this way: “A.I. is one of the most powerful things humans have ever invented for improving the quality of life of everyone, but it will take time. It should take time. We've always tackled super-challenging problems through technology. And so, we can either tell ourselves a good story about the future or a bad story about the future, and, whichever one we choose, that's probably the one that'll come true.” The question is what we choose to do with it. I think the best companies start with a vision: What is the world we want this technology to help create? Then work backwards... Build toward that vision AND put the safeguards in place to do it responsibly. The answer isn’t to start from fear, or to ignore the risks. It’s to be clear about the future we want, and build toward it with safeguards from the start. This is a time for real leadership. Technical brilliance matters enormously. But so do judgment, experience, empathy, and an understanding of how businesses and institutions actually work, and how change actually lands on people. The technology isn’t going to decide what kind of future we build with it. We will.
-
Eyal Carmi reposted thisEyal Carmi reposted thisכולם מדברים על SDLC. אבל כשסקילים הופכים לחלק מהדרך שבה אנחנו מפתחים, הם לא רק תומכים ב־SDLC. גם לסקיל עצמו יש lifecycle. הוא מתחיל ב־session אמיתי. לא ב״בואו נכתוב סקיל״, אלא בעבודה חוזרת שבה מזהים דפוס: אותה שאלה, אותו context שצריך לאסוף, אותן טעויות, אותו תהליך שאפשר להפוך ליותר עקבי. רק אז באמת יש בסיס לבנות סקיל. ואז מגיע שלב ה־evaluate. וכאן קל לעצור מוקדם מדי. ל־evaluation של סקיל יש לפחות שני חלקים: האם הסקיל יעיל? האם הוא באמת עושה את העבודה שלשמה נבנה, ומשפר את התוצר? ולא פחות חשוב - האם הוא נותן תוצאה טובה יותר מ־prompt פשוט, ללא הסקיל? האם הסקיל מופעל כשצריך? בסקילים שאמורים להיות מוטרגים אוטומטית, לא מספיק שהסקיל יהיה מצוין כשהוא כבר רץ. צריך לבדוק שהוא נכנס לפעולה במקרים הנכונים — ולא מפספס אותם, או מופעל במקרים שבהם הוא בכלל לא רלוונטי. סקיל שעבד יפה בכמה הרצות אינו בהכרח סקיל שאפשר לסמוך עליו. צריך לבדוק אותו מול baseline הוגן, על תרחישים שונים, ולוודא שמודדים את הדבר הנכון — גם איכות ה־output והערך שנוצר, וגם איכות ה־triggering. ואז משתמשים בו. אבל גם זה לא סוף הסיפור. המודלים משתנים. ה־harness משתנה. הכלים הזמינים לסוכן משתנים. לפעמים גם הבעיה העסקית או הקוד שעליו הסקיל עובד משתנים. כל אחד מאלה יכול להפוך סקיל שהיה מצוין אתמול לפחות רלוונטי, או אפילו מזיק, היום. לכן השימוש בסקיל צריך להחזיר אותנו בחזרה להערכה. וכמו ב־SDLC, זה לא תהליך בירוקרטי. זו הדרך לבנות משהו שאפשר באמת לסמוך עליו. כי סקיל הוא לא prompt ששמור בקובץ. הוא מוצר קטן: עם משתמשים, התנהגות, איכות, תחזוקה ושינויים בסביבה שבה הוא פועל. אולי הגיע הזמן שנתחיל לדבר לא רק על איך כותבים skills, אלא גם על איך מנהלים אותם לאורך החיים שלהם. InsideOut AI #AI #AIAgents #Skills #LLMEvals #SoftwareEngineering #ProductDevelopment
-
Eyal Carmi reposted thisEyal Carmi reposted thisBuild apps in Cowork, not just HTML - but managed, connected, and secure apps. Today we enabled the new app skill in Cowork for Frontier customers to build apps directly in the flow of work in Copilot Cowork. We also announced the same skill being available being available in Copilot Studio next week in the full Copilot Studio IDE. I'm beyond proud of what the team has been able to acheive to bring not just another vibe coding tool, but to bring one that's connected across multiple surfaces in Microsoft to meet you where you are and focused on governance from day one. Some other fun things to try: 🔐 Apps built in Cowork are governed and visible in the Microsoft admin center, giving you control over the apps themselves, but also what connectors can be used, and how they can be shared across an organization. 🤖 You can use the skill with the latest models across families, you're not bound to just one, below is a selection of apps I've built with Astra, Opus, SOL, and Fable. 🧠 You can create custom skills in Cowork, I have a skill that generates docs, a training presentation, and an accessibility report for my app in the format that works for me and my team. Right in the same conversation. 🕸️ Apps are connected, with support for over 1500+ connectors to bring real data into your apps. You can also build sites and tables for SharePoint (available now) and for Dataverse in the coming weeks. ℹ️ You get to build in the context of your work, you can reference meetings, people, docs, presentations - all the amazing power of Cowork, now with apps. If you're keen to learn more: - You can read about the Cowork updates here : https://lnkd.in/gwuhnuTF - You can see Ryan's announcement here : https://lnkd.in/p/gFUcziu6 - You can read about the updates coming to Copilot Studio here : https://lnkd.in/gKE2MdXp Below are some of the apps we've built for internal processes, scheduling, career management, AI adoption maturity, and even our team scrum app. We can't wait to see what our Frontier customers build, and super excited to bring this to Copilot Studio.
-
Eyal Carmi reposted thisEyal Carmi reposted thisGrateful to 🦄 Peter Steinberger, creator of OpenClaw, for a fascinating conversation with our org today on the future of engineering. Some interesting insights from the discussion on multi-agent systems: 👀 Ask agents to think like users. Don’t just ask them to complete a task—ask them to experience it, think of scenarios/edge cases, challenge their own output, and “read it again.” 🎥 Make agents show their work. Ask an agent to record a video of what it’s doing. Watching the agent video playback of using the product can surface errors you might otherwise miss. 💬 Even simple prompts can matter. Words like " I believe in you.” could do wonders. Apparently, implicit screaming at LLMs through CAPS LOCK is undergoing evolution too. 😄 🔐 Permissions + context become a whole new problem. As agents start talking to other agents—and operating across multiple context windows—permission management and shared context get messy fast. Peter shared his open-source project https://lnkd.in/gb8YGTJw 🛡🛡️ Guardrails need to become comprehensive. Web access, dependencies, which agent to trust more and testing coverage all become part of the engineering surface. The bigger takeaway for me: engineering with agents isn't just about making agents more capable. It's about learning how to collaborate with, constrain, verify, and trust them. A new kind of engineering craft is emerging. 🚀 #OpenAI #FutureofEngineering #AgenticEngineering
-
Eyal Carmi reposted thisEyal Carmi reposted thisI thought code would go first with AI - and it is. Now I think AI, and code, are coming for everything. There are dangers when someone else controls the interface between you and the world.
-
Eyal Carmi reposted thisEyal Carmi reposted thisGitHub Copilot is live on Microsoft Teams today! This integration makes GitHub Copilot a multiplayer agent running where you and your colleagues collaborate. Cloud agent sessions can now be started from Teams in group chats and channels. This is different from a bot that files issues for you, since in Teams the session is shared. Anyone in the thread can add context or redirect the agent mid-task. Write access to the repo is what gates the ability to trigger changes, so read-only participants can still contribute to planning without being able to modify code. The flow is: - Install the GitHub app for Teams and connect your GitHub account. - For public channels, configure a default repository. DMs don't use one. - Mention @GitHub with your task. GitHub Copilot runs asynchronously in a cloud sandbox and creates a code channel for the session. Agent-generated artifacts are portable back to your terminal, the Copilot app, or your IDE. Give it a try! https://lnkd.in/gRxSU-Fr #MicrosoftTeams #GitHubCopilot #AIAgents #DeveloperTools #DevEx
-
Eyal Carmi reacted on thisEyal Carmi reacted on thisHands up for Made On YouTube! 🙌 So great to take the stage at YouTube’s annual product launch event, as a first-time speaker and guest. To stay ahead of how the world watches, we’re evolving discovery and helping creators find new ways to tell their stories, across every screen and every format. I loved sharing more about the products our YouTube teams work on every day, from Ask YouTube to Likeness ID, and the impact they have on millions of people around the world. Shout out to everyone who brought the show to life in NYC! Already excited for next year. https://lnkd.in/grw9W4hj
-
Eyal Carmi reacted on thisEyal Carmi reacted on thisThe evolution from "Copilot as chat" to "Copilot as operating system for your company" runs deeper than the ability to build and automate. It means expanding Copilot into a platform that runs, secures, and governs the things you build. That's why in addition to adding Code and Autopilots today, we're also adding a Managed Runtime to Copilot. Now you can build anywhere -- in the flow of work, in the Code tab, in Copilot Studio, or even with tools and coding agents beyond Microsoft -- and deploy all those solutions to the Copilot Managed Runtime. The integration helps agents make the right choices up front, steering them to build assets that respect enterprise data policies, always authenticate with Entra, and reject sketchy code. As a result, everything you deploy to the Managed Runtime "just works" inside your company. Users can share apps like a documents; a developer can pick up in Code or Copilot Studio where a maker left off in Cowork; admins can see and manage the whole estate in one place. Both "no code" and "no cliffs" in a single platform. Today is a big day for builders!
-
Eyal Carmi liked thisEyal Carmi liked thisDefensive patent: Approaches for Incorporating a Variety of Metadata in Transformer Operation - with Samer Hassan and Afroz Mohiuddin A plain transformer model typically leverages only one piece of metadata - position encoding - directly in the transformer model. The use of transformers typically involves expensive and complex external scaffolding before or after output generation to avoid issues such as hallucination, irrelevance, etc. This disclosure describes techniques to incorporate a variety of metadata types into the native architecture of transformer models. The additional signals can help avoid hallucinations, improve relevance, and minimize the need of expensive external scaffolding. Generalizing transformer operation to incorporate a diversity of metadata can be achieved in various ways such as adding a metadata embedding layer, conditioning self-attention on the metadata, conditioning with gated self-attention, employing a different encoder-decoder architecture, etc. Different types of metadata can help in different ways to improve the quality of the output generated by the transformer and reduce hallucinations. The techniques described in this disclosure can also support multimodal data, such as images, audio, video, or text, etc., with the metadata used representing the specific mode. https://lnkd.in/eway2F46
-
Eyal Carmi liked thisEyal Carmi liked thisPeople started by asking AI questions. Now they're delegating tasks, building solutions, and automating work that runs in the background. We see those patterns in trillions of productivity signals across Microsoft Copilot. Office set the shape of productivity for the PC era. Copilot is doing that for the AI era, expanding both what people can do and who gets to do it. Today we’re introducing the new Copilot, an evolution of the Copilot app that connects the tools people rely on with the next generation of capabilities they’ll need to build, customize, and scale AI across work. Copilot has three new capabilities: Office in Copilot brings the full value of Word, Excel, and PowerPoint right where the conversation’s happening. Code in Copilot empowers every knowledge worker to turn ideas into solutions using natural language. And the new Autopilots tab in Copilot where you can create a personal autopilot that is persistent and proactive. Learn more: https://msft.it/6044a956o
-
Eyal Carmi liked thisEyal Carmi liked thisThe New Bottleneck The bottleneck is no longer intelligence. Over the last six months, intelligence has accelerated dramatically. But smarter models are not enough for everyone to benefit from novel intelligence. New form factors are required. GPT-3 needed Chat. Opus 4.5 needed Code and Cowork. Today, with Astra and Fable, we’re in a familiar position. Intelligence is advancing faster than it can reach people in useful forms. The model overhang is back. The new bottleneck is diffusion. Increased intelligence needs to translate to human progress. Microsoft’s mission is to empower every person and every organization to achieve more. If we fail to create new ways to utilize increased intelligence, this mission is at risk. The New Copilot Our next step is the new Copilot. Home, Code, and Autopilot. Home is your new starting point, unifying Chat and Cowork into a single experience. Ask a quick question or delegate a whole project. No need to decide where to start. Home also features Today, our new proactive view that spans Outlook, Teams, meetings, and tasks. Today doesn’t just flag things you missed; it moves work forward. The draft is already written, and the schedule change is already proposed. We’re also bringing Office into Copilot. For the first time, you don’t have to choose between AI editing and human collaboration. We’ve brought the full power of Word, Excel, and PowerPoint into Copilot, so you can create, edit, and collaborate with colleagues and Copilot at the same time. Next, Code is coming to Copilot. Models can write code but environments, hosting, and sharing get in the way. With Code, just describe an app, tracker or workflow in plain English. Code runs sandboxed on your machine or safely in your tenant in the cloud. Now, hosting and sharing an app is as easy as a Word document. Finally, Autopilot. Your digital teammate. Give it a name, an appearance, a role, and a goal. Your Autopilot has it’s own identity, memory, computer, and workspace. Autopilot is enterprise grade, with real permissions, audit, and governance, all from Agent365. It can monitor work, coordinate with your team, and keep projects running over weeks -- just tag it in a document, forward it an email, or message it on Teams! Home. Code. Autopilot. Intelligence is here. Put it to work.
-
Eyal Carmi liked thisEyal Carmi liked thisWe're thrilled to announce Business Applications in Work IQ 🚀 Every business has its own language, business data, and processes. Business Applications in Work IQ brings that context from Dynamics 365 and Power Apps into Copilot and agents through semantic models, business skills, and governed actions. The result: Copilot and agents can better understand the business and help people move from questions to decisions to action. A proud moment for our team ❤️ and an incredible collaboration across Microsoft. Public Preview starts Sep 30, rolling out all the way through October. Excited to see customers put this to work! Learn more here: 📰 Blog post: https://lnkd.in/gdGuH89Q 📖 Documentation: https://lnkd.in/gDH38Wjw 📺 Watch Reenvisioning Sales Workflows with Business Apps in Work IQ: https://lnkd.in/gYvkCwWW #BizAppsInWorkIQ #WorkIQ #MicrosoftCopilot #MicrosoftDataverse #Dynamics365 #PowerApps #PowerPlatform #PowerPlatformCommunity #M365Community 🙏 James Oleinik , Sharad Bajaj , Nitish Jain , Viraj Malkar , Jeff Anderson, Cooper Wynn , Siddharth Singh , Kent Weare, Nathan Helgren, Jason Huang, Prithvi Khosla, Divya Rajaram, Karissa Rohde, Mankirat Kaur Bajwa, Jennifer Bost, Reza Assadi, Deepak Palled, Sparsha Sundaresh, Adrian Orth, Jeremy Jensen, Mike Stall, Jared Hall, Abhishek Ghosh, Evan Lew, Philip Li, Brian Day, Tolga Kilicli, Stephanie Jorgenson, MBA
-
Eyal Carmi liked thisEyal Carmi liked thisOpus 5.5 designing LEGO 👀 Victor Mustar asked Opus 5.5 to design a Microduck he could build with real LEGO pieces. The model: > designed it life-size using 1113 real LEGO parts > verified: 3,204 connections, 0 collisions, every step buildable, centre of mass inside the feet 🤯 > made a 141-page LEGO-style booklet (237 steps) > priced every piece in the browser and prepared the orders on BrickLink Next let's try this with an open model! 🔥
Experience
Education
Publications
-
Is Oprah Contagious? Identifying Demand Spillovers in Online Networks
ICIS 2009 Honorable Mention; NET Institute Working Paper
We study the spread of exogenous demand shocks generated by book reviews featured on the Oprah Winfrey TV show and published in the New York Times through the online co-purchase recommendation network on Amazon.com. We leverage the co-purchase recommendation network on Amazon.com to determine how such exogenous events might affect the demand for books that were not explicitly mentioned in a review but are located “close” to reviewed books in a product network. Our results show that the demand…
We study the spread of exogenous demand shocks generated by book reviews featured on the Oprah Winfrey TV show and published in the New York Times through the online co-purchase recommendation network on Amazon.com. We leverage the co-purchase recommendation network on Amazon.com to determine how such exogenous events might affect the demand for books that were not explicitly mentioned in a review but are located “close” to reviewed books in a product network. Our results show that the demand shock diffuses to books that are up to three links away from the reviewed book. However, the magnitude of diffusion varies widely across books at the same network distance from reviewed products. We also describe how product characteristics, assortative mixing and network properties can explain this variation in the depth of contagion. We discuss the significance of these results and their implications for the design of networks of products as well as for optimizing digital marketing spillovers.
Other authorsSee publication -
Resolution enhancement in MRI
Magnetic resonance imaging
We consider the problem of Super Resolution Reconstruction (SRR) in MRI. Sub-pixel shifted MR Images were taken in several fields of view to reconstruct a high-resolution image. A novel algorithm is presented. The algorithm can be applied locally and guarantees perfect reconstruction in the absence of noise. Results that demonstrate resolution improvement are given for phantom studies (Mathematical Model) as well as for MRI studies of a phantom carried out with a GE clinical scanner. The method…
We consider the problem of Super Resolution Reconstruction (SRR) in MRI. Sub-pixel shifted MR Images were taken in several fields of view to reconstruct a high-resolution image. A novel algorithm is presented. The algorithm can be applied locally and guarantees perfect reconstruction in the absence of noise. Results that demonstrate resolution improvement are given for phantom studies (Mathematical Model) as well as for MRI studies of a phantom carried out with a GE clinical scanner. The method raises questions that are discussed in the last section of the paper. Open questions should be answered in order to apply this method for clinical purposes.
Other authors -
Assessing Value in an Online Network of Products
ICIS 2012 Best Paper Award; To appear in Journal of Marketing, May 2013
Traditionally, the value of a product is assessed according to its direct revenues. However, products do not exist in isolation but rather influence one another's sales. Such influence is especially evident in eCommerce environments, where products are presented as a large-scale product network. We present the first attempt to use a systematic approach to estimate products' true value to a firm in such settings. We separate a product’s value into its own intrinsic value, the value it receives…
Traditionally, the value of a product is assessed according to its direct revenues. However, products do not exist in isolation but rather influence one another's sales. Such influence is especially evident in eCommerce environments, where products are presented as a large-scale product network. We present the first attempt to use a systematic approach to estimate products' true value to a firm in such settings. We separate a product’s value into its own intrinsic value, the value it receives from the network, and the value it contributes to the network. Using data from the network of books on Amazon, we examine the relationship between revenue and the sources of value. We show that the value of low-sellers is underestimated when focusing on direct revenue, while the value of bestsellers is overestimated. We explore the sources of this discrepancy and discuss the implications for managing products in an environment of product networks.
Other authorsSee publication
Patents
-
Knowledge Panel
Issued US US20170011102
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for providing knowledge panels. In one aspect, a method includes determining an entity that is referenced by a received search query. Content items relevant to the entity are identified. A set of knowledge modules is selected for presentation in a knowledge panel for the entity. The knowledge panel can be a portion of a search results page in which knowledge modules are presented. Data that causes…
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for providing knowledge panels. In one aspect, a method includes determining an entity that is referenced by a received search query. Content items relevant to the entity are identified. A set of knowledge modules is selected for presentation in a knowledge panel for the entity. The knowledge panel can be a portion of a search results page in which knowledge modules are presented. Data that causes the knowledge modules to be presented in the knowledge panel is provided to a user device.
Other inventorsSee patent -
Reconciling data
Issued USPTO
See patentMethods, systems, and apparatus, including computer program products, are described for reconciling data. In one implementation, a method includes generating co-occurrence scores indicating whether data in entries in a first source of data co-occur within documents in a plurality of documents with data in entries in a second source of data. The co-occurrence scores for a given entry in the first source of data are used to identify a plurality of candidate matching entries in the second source…
Methods, systems, and apparatus, including computer program products, are described for reconciling data. In one implementation, a method includes generating co-occurrence scores indicating whether data in entries in a first source of data co-occur within documents in a plurality of documents with data in entries in a second source of data. The co-occurrence scores for a given entry in the first source of data are used to identify a plurality of candidate matching entries in the second source of data for the given entry. Data in fields in the given entry are compared to that of one or more of the candidate matching entries to produce field similarity scores. The field similarity scores and the co-occurrence scores are used to determine a match for the given entry among the plurality of candidate matching entries.
-
Configuration-management user interface employing searchable tags
Issued US 9,026,949
Languages
-
English
Full professional proficiency
-
Hebrew
Native or bilingual proficiency
Recommendations received
9 people have recommended Eyal
Join now to viewOther similar profiles
Explore more posts
-
Chandra Mohan Singh Negi
🎇🎇Demystifying Transformer Inference: The Mathematics Behind LLM Performance 🚀 Ever wondered why your LLM responses vary in speed? The answer lies in the delicate balance between memory bandwidth and compute capacity. Carol Chen's deep dive into transformer inference arithmetic reveals critical insights for anyone deploying foundation models in production Key Takeaways: 🔹 The KV Cache Tradeoff: Caching key-value vectors saves massive computation (avoiding quadratic time complexity) but consumes significant memory. For a 52B parameter model on A100 GPUs, you're looking at ~2MB per token - that's 8,000 tokens max on a 3-GPU setup. 🔹 Memory vs. Compute Bound: The magic number? 208 tokens for A100s. Below this, you're memory bandwidth bound (1.5TB/s); above it, you're flops bound (312 TFLOPS). This fundamentally changes optimization strategy. 🔹 Batch Size Economics: Small batches (<208) should actually skip KV cache and recompute - you're paying memory cost anyway. Large batches need tensor parallelism across multiple GPUs to stay compute-bound. 🔹 Model Parallelism Reality: Tensor parallel beats pipeline parallel for inference. But communications cost matters: 4 synchronization points per layer at ~300GB/s NVLink bandwidth. For models with d_model > 1024, you stay safely compute-bound. For Physical AI & Robotics practitioners: These principles directly apply when deploying VLA models at the edge. Understanding the 2P (parameter count × 2) flop calculation and memory-to-compute ratios is essential for real-time robotic control systems. The article includes practical formulas for latency calculation and validates them against real Faster Transformer benchmarks - achieving 76-87% accuracy in predicting actual inference times. Full technical deep-dive: https://lnkd.in/g4ykzZPc #PhysicalAI #LLMOptimization #MachineLearning #RoboticsAI #DeepLearning #AIInfrastructure #TransformerModels
-
Gabriele Garzoglio
By benchmarking real-world image transformation tasks, I found that Google Axion’s generous cache, offering larger L2 cache per core provides a clear architectural advantage. Key takeaways: * Performance: C4A (Axion CPU) was roughly 1.8x faster than a comparable Intel Emerald Rapids CPU for high-resolution image resizing. * Cache Efficiency: Axion maintained a 99% L2 cache hit ratio, significantly reducing CPU stalls compared to x86 counterparts. Read the full technical deep dive here: https://lnkd.in/g4S9c_3m
-
calin turcanu
Innovators in science and engineering face a common barrier: the search space for solving complex problems — like designing a new chip or discovering a drug molecule — is often too vast for standard brute-force methods to explore effectively. To help you overcome this challenge, we are releasing AlphaEvolve, a Gemini-powered coding agent for designing advanced algorithms, to Google Cloud, in private preview. https://lnkd.in/egQK-tr5
-
MIS Digital Factory
This post was written with Dominic Catalano from Anyscale. Organizations building and deploying large-scale AI models often face critical infrastructure challenges that can directly impact their bottom line: unstable training clusters that fail mid-job, inefficient resource utilization driving up costs, and complex distributed computing frameworks requiring specialized expertise. These factors can lead to unused GPU hours, delayed projects, and frustrated data science teams. This post demonstrates how you can address these challenges by providing a resilient, efficient infrastructure for distributed AI workloads. Amazon SageMaker HyperPod is a purpose-built persistent generative AI infrastructure optimized for machine learning
-
Ali Arsanjani, PhD
🚀 Enterprise AI Just Got a Whole Lot Smarter: Enhanced Tool Governance in Vertex AI Agent Builder 💡 Today’s announcement from Google Cloud represents a meaningful step forward for organizations building production-grade AI agents. With the integration of Cloud API Registry directly into Vertex AI Agent Builder, enterprises now have enterprise-grade tool governance and lifecycle support for agentic systems — a critical capability for scaling responsible and secure AI. 🔐 Governance Meets Scale One of the biggest challenges in enterprise AI adoption has been balancing developer agility with platform control. With enhanced tool governance in Vertex AI: ✅ Admins can curate a centralized catalog of approved tools that developers can safely leverage across projects. ✅ A new private registry experience in the Agent Builder console makes it easy to view, manage, and govern tool availability. ✅ Tools backed by the Cloud API Registry — including pre-built connectors to services like BigQuery and Google Maps — reduce duplicated development effort and increase consistency. ✅ Support for custom MCP servers via Apigee unlocks legacy or internally-managed APIs without sacrificing governance controls. ✅ Developers benefit from a new ApiRegistry object in the Agent Development Kit (ADK) to seamlessly consume governed tools. This is a real win for platform teams — it means secure, compliant, and discoverable tool integration becomes part of the standard AI development workflow. ⸻ 🛠️ Beyond Governance: Faster Development & Richer Agent Experiences But the update doesn’t stop at governance. Recent enhancements across the Vertex AI Agent Builder ecosystem further enable developers to materialize intelligent agents faster and more robustly: 🔹 ADK Support for Gemini 3 Pro & Flash – Bring Google’s most capable models into your agent logic. 🔹 ADK in TypeScript – Broader language choice improves accessibility for engineering teams. 🔹 Advanced State Management – Resiliency features like native state recovery, rewind, and human-in-the-loop control improve consistency and iteration speed. 🔹 Interactions API & A2UI – Simplify building multimodal experiences and shared UI workflows without security trade-offs. 📈 What This Means for the Future of AI Agents 🔍 Standardize what tools agents can call. 🔄 Eliminate redundant API tooling efforts across teams. 🔐 Anchor agent behavior in organizational security and governance practices. 📈 Scale intelligent agent deployment without sacrificing guardrails. ⸻ Great job Michael Vakoc and team ! 🔗 Read the full announcement here: https://lnkd.in/gvSmt63e #AI #MachineLearning #VertexAI #GoogleCloud #AIGovernance #EnterpriseAI #AgentBuilder #GenAI
1 Comment -
Ashok Kumar Singh
Your agent observability strategy is an existential threat to your infrastructure budget. 🛑📉 Standard application performance monitoring (APM) and telemetry logic completely break when applied to autonomous AI. A single complex agent run easily emits over 500+ spans. Scale that to a modest 10K daily runs, and your pipeline forces your data backend to swallow 5 million spans per day, per agent. If your engineering team is blindly dumping un-sampled, high-cardinality trace logs into a generic observability platform, your storage and ingestion bills will grow exponentially faster than your enterprise value. True architectural authority means executing intelligent sampling at the orchestration layer, filtering out execution noise before it leaves your perimeter. Sampling at the orchestration layer is the exact line between being observable and going bankrupt. Stop bleeding capital on raw telemetry noise. Architect elite, cost-optimized AI infrastructure today at www.mcal.in. 👇 #LLMOps #AIEngineering #FinOps
-
Hojjat Jafarpour
Excited to share that DeltaStream will be the sponsor of an upcoming hands-on workshop on building and operating AI agents with Amazon Web Services (AWS) Bedrock AgentCore in Palo Alto next Thursday. In this interactive session, attendees will build AI agents using AWS Bedrock AgentCore and see how DeltaStream delivers fresh, real-time context that agents need to make accurate decisions. For many enterprise AI use cases, agents are only as good as the fresh data and context they can access, and that’s exactly where DeltaStream shines. You’ll see AWS Bedrock AgentCore and DeltaStream in action together, enabling agents that operate on continuously updated data rather than stale snapshots. If you're building AI agents or exploring production-grade GenAI architectures, this is a session you won’t want to miss. 👉 Register here: https://lnkd.in/gvR589Zb Breakfast and lunch will be served! 🙂 #AI #GenAI #AIAgents #AWS #AmazonBedrock #StreamingData #RealTimeData #DeltaStream
1 Comment
Explore top content on LinkedIn
Find curated posts and insights for relevant topics all in one place.
View top content