Gokul Reddy Muddarla

What is Function calling & MCP for LLMs? (explained with visuals and code) Before MCPs became popular, AI workflows relied on traditional Function Calling for tool access. Now, MCP is standardizing it for Agents/LLMs. The visual below explains how Function Calling and MCP work under the hood. Today, let's learn: - Function calling by building custom tools for Agents. - How MCPs help by building a local MCP client with mcp-use and using tools from Browserbase MCP server. In Function Calling: - The LLM receives a prompt. - The LLM decides the tool. - The programmer implements a procedure to accept a tool call request from the LLM and prepare a function call. The tool call request is found in the LLM's response when you prompt it. - A backend service executes the tool. This Function Calling takes place within our stack: - We host the tool. - We implement a logic to determine the tool to invoke and its parameters. - We execute it. So Function Calling requires us to wire everything manually. MCP simplifies this! Instead of hard-wiring tools, MCP: - Standardizes defining, hosting, and exposing tools. - Makes it easy to discover tools, understand schemas, and use them. - Demands approval before invoking them. - Detaches implementation from consumption. For instance, whenever you integrate an MCP server, you never write a line of Python code to integrate the tools. Instead, you just integrate the MCP server and everything beyond this follows a standard protocol handled by the MCP client and the LLM: - They identify the MCP tool. - They prepare the input argument. - They invoke the tool. - They use the tool’s output to generate a response. Everything happens through a standard (but abstracted) protocol. So here’s the key point: MCP and Function Calling are not in conflict. They’re two sides of the same workflow. - Function Calling helps an LLM decide what it wants to do. - MCP ensures that tools are reliably available, discoverable, and executable, without you needing to custom-integrate everything. For example, an agent might say, “I need to search the web,” using function calling. That request can be routed through MCP to select from available web search tools, invoke the correct one, and return the result. Check the workflow in the diagram below. In this setup, to build a local MCP client, I used mcp-use because it lets us connect any LLM to MCP servers & build private MCP clients, unlike Claude/Cursor. - Compatible with Ollama & LangChain - Stream Agent output async - Built-in debugging mode, etc Find the mcp-use GitHub repo in the comments! ____ Find me → Avi Chawla Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs.

269

11 Comments

Pretty exciting new work from Kuaishou in the Generative Recommender domain. They replaced the entire video recommendation funnel (retrieval - ranking - value model) with a single encoder/decoder architecture trained on user action sequences with next-item prediction task, similar to how LLMs are pre-trained to predict the next token. The key to make this all work is their tokenizer, which is trained to map video content and metadata to semantic item IDs — so that similar videos share similar IDs, making generation feasible even at billion-scale vocabularies. On top of that, they apply reinforcement learning with user feedback to break through the ceiling of the traditional recommender stack. Based on their A/B tests this resulted in +0.5% App Stay time in the Kuaishou app. This finding shows just how much headroom there is left on top of traditional recommender systems! More about this in my blog: https://lnkd.in/gCr6k-MD

148

2 Comments

You're in a ML Inference Engineer interview at NVIDIA, and the interviewer asks: "Our team wants to switch from API to a self-hosted. Which serving framework and why?" Here's how you answer: Don't say: "vLLM is fastest" or "Ollama is easiest." Wrong framing. The real answer isn't about features - it's about matching serving philosophy to your constraints. Local prototype vs. production scale vs. complex workflows = completely different frameworks. NOTE - if you want to read this kinda content daily, consider subscribing my free newsletter - https://lnkd.in/g8ZJGsWj Most teams pick based on benchmarks, not bottlenecks Your serving choice matters less than your use case. PagedAttention isn't magic - it's just better memory management "Fastest" framework can be slowest for YOUR workload The infrastructure reality everyone misses: Single user prototype = Ollama/LLaMA.cpp wins Multi-user production = vLLM dominates AI agent workflows = SGLang is your only option Edge deployment = LLaMA.cpp or bust Architecture drives the decision, not features. "But what about performance?" Interviewer: "How do you handle 1000 concurrent users?" Performance without context is meaningless. > vLLM's PagedAttention gives you 24x throughput, but only if memory fragmentation was your bottleneck. > SGLang's RadixAttention only helps with cache reuse scenarios. By the way, just reading about inference frameworks won't help you land a job. I personally learnt a lot by doing stuff. If I were to do all over, I would rent a gpu from Hyperbolic for dirt cheap, and play with some open source model with different frameworks. The business framework that matters: > Rapid prototyping + Local dev = Ollama > Known high load + GPU budget = vLLM > Complex multi-step LLM workflows = SGLang > Constrained environments + Portability = LLaMA.cpp Match the tool to the business constraint. The evolution path most teams miss: > Start: Ollama (prototype fast) > Scale: vLLM (handle users) > Optimize: SGLang (complex workflows) > Edge: LLaMA.cpp (deploy anywhere) It's not replacement - it's graduation based on needs. The answer that gets you hired: "Framework choice isn't about better vs worse. It's about constraints vs capabilities. Ollama gives you speed-to-market. vLLM gives you user scale. SGLang gives you workflow complexity. Pick based on your biggest constraint, not your biggest benchmark." That's it for today. See ya tomorrow! And if you want to read more about these frameworks, checkout this blog: https://lnkd.in/eiH2sphA #llm #machinelearning #gpu #chatgpt #ai #agents

398

17 Comments

A recent study by Carnegie Mellon University ranked GPT Researcher #1 across most popular benchmarks, beating OpenAI's deep research, Perplexity Sonar, Hugging Face, and other leading research agents. The CMU team tested us on 1,000 complex queries. GPT Researcher led in: - Information coverage (64.67% Key Point Recall) - Citation accuracy (85.36% precision) - Source thoroughness (90.82% recall) How did an open source project outperform billion dollar companies? This is the true testament of a community of builders. Every contribution mattered. Every issue reported, every feedback, every PR merged, every discussion in Discord built toward this. Thank you to everyone who's been part of this journey. Full paper: https://lnkd.in/daRkmS9Y

175

19 Comments

Everyone’s talking about LLMs… but fewer people are asking: Where does all that knowledge actually live? The truth is, every great AI application has a “memory layer” behind it — and that’s where vector databases come in. They’re the quiet infrastructure powering similarity search, recommendations, and retrieval-augmented generation (RAG). Without them, most of the magic we see in GenAI wouldn’t scale. Here’s a quick breakdown of the open-source landscape 👇 🔹 Purpose-Built Vector Databases Milvus → the market leader, handles billions of vectors, cloud-native, supports IVF, HNSW, DiskANN. Weaviate → versatile hybrid search + built-in AI modules. Qdrant (Rust-based) → strong performance + JSON payload filtering. ChromaDB → developer-friendly, deeply integrated with LangChain & LlamaIndex. Deep Lake → optimized for multimodal data (text, audio, video). Vespa / Vald → trusted for high-scale, cloud-native search. 🔹 Database Extensions Adding Vector Support Postgres + pgvector → if you live in SQL-land, this is the bridge. Elasticsearch / OpenSearch → blend classical keyword search with vectors. Cassandra 5.0 → distributed-first, now with vector queries. Redis / Neo4j → flexible overlays: graph + cache + vectors. 🔹 Libraries & Frameworks Faiss (Meta) → the workhorse for vector similarity search. ScaNN (Google) → speed + accuracy using partitioning & quantization. Annoy (Spotify) → an early classic, now succeeded by Voyager. Lucene / Solr → battle-tested search engines with added vector power. Picking a vector store isn’t just about speed — it’s about fit. Need huge scale? Milvus, Weaviate, Qdrant. Already love Postgres? Use pgvector. Building custom? Start with Faiss or ScaNN. In short: the best database is the one that meets your needs, not the other way around. If you were building a production LLM app today, would you go “all in” with a purpose-built vector DB — or extend the database you already know (Postgres, Elastic, etc.)? Why?

44

13 Comments

There’s a new RL algorithm in town… and it’s already shaking up the LLM research community. 🤯 Group Sequence Policy Optimization (GSPO) was created by the research team at Qwen and is purpose built for operating at the sequence level rather than the token level. This means it evaluates, clips, and optimizes entire response sequences by their overall likelihood. The important upshot is that it directly aligns the optimization process with how rewards are actually assigned in real world scenarios! Some key reasons to be excited! 1. Better stability. GSPO improves training stability and helps avoids model collapse that can happen with algorithms like GRPO. 2. Scalable RL for mixture of experts (MoE) models. GSPO inherently solves stability challenges teams face when scaling RL for MoE architectures. Large scale RL becomes more practically feasible! 3. Simplifies Infra. Sequence-level optimization is inherently less sensitive to small numerical differences between training and inference systems compared to token-level methods, which should lead to more robust RL pipelines! We can already see signs that this has paid off in big ways for the open-source community. Qwen3 models are doing really well on benchmarks, boasting competitive performance with top models like Gemini-2.5 Pro on GPQA, AIME25, and LiveCodeBench! https://lnkd.in/gCZMQvUj P.S. 👀 If you want to run RL on your own Qwen3 model with GRPO/GSPO, keep an eye on our open-source post-building repo 🧡 : https://lnkd.in/gXZ32qkm #AI #LLM #ReinforcementLearning

82

10 Comments

🎉 New Amazon Web Services (AWS) AI blog! A few weeks ago, Dr. Natalie Perez, Ph.D. and I had collaborated with some brilliant minds on a research paper related to using LLMs as judges of qualitative product feedback. Over the last few weeks, our paper received a lot of engagement, not only from scientists and academic researchers, but also program and product managers who wanted to implement this for their own use cases and products. 💡 One question kept popping up: “This is awesome...but can you walk us through how to do it step by step?” So, we did the next logical thing: We partnered with two amazing Amazonians: John Kitaoka (who can translate complex customer problems into easy-to-implement, scalable solutions over coffee chats) and Dr. Elizabeth Conjar, Ph.D. (our go-to expert for all things IO psychology and evaluation) to create a step-by-step blog guide on how to implement this approach using AWS Bedrock. 📘 Whether you're a PM, UX researcher, or scientist looking to scale your qualitative text related feedback analysis, this blog is for you. Link: https://lnkd.in/evweGSRx Let us know how you implement this for your use cases! Amazon Web Services (AWS) Amazon Science

84

11 Comments