Srinivas Malyala’s Post

I recently completed a hands-on bootcamp on building and productionalizing AI agents. A big thank-you to Eumar Dias de Assis and DataExpert.io for creating such a practical learning experience. We built Zachy, a multi-persona AI agent. In the end-state design, requests from a web app, Slack, or an IDE enter a Model Serving endpoint protected by guardrails and rate limits. A Supervisor Agent—implemented with Agent Bricks or a LangGraph router—then directs each request to the right specialist: Prospective Student: managed Knowledge Assistant over the bootcamp catalog Enrolled Student: Knowledge Assistant plus Vector Search over course and GitHub content Employee: Unity Catalog functions and Genie for T&E queries, receipt status, and submission Unity Catalog governs Volumes, tables, functions, models, Vector Search, and MCP servers. MLflow provides tracing, evaluation, model registration, and LLM-as-a-judge. One hands-on implementation began with synthetic receipts. We loaded the Voxel51/scanned_receipts Hugging Face dataset and used a Pandas UDF to enrich the images with synthetic employee metadata at scale. We then built a SQL-native document pipeline—without training a custom model: read_files(BINARYFILE) → ai_parse_document → combine element text → ai_extract → structured fields The distinction matters: ai_parse_document accepts raw bytes and understands layout, including tables, columns, and bounding boxes. ai_extract works on text to extract vendor, amount, date, and category. ai_classify flags potentially noncompliant expenses, such as alcohol or luxury hotels, for review. Retrieval and chunking were persona-specific. We explored fixed-size, recursive, semantic, structure-aware, parent-child, and code-aware chunking. Dense search suited similar-code discovery, hybrid search worked well for documentation, and live GitHub access handled exact, up-to-date files. For actions—not just answers—we exposed governed Unity Catalog functions through MCP. This allowed Zachy to query T&E summaries, retrieve files, check receipt status, and submit receipts while retaining permissions and auditability. Finally, we productionized the agent with: LangGraph and MCP for tool-calling orchestration MLflow ResponsesAgent, tracing, registration, and deployment LLM judges for relevance, safety, groundedness, and correctness Human feedback and labeling sessions for expert review Unity AI Gateway for PII guardrails, rate limits, inference tables, and cost visibility My biggest takeaway: agent quality is a system property. A production agent is not just an LLM with a prompt. It combines retrieval, governed tools, runtime controls, observability, evaluation, and continuous human feedback. #AIAgents #GenerativeAI #Databricks #MLflow #RAG #MCP #LangGraph #MLOps