Published 16 Jan 2026
Duration: 00:58:23
MLflow is adapting to support traditional machine learning and emerging agent-based AI systems, tackling challenges in chatbot development such as observability and effective memory management.
Corey Zumar is a Product Manager at Databricks, working on MLflow and LLM evaluation, tracing, and lifecycle tooling for generative AI.Jules Damji is...
The podcast explores how MLflow is expanding its capabilities to support not only traditional machine learning but also agent-based AI systems, which are becoming increasingly common in production environments. Chatbots, in particular, are evolving from simple text-based interfaces to more complex, multimodal systems capable of tool calling and image processing. To keep pace with these advancements, MLflow is being adapted to accommodate new agent paradigms while staying true to its open-source ethos. Challenges in this space include improving observability, developing multi-turn evaluation frameworks, and managing memory to ensure coherent, context-aware interactions. Techniques like multi-turn judges from systems such as DeepEval and Ragaz are being used to assess agent performance across entire conversations.
The discussion also touches on key areas such as prompt engineering, feedback loops, and governance in AI systems, emphasizing the need for secure data handling and access control. There is a growing need for tools that can support the evaluation, feedback collection, and development of both models and agents within a single platform. As the boundaries between data science and agent development blur, MLflow and similar tools are playing a crucial role in unifying these workflows and addressing the complex challenges of building and maintaining advanced AI systems.
19 Jun 2026 Sandboxing, Agent Harnesses, and Agent Teamwork
The text examines "Harness" componentsprompts, tools, and feedback systemsthat balance AI agent autonomy with control through adaptive strategies, human oversight, and iterative testing to improve reliability and alignment with human judgment in dynamic tasks.
16 Jun 2026 MCP Servers Are Becoming the UI for AI Agents
Gateways as proxies for AI via MCP address security, traffic control, and cost management while tackling server development challenges, optimization of tool calls, microservices scaling, protocol tracing limitations, ownership shifts, and the need for unbiased evaluations and agent-driven usability assessments.
12 Jun 2026 MCP, Agents & the $40M Bet on Multiplayer AI
Recommended: Multiplayer Bots as a Action Paradigm
The integration of AI into work practices shifts toward collaborative "multiplayer" systems using flocking-inspired dynamics, addressing challenges like limited AI time horizons, technical tools for shared collaboration, balancing human-AI roles, infrastructure scaling, and the need for adaptive governance and futureproofing.
9 Jun 2026 From Single-Player to Multi-Player: Operating AI Agents at Scale
AI agent infrastructure and governance require control planes for security, compliance, and risk mitigation, addressing operational challenges, productivity gains, and the need for standardized frameworks, modular designs, and transparent collaboration.
5 Jun 2026 The Control-vs-Magic Spectrum Building Agents
iFood Pago leverages AI-driven tools like ChatBank to automate financial services for Brazilian restaurants, balancing automation with personalization while addressing challenges in scaling AI, risk management, and the impact of declining training costs on software accessibility.