More MLOps.community episodes

Conversation with the MLflow Maintainers thumbnail

Conversation with the MLflow Maintainers

Published 16 Jan 2026

Duration: 00:58:23

MLflow is adapting to support traditional machine learning and emerging agent-based AI systems, tackling challenges in chatbot development such as observability and effective memory management.

Episode Description

Corey Zumar is a Product Manager at Databricks, working on MLflow and LLM evaluation, tracing, and lifecycle tooling for generative AI.Jules Damji is...

Overview

The podcast explores how MLflow is expanding its capabilities to support not only traditional machine learning but also agent-based AI systems, which are becoming increasingly common in production environments. Chatbots, in particular, are evolving from simple text-based interfaces to more complex, multimodal systems capable of tool calling and image processing. To keep pace with these advancements, MLflow is being adapted to accommodate new agent paradigms while staying true to its open-source ethos. Challenges in this space include improving observability, developing multi-turn evaluation frameworks, and managing memory to ensure coherent, context-aware interactions. Techniques like multi-turn judges from systems such as DeepEval and Ragaz are being used to assess agent performance across entire conversations.

The discussion also touches on key areas such as prompt engineering, feedback loops, and governance in AI systems, emphasizing the need for secure data handling and access control. There is a growing need for tools that can support the evaluation, feedback collection, and development of both models and agents within a single platform. As the boundaries between data science and agent development blur, MLflow and similar tools are playing a crucial role in unifying these workflows and addressing the complex challenges of building and maintaining advanced AI systems.

Recent Episodes of MLOps.community

9 Nov 2026 My AI Agent Bought Me A Car

Hermes agent uses reinforcement learning and trace-to-environment synthesis to enable continual self-improvement, balancing token efficiency with inference-time steering for better agentic performance.

2 Nov 2026 How Stanford Teaches AI Coding Now

AI agent development, verification loops, and go-to-market automation are reshaping software engineering and sales strategies.

29 Oct 2026 From Slop Code to Software Factories

"Tandem biking in Amsterdam's chaotic cycling culture led to a critique of AI-assisted coding's flaws, particularly its tendency to produce unmaintainable, poorly structured code, raising concerns about long-term software reliability."

22 Oct 2026 Token Maxxing Is Over and Here Is What Comes Next

"AI in software development offers benefits but faces challenges like nonsensical outputs and over-reliance; smaller, controlled models like Sonnet improve adoption, while measuring ROI and human-AI collaboration are key to success."

More MLOps.community episodes