How to Find the Agent Failures Your Evals Miss with Scott Clark

The text outlines Distributional, an AI analytics platform focused on improving agent quality through analysis of production data. It emphasizes a hierarchical approach to observability, starting with foundational telemetry (logging system behavior), progressing to monitoring (real-time tracking of predefined metrics), and culminating in analytics (uncovering hidden patterns in production data to refine agents via unsupervised learning and feedback loops). The platform leverages Bayesian statistics and insights from Scott Clarks work on optimization (e.g., Bayesian methods at Yelp and SIGOpt) to address challenges like overfitting in black-box optimizers and the need for meaningful, domain-specific goals in AI system design. It shifts from pre-production testing to post-production analytics to better align with real-world dynamics, such as detecting agent "hallucinations" (e.g., false tool calls in financial agents) and identifying "unknown unknowns" through statistical anomalies in behavioral patterns.

The text also highlights analytics role in continuous adaptation, using techniques like vector mapping, clustering, and large language model (LLM)-driven analysis to detect subtle deviations from expected behavior (e.g., unusual tool call distributions or emergent risks in non-stationary environments). It contrasts monitoring (ensuring system health) with analytics (identifying optimization opportunities), both being critical for iterative system improvement. Key challenges include parsing unstructured data (e.g., logs), aligning evaluation metrics with business needs, and managing complexity in agentic systems. Security is emphasized as a critical application area, using analytics to uncover hidden signals or anomalies in agent behavior. The platform is framed as a post-production tool, designed for enterprises, with open-source deployment options and a focus on bridging gaps between model performance and real-world reliability. It also addresses the need for new benchmarks to evaluate analytics tools in domains like cybersecurity, where detecting subtle risks is crucial.