Published 9 Jan 2026
Duration: 4694
Artificial Analysis, an independent AI benchmarking platform, provides standardized evaluations and reports to address the lack of comprehensive and impartial AI benchmarks.
dont miss Georges AIE talk: https://www.youtube.com/watch?v=sRpqPgKeXNkFrom launching a side project in a Sydney basement to becoming the independent...
The podcast outlines the development of Artificial Analysis, an independent benchmarking platform launched in January 2024 to evaluate AI models and hosting providers. Initially a side project, it has evolved into a service offering public and private benchmarking, standardized reports, and custom evaluations for enterprises and AI companies. The platform aims to address the lack of comprehensive and impartial AI benchmarks, focusing on challenges such as model accuracy, cost, and performance in AI development. It operates through a free website, generating revenue from enterprise subscriptions and private evaluations.
The discussion touches on the evolution of benchmarking methodologies, the difficulties in evaluating AI models, and the introduction of new metrics like the Omniscience Index. It also highlights the increasing importance of measuring hallucination rates and model openness. Other topics include trends in AI model costs and performance, the emergence of agentic workflows, and the growing complexity and diversity of the AI ecosystem.
30 Mar 2026 Mistral: Voxtral TTS, Forge, Leanstral, & what's next for Mistral 4 w/ Pavan Kumar Reddy & Guillaume Lample
Mistral's Voxtral TTS is a 3B-parameter text-to-speech model leveraging neural audio codecs, semantic/acoustic token splitting, and efficient flow matching for multilingual real-time applications, balancing quality and cost while exploring future refinements in architecture, tokenization, and domain-specific training.
24 Mar 2026 Why There Is No "AlphaFold for Materials" AI for Materials Discovery with Heather Kulik
AI's role in education and materials science is debated, highlighting its potential to accelerate research in chemistry and physics through machine learning, yet acknowledging limitations in complex molecular design, data quality challenges, and the need for interdisciplinary collaboration and robust validation.
20 Mar 2026 Dreamer: the Personal Agent OS David Singleton
Dreamer is an AI platform democratizing access to agentic tools for non-technical users via customizable AI assistants, community-built apps, cross-device integration, and privacy-focused features, with a beta emphasis on accessibility, real-world productivity use cases, and third-party developer opportunities.
17 Mar 2026 Why Anthropic Thinks AI Should Have Its Own Computer Felix Rieseberg of Claude Cowork & Claude Code Desktop
Cowork is a cloud-native tool simplifying Cloud Code access for non-technical users through a balance of simplicity and flexibility, integrating with cloud services and Chrome while supporting diverse workflows via a lightweight virtual machine, developed in 10 days with a focus on modular components and ethical trade-offs in AI infrastructure.
12 Mar 2026 Retrieval After RAG: Hybrid Search, Agents, and Database Design Simon Hrup Eskildsen of Turbopuffer
TurboPuffer is a next-gen database platform focusing on AI-powered vector search, full-text search, and scalability, with a strong emphasis on hiring top talent and customer-driven innovation.