![[LIVE] Anthropic Distillation & How Models Cheat (SWE-Bench Dead) | Nathan Lambert & Sebastian Raschka thumbnail](https://substackcdn.com/feed/podcast/1084089/post/189277598/ca7468da5614a246d2906ee8926f6de7.jpg)
Published 26 Feb 2026
Duration: 3137
A podcast discusses the application of distillation in machine learning, particularly in training smaller language models, and raises concerns about the potential for malicious use and the need for improved detection and regulation.
Swyx joined SAIL! Thank you SAIL Media, Prof. Tom Yeh, 8Lee, Hamid Bagheri, c9n, and many others for tuning into SAIL Live #6 with Nathan Lambert and...
The podcast covers the concept of distillation in machine learning, focusing on its application in training smaller language models using the outputs of larger ones. This process can involve using logits from traditional models or synthetic data generated by large language models (LLMs), enabling the creation of more efficient, deployable models. Major companies like DeepSeq and Google are highlighted as using distillation to reduce the size and computational demands of their models while maintaining performance.
The discussion also addresses ethical and legal challenges, particularly around "distributed distillation attacks," where third-party AI labs may use outputs from competing models to train their own systems. This raises concerns about AI geopolitics and the enforcement of terms of service. The podcast explores detection methods for such attacks, such as identifying unusual usage patterns, but notes the difficulty in distinguishing between legitimate training practices and malicious intent. Additionally, the episode touches on the broader implications of distillation for AI innovation and regulation, while examining the role of benchmarks like Sweet Bench in evaluating model performance, despite their known limitations in accurately assessing LLM capabilities across varied domains.
5 May 2026 Doing Vibe Physics Alex Lupsasca, OpenAI
AI is advancing theoretical physics by rapidly solving complex problems like quantum field theory calculations and simulating models such as SYK, though it still relies on human collaboration for original insights and contextual validation, reshaping research methodologies and education.
27 Apr 2026 Physical AI that Moves the World Qasar Younis & Peter Ludwig, Applied Intuition
Applied Intuition develops safety-critical physical AI for automotive, construction, mining, and defense sectors, selling AI technology to manufacturers and governments through simulation, infrastructure, and proprietary systems to advance industrial innovation with reliable autonomy.
23 Apr 2026 AIE Europe Debrief + Agent Labs Thesis: Unsupervised Learning x Latent Space Crossover Special (2026)
The text discusses AI's evolving landscape, focusing on experimental agents potentially breaking containment by 2026, market disruptions from foundation models, infrastructure advancements like RAG, debates between infrastructure and application firms, outsourcing strategies, pre-2023 training data advantages, competitive coding AI sectors, and future trends in personalization and industry transformation amid scalability and quality challenges.
Shopify's AI strategies involve in-house tools like Tangled and QMD to automate workflows, collaborate with the AI community, address challenges in token usage and code quality, and explore applications in e-commerce, CI/CD optimization, and scalable AI experimentation.
CLIs and MCPs are emphasized for enterprise efficiency, alongside challenges in early AI integration, custom agent development for automation, strategic AGI management, and balancing automation with oversight, pricing, and collaboration tools like Notion.