Scaling Past Informal AI - Carina Hong, Axiom Math

The podcast explores the role of verified AI and formal verification in advancing AI systems, emphasizing their potential to enhance human capabilities rather than merely address flaws. It highlights formal verification as a foundational approach for ensuring correctness in AI, drawing parallels to historical mathematical achievements like Ramanujans work. Axiom Maths, a company leveraging formal verification, is profiled for its AI systems that excel in mathematics, including a perfect score on the Putnam competition and recent significant funding. The discussion contrasts human-human, human-AI, and future AI-agent collaboration models, framing verification as a tool for scaling brilliance rather than compliance. Key tools like Lean, a formal proof language, are detailed, with their integration into coding and proof generation, and their potential to automate low-level tasks while enabling complex reasoning. Challenges include the difficulty of auto-formalizing mathematical statements, generalization across domains, and the limitations of AI in creative combinatorial reasoning despite progress in formal proofs. The conversation also touches on the importance of mathematical infrastructure, such as mathlib, and the role of transfer learning in bridging formal proof systems with broader AI applications.

The podcast delves into the broader implications of formal verification, noting its historical use in industries like aerospace and software, and its growing relevance in AI for ensuring reliability in hardware and software systems. It addresses the tension between formal and informal verification methods, asserting that the former offers greater scalability and precision, though it faces challenges in handling complex proofs and creative problem-solving. Axioms focus on developing verified AI as a superhuman mathematician is contrasted with the limitations of large language models in handling extremely large formal proofs. The discussion also covers the role of human intuition and cultural factors in defining elegance in solutions, as well as the need for interdisciplinary collaboration to overcome fragmentation in the AI field. Finally, it outlines future visions for AI, including the potential of verified reasoning engines to tackle ambitious goals beyond traditional applications, while acknowledging the risks of overclaiming solutions and the importance of rigorous verification processes in advancing AI capabilities.

• What if you focused on building a tool that integrates Lean verification directly into your code development workflow?

  • Move: Develop a plugin or API that auto-generates and validates formal proofs for critical code segments using Lean, even for non-mathematical applications (e.g., blockchain smart contracts).
  • Why Now?: The text emphasizes the role of formal verification in scaling brilliance and performance gains, especially for startups with limited resources. Leans ability to handle code and proofs via Curry-Howard correspondence makes this feasible.
  • Expected Upside: Your tool could reduce debugging time, increase trust in your software, and position you as a pioneer in verified AI for practical domains beyond pure math.

• What if you launched an MVP for an auto-formalization tool targeting a specific mathematical domain (e.g., number theory)?

  • Move: Create a system that trains on synthetic data (like Axioms lean proofs) to auto-generate formal proofs for problems in a niche area, using recursive decomposition techniques mentioned in the text.
  • Why Now?: The challenge of auto-formalization is highlighted as a bottleneck, but Axioms success in math-specific domains shows its viable. Leans infrastructure and existing benchmarks (e.g., PUNM exam) provide a clear path.
  • Expected Upside: Capture early traction in specialized markets (e.g., education or finance) and attract partnerships with research groups focused on formal math infrastructure.

• What if you contributed to open-source math infrastructure by creating a lean-based platform for collaborative formalization?

  • Move: Build a shared platform (like Axle) that allows non-experts to participate in formalizing proofs using AI-assisted tools (e.g., LLM-based repair methods for Lean), inspired by community-driven projects.
  • Why Now?: The text stresses the importance of community collaboration and shared standards for formal math. Open-access platforms could democratize verification and reduce the expertise barrier.
  • Expected Upside: Accelerate adoption of formal verification in your projects, gain credibility in the math and AI communities, and potentially open doors to partnerships with academic or industry research groups.

• Prioritize Formal Verification with Lean or Similar Tools for Robust Code Development
  Integrate formal verification tools like Lean into your software workflows to ensure correctness, especially for critical systems. Use Leans grind tactics to automate low-level proof steps, allowing you to focus on high-level design and innovation, even as a solo developer.

• Leverage Verified AI as a Foundation for Scaling Expertise
  Adopt verified AI models (e.g., Axioms systems) to scale human brilliance in domains like math, code generation, or proof construction. Focus on using verification not for compliance but to amplify your own reasoning capabilities and reduce error-prone manual tasks.

• Invest in Math-Heavy Infrastructure for Transferable AI Capabilities
  Build or integrate math-driven infrastructure (e.g., Leans mathlib, synthetic proof datasets) to train AI systems that can generalize across reasoning tasks. This aligns with Axioms success in turning math into a scalable foundation for AI, enabling cross-domain problem-solving.

• Create Synthetic Data Repositories for Auto-Formalization
  Develop and curate synthetic datasets of formal proofs (e.g., using Lean) to train AI models in auto-formalization. This addresses the bottleneck of training data localization and helps your system generalize beyond niche domains with minimal human intervention.

• Collaborate with the Formal Math Community for Tool Adoption and Validation
  Engage with open-source communities (e.g., Leans ecosystem) to adopt standardized tools like Axle (Axioms API) and contribute to shared formalization projects. Use community-driven benchmarks (e.g., IMO problems) to validate your AIs reasoning capabilities and accelerate development.