From Math Olympiads to Navier-Stokes: How Fast Is AI Progressing? with Greg Burnham

The podcast discusses the rapid advancement of AI capabilities, particularly in software development and mathematics. In software engineering, autonomous platforms like Blitzi use specialized agents to reverse-engineer complex codebases, enabling large-scale modernization of legacy systems and accelerating feature development. Meanwhile, AI has made extraordinary progress in mathematics, moving from struggling with basic problems to solving advanced graduate-level challenges and even contributing to long-standing unsolved problems such as those posed by Paul Erdos and the Navier-Stokes equations - one of the Millennium Prize Problems. These achievements highlight a shift from AI performing well on structured exams to engaging in open-ended research, though its solutions often build on existing human knowledge rather than introducing entirely novel theories.

A major focus is on how AI progress is measured through benchmarks, with observations showing steady, linear improvement across domains despite the increasing difficulty of tasks. The correlation between performance in different areas - such as coding, math, and chemistry - suggests broad generalization abilities, possibly due to underlying improvements in reasoning or training scale. However, challenges remain in assessing true innovation, as current benchmarks often rely on all-or-nothing outcomes and struggle to capture partial progress or creativity. Researchers are also exploring whether AI can generate genuinely new ideas or learn dynamically during tasks, with experiments using board and video games to test on-the-fly adaptation. While AI excels in specific, well-defined domains, it still lags in open-ended problem-solving and self-driven learning, raising questions about the potential for recursive self-improvement and the future trajectory of superhuman capabilities.

• What if you built a solo AI-powered legacy code modernization service using autonomous agent patterns?

  • Move: Use open-source LLMs (e.g., CodeLlama, DeepSeek-Coder) combined with a lightweight agent framework (e.g., LangGraph or SmythOS) to reverse-engineer and refactor small-to-medium legacy codebases (e.g., Python monoliths, old Django apps). Offer automated dependency mapping, tech stack migration (e.g., Django FastAPI), and test generation as a turnkey service on platforms like Gumroad or via direct outreach.
  • Why Now?: AI can now parse complex architectures and generate coherent, validated code - mirroring Blitzi's capabilities at enterprise scale - but no one has productized this for indie developers or small businesses drowning in technical debt. The gap between AI capability and accessible tooling is widest here.
  • Expected Upside: Capture early-mover advantage in a high-friction niche: charge $500 - $2,000 per project with <10 hours of manual oversight. At 10 projects/month, this scales to $10k - $20k MRR with minimal overhead.

• What if you launched a micro-SaaS that benchmarks AI model performance on real-world coding tasks instead of synthetic exams?

  • Move: Create a self-hosted benchmarking suite that evaluates AI models (e.g., local LLMs, API-based) on practical software development workflows - e.g., "Fix this production bug," "Add OAuth to this Flask app," or "Migrate this SQL schema." Publish live leaderboards and offer paid reports comparing GPT-4o, Claude 3.5, Gemini, etc., for specific dev tasks.
  • Why Now?: As shown in the text, traditional benchmarks are gamed and plateau quickly. Real-world task performance is uneven ("jagged frontier") and poorly measured - yet critical for developers choosing tools. With AI progress accelerating, demand for actionable, non-academic evaluations is rising.
  • Expected Upside: Monetize through affiliate links to model APIs, premium reports for teams, and sponsorships from dev tool vendors. Even 1,000 targeted visitors/month could convert into $3k - $5k MRR within six months.

• What if you trained a specialized AI agent to solve niche, unsolved technical problems for bounties or IP licensing?

  • Move: Curate a list of publicly posted technical challenges with monetary rewards (e.g., GitHub bounties, NASA open innovation, cryptography puzzles) or unsolved industry pain points (e.g., optimizing cold chain logistics). Fine-tune a reasoning model (e.g., DeepSeek-R1, o1-preview-style) on domain-specific literature and deploy it to generate novel solutions, submitting only when confidence is high.
  • Why Now?: AI recently solved Erdos and Navier-Stokes-level problems - proof that it can tackle human-stuck challenges. These systems use persistence and cross-domain knowledge, which a focused solo operator can replicate at low cost using fine-tuning and retrieval-augmented generation (RAG).
  • Expected Upside: Win bounties ($1k - $50k+), license generated algorithms, or build a reputation to sell consulting or spin out a product. One successful solve could yield more than a year of freelance income.

• Monitor and integrate emerging AI coding agents like Blitzi to reverse-engineer and modernize legacy codebases, reducing technical debt and accelerating feature delivery.
• Use curated, domain-specific benchmarks (e.g., Frontier Math-style problem sets) to stress-test AI tools on unsolved or high-difficulty problems relevant to your software niche.
• Focus on measurable, partial-credit metrics when evaluating AI-generated code or solutions, especially for complex tasks where all-or-nothing outcomes obscure incremental progress.
• Differentiate between model improvements and harness-based gains by testing AI performance with minimal external tooling before adopting new systems for core development work.
• Proactively test AI's ability to learn on-the-fly in novel environments (e.g., new games or workflows) as a proxy for adaptability in real-world client projects requiring zero prior training data.