More Dev Interrupted episodes

Bet on research, not on vibes | Together AIs Dan Fu thumbnail

Bet on research, not on vibes | Together AIs Dan Fu

Published 3 Nov 2026

Duration: 00:45:58

Domain expertise amplifies AI's impact on GPU kernel development, while organizations must adapt management practices to attribute costs and distribute individual engineering gains.

Episode Description

This week, Dev Interrupted takes a field trip all the way down to the GPU. Leading the way is Together AI's VP of Kernels Dan Fu, a co-author of Flash...

Overview

Dan Fu, VP of Kernels at Together AI and assistant professor at UC San Diego, describes how his team runs the entire inference stack from data center request down to the GPU. He explains that research ideas like "looped models," where a transformer loops back through layers rather than processing them sequentially, now travel from lab curiosity to industry implementation in a far shorter window than during his PhD. A news report that "GPT-6 Astra might be a looped model" illustrated how compressed that gap has become.

The conversation turns to building an AI cloud, drawing an analogy to AWS where infrastructure extends beyond raw compute into databases and logging. Fu emphasizes that "training is a one-time cost in some sense" while value is delivered at inference. He stresses that domain expertise amplifies AI tooling: without deep understanding of the hardware and software stack, AI-generated code "might not be the right code." His team built custom Perfetto and trace viewers to open "windows" into the GPU black box before AI could write kernels at all.

On management, Fu argues that DRI ownership and engineering-manager responsibilities remain largely unchanged, though velocity and AI-spend attribution introduce new failure modes. He frames AI as an accelerant for individual engineers rather than a factory line, warning that concentrating gains in one "1000X" person creates a black hole where visibility is lost.

What If

  • What if you built a "window" into your system's black box before reaching for AI code generation?

    • Move: Build a lightweight trace or profiling viewer for your most opaque component (latency, throughput, data flow), mirroring how Fu's team created "our own Perfetto viewers, our own Trace viewers" to understand GPU behavior.
    • Why Now? Fu notes that "even before AI was able to write kernels at all, the ability to build those visualizations was hugely impactful." Without observability, AI output cannot be validated and may "not even be corrected because of some ABC different edge cases."
    • Expected Upside: You create a feedback loop where AI suggestions are checked against observed behavior, reducing wasted debugging cycles. (Proposal: for a solo developer, this substitutes for the team review that would normally catch incorrect code.)
  • What if you treated your AI tooling spend like a DRI-owned project with a monthly review?

    • Move: Track your AI consumption against shipped output (PRs, features, fixes) and do a monthly calibration, mirroring Fu's description of bringing "the AI spend of everybody in the organization" to managers for a "thumbs up" check.
    • Why Now? Fu identifies a "new failure mode" where someone "spend this much money on AI spend" without clear output. For a solo operator, untracked consumption is the only path to discovering you are paying for capability you are not leveraging.
    • Expected Upside: You can reallocate budget toward the highest-leverage AI workflows and cut the rest. (Possibility: this mirrors the calibration Fu describes where "most of the time it matches" between spend and impact.)
  • What if you wrote a one-page "agent readme" for your codebase before onboarding any AI agent?

    • Move: Create a markdown document describing architecture, conventions, and edge cases so that "an agent can pick up a piece of work and really start running with it," as Fu's team has invested in "markdown agent readme's about what's going on."
    • Why Now? Fu notes "there's a new set of best engineering practices that is developing around coordinating, orchestrating using agents." Without structural context, AI output lacks the domain grounding that makes it correct.
    • Expected Upside: You reduce the "spiky" quality of agent output by supplying the scaffolding Fu says is missing, potentially making generated code closer to production-ready on the first pass. (Proposal: this is a low-cost, high-leverage step for a solo developer who cannot rely on peer review.)

Takeaway

  • Build a trace or profiling viewer for your most opaque system component before relying on AI to generate code for it, so you can validate output against observed behavior rather than accepting plausible-looking results.
  • Write a markdown "agent readme" documenting your architecture, conventions, and known edge cases so AI agents have the domain context needed to produce correct code on the first pass.
  • Track your AI tooling spend monthly against shipped output (PRs, features, fixes) and cut or redirect any workflow where the cost-to-value ratio is unclear, treating it as a DRI-owned project.
  • Keep a named DRI for every feature or project and update your planning cadence to match the faster velocity AI introduces, rather than stretching old timelines that no longer reflect actual throughput.
  • Distribute AI gains across your workflow rather than concentrating them in one area, to avoid creating a black hole where you lose visibility into what is actually being built and its current state.

Recent Episodes of Dev Interrupted

More Dev Interrupted episodes