Making AI agents work in your team with Dennis Pilarinos

The podcast discusses the growing use of AI in software development, highlighting a significant increase in AI token consumption across industries without a corresponding rise in productivity. While AI tools have led to higher output - such as a 30% repeated increase in pull requests - teams are struggling with bottlenecks, particularly in code review capacity. The surge in AI-generated code has outpaced human review capabilities, creating inefficiencies and raising concerns about sustainability, code quality, and return on investment.

A key theme is the importance of context in enabling effective AI-assisted development. The discussion emphasizes that AI agents require access to up-to-date, synthesized organizational knowledge - including documentation, source code, messaging platforms, and runtime systems - to make accurate decisions efficiently. Static documentation is seen as insufficient and quickly outdated, leading teams to adopt dynamic systems and pre-computed context layers that provide real-time, reconciled information. This approach reduces token waste, improves decision-making, and supports scalable automation in development workflows.

The podcast also explores practical implementations of AI in engineering teams, such as automated issue detection and resolution using integrated tools like Sentry, Slack, and GitHub. Systems are being designed to allow AI agents to propose and deploy low-risk fixes autonomously while reserving human oversight for high-risk or product-facing changes. There is a strong focus on balancing automation with security and maintainability, as well as evolving release strategies - such as immediate rollouts to all users - to accelerate feedback loops. The vision centers on building robust, context-aware agent infrastructures that enable developers to focus on higher-level tasks while AI handles routine development and maintenance.

• What if you redirected AI token spending from code generation to pre-computed context synthesis?

  • Move: Audit your AI usage over the last month; identify % of tokens spent on redundant context gathering (e.g., querying docs, Slack, code). Build a pre-indexed context layer (e.g., using a local vector store or knowledge graph) that agents query instead of real-time sources.
  • Why Now?: Token costs are rising while productivity gains stall - teams are hitting diminishing returns from raw generation. Pre-computing context cuts token waste by up to 40% (based on Unblocked's efficiency data) and accelerates agent response.
  • Expected Upside: Reduce AI spend by 30 - 50% while improving output quality and consistency. Free up budget to invest in higher-leverage automation (e.g., auto-merging low-risk PRs).

• What if you automated documentation updates as a side effect of every pull request?

  • Move: Set up a workflow where AI agents detect code changes, infer documentation impact, and open PRs to update .md files in Notion or GitHub. Use risk scoring to auto-merge low-risk doc updates; flag high-risk ones for review.
  • Why Now?: Static docs decay rapidly, creating knowledge gaps that hurt both humans and AI agents. With AI-generated PR volume up 30%+ per cycle, manual doc updates are unsustainable.
  • Expected Upside: Maintain accurate, up-to-date documentation with zero ongoing effort. Improve onboarding speed and agent accuracy by ensuring context sources reflect current reality.

• What if you shifted code review left by deploying AI agents inside your editor to catch issues during coding?

  • Move: Integrate a context-aware AI agent into your IDE (e.g., VS Code) that reviews code as you type - flagging anti-patterns, security risks, and deviations from ADRs - before a PR is even created.
  • Why Now?: PR review bottlenecks are worsening due to AI-generated code floods. Human reviewers are overwhelmed; catching issues earlier reduces rework and accelerates merge velocity.
  • Expected Upside: Cut PR review time by 50%+ and reduce post-merge bugs by catching issues at the source. Achieve faster throughput without adding headcount or increasing review burden.

• Implement a risk-based code review system that automatically merges low-risk AI-generated changes while flagging high-risk ones for human review.
• Integrate AI agents with existing collaboration tools (e.g., Slack, GitHub) to pull real-time context from code, docs, and messages without requiring manual updates.
• Shift to releasing features immediately to all users by default - using feature flags - to accelerate feedback loops, reserving staged rollouts only for high-risk changes.
• Pre-compute and index organizational knowledge (e.g., best practices, architecture decisions) into a centralized context engine to reduce AI token waste and improve output quality.
• Automate documentation updates by having AI agents create and submit pull requests for customer- and engineering-facing docs, ensuring they stay in sync with code changes.