Your agents are starving! Airbytes Michel Tricot on the data ingestion crisis

The podcast discusses the growing challenges of data sprawl in SaaS environments, where user identities and activities are fragmented across multiple platforms like GitHub, Jira, and Salesforce. A central theme is the difficulty of linking these disparate data points - known as entity resolution - and the need for systems that can autonomously map user actions across tools to provide meaningful context. This becomes critical as organizations increasingly rely on AI agents that require accurate, real-time, and historical data to function effectively, necessitating robust data ingestion processes and precise access controls.

A major focus is on the evolving role of AI agents in data workflows, emphasizing the balance between autonomy and oversight. Agents must operate with just-in-time, task-specific context to avoid inefficiencies, while also being able to request access to new data sources when needed. The discussion highlights the limitations of current data infrastructure, the importance of semantic layers and metadata management, and the shift from viewing data as a cost center to a strategic asset. As AI adoption grows, the conversation underscores the need for long-term fluency in data and AI practices, improved search and discovery capabilities, and scalable solutions that reduce manual intervention in both data querying and system integration.

• What if you could resolve user identities across SaaS tools to automate customer onboarding?

  • Move: Build a lightweight identity resolution layer that syncs user records from GitHub, Jira, and Salesforce using deterministic matching (e.g., email domains, activity timestamps) and logs mismatches for review.
  • Why Now?: AI agents are being deployed in customer workflows but fail due to inconsistent user mapping - fixing this now prevents downstream automation errors as agent usage scales.
  • Expected Upside: Reduce manual onboarding effort by 40% and increase accuracy of customer health scoring by linking behavior across systems, directly improving retention forecasting.

• What if you could let your AI agent request access to restricted data like a human would?

  • Move: Implement an agent permission gateway with an "ask-for-access" protocol - when an agent hits a data wall (e.g., no Salesforce access), it logs a request with context (task, data needed, urgency) for approval or auto-provisioning based on rules.
  • Why Now?: Broad-scoped agents are risky, but overly restricted ones fail silently - this balances autonomy and control just as teams are adopting agentic workflows at scale.
  • Expected Upside: Cut agent failure rates by 50% while maintaining security, enabling reliable automation of cross-system tasks like support ticket escalation or renewal forecasting.

• What if you built real-time ingestion pipelines for only the data your agent needs - just in time?

  • Move: Replace full historical syncs with event-triggered micro-ingestion (e.g., "when a new Figma comment mentions 'launch,' pull related Jira ticket and CI/CD status") using webhook-driven Lambda functions.
  • Why Now?: The ingestion crisis is peaking - developers waste 60%+ of agent runtime waiting for or filtering irrelevant data; lean ingestion unlocks speed and cost efficiency today.
  • Expected Upside: Reduce data processing costs by 70% and cut agent response latency from minutes to seconds, enabling real-time use cases like automated QA validation or release approvals.

• Implement a lightweight identity resolution system to map users across SaaS tools (e.g., GitHub, Jira, Salesforce) using deterministic identifiers like email or ID hashes, enabling accurate user lifecycle tracking.
• Design agent permissions using a "broad first, then narrow" approach: grant temporary wide access during development to identify required data paths, then enforce strict, task-specific scopes via MCP-like schemas or gateways.
• Build or integrate a metadata registry that catalogs available data sources, access policies, and ETL pipelines, allowing agents and developers to discover and validate data sources without direct access.
• Prioritize real-time ingestion from critical SaaS APIs alongside historical data syncs to support agent workflows that require up-to-date context (e.g., CI/CD status, ticket updates) for accurate decision-making.
• Develop lean, task-specific context retrieval logic for agents - only fetch data relevant to the current operation (e.g., related Jira tickets, Zendesk records) to avoid bloating and maintain precision.