AI agents fail because teams skip this step

This article was originally published by Rich Waldron, Tray.ai co-founder and CEO, on LinkedIn.

A recent MIT report found that 95% of enterprise AI pilots fail. That stat is getting a ton of attention. But the real story isn’t that building AI agents is hard. It’s why so many of these projects fail. Here's what the MIT report won't tell you: Most teams are skipping the foundational decisions that determine long-term success.

In my last article, I wrote about how many teams get stuck after launching their first agent. Agent #1 gets built, but agents 2, 3, 4, and so on tend to stall out. And the root problem is usually the same: Architecture.

If you’re not making the right choices early on about orchestration, about where memory and knowledge live, about how agents get triggered and take action, then you're just building smarter silos.

What Gartner® got right about agent architecture

That’s why I found Gartner’s recent report, How to Choose the Right Architecture to Build AI Agents, to be especially timely. It offers one of the clearest breakdowns I’ve seen of the architecture patterns available to enterprise teams, and the trade-offs that come with each.

Gartner outlines four reference architectures for AI agents:

1. Tightly coupled in-app agent

2. Adjacent agent attached to an app

3. Centralized orchestration with task-specific agents

4. Composable, event-driven platform

The first two are fine for demos. They’re fast to build and often bundled into your existing SaaS stack. But they don’t scale. They rely on narrow context, have limited access to enterprise data, and force you to duplicate governance and policy logic every time you build something new.

If you’re serious about deploying agents that operate across workflows, departments, and systems, look at architectures 3 and 4.

In particular, the fourth architecture, a composable, event-driven platform, is what we’ve seen deliver the most long-term value for enterprises. This model helps you scale beyond the first agent and build an ecosystem of agents that can be reused, governed, audited, and improved over time.

Gartner calls this out directly in the report:

“As the number of agents increases, organizations need a sustainable strategy to integrate them into operations, connecting the agents to existing applications, processes, and data sources.”

A strategy for AI survivability

Too many teams are building one-off agents inside disconnected tools, hoping it will all come together later. But that’s how you end up in the 95%. Agents need shared memory. Shared grounding. Shared policies. Shared infrastructure.

That’s the tipping point the MIT study alludes to: when teams stop building AI experiments and start building AI systems.

And the good news is you don’t need to reinvent your stack to get there. A composable platform approach lets you connect best-of-breed tools, while still maintaining visibility and control over how agents are built, deployed, and scaled.

Because if AI agents are going to change how work gets done, you’ll need more than just a smart brain. You’ll need the body to match.

Grab the Gartner report.

GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.

Source: Gartner, Inc. How to Choose the Right Architecture to Build AI Agents, Tigran Egiazarov, Gary Olliffe, Adrian Leow, Jim Scheibmeir, Tom Coshow, Arun Batchu, 5 June 2025