AI agents in production: where pilots stall

The pilot went well. For six weeks the agent answered questions about the credit policy in a Slack channel, and the twelve people testing it liked it. Then the team asked it to do something: issue a customer credit under $500. It needed a NetSuite connection, so someone connected it with the admin account from the demo. It needed to know who was asking, and it had no way to check. It needed to stop for anything over $500 and tell finance when it did. Three months later, the agent still only answers questions.

Most agent pilots stall at this point, and the model is rarely the reason. A pilot proves the agent can answer. Production asks what it can change, as whom, who picks up what it can’t finish, and how anyone finds out what it did.

This post walks through the five stages of AI agent deployment, where each one breaks, and what to fix first.

The five stages, and where each one breaks

1. Ground it in company knowledge. The agent answers from current documents, and only the ones the person asking may read. It breaks when the agent quotes a policy that was retired last year. How a knowledge agent answers from your own documents.
2. Scope its tool access. Each agent gets the few tools its job needs, running with the access of the person asking. It breaks when the agent was connected with an admin account in the demo and can still change any record. Why tool design decides how agents behave.
3. Test it before every release. Real requests run against every change, scored on what the agent did as well as what it said. It breaks when one line changes in the instructions and the agent stops escalating what it used to. Why agents pass the demo and fail the week after.
4. Hand off to a person. When the agent should stop, the whole case goes to the right team and the user hears who has it. It breaks when the agent tells the user to raise a ticket, and they explain the problem again from the start. When an AI agent should stop.
5. Watch every run. Each request is traced from question to action, so a wrong answer can be found and explained. It breaks when a customer reports a wrong refund and nobody can say which step made it. How to measure agent performance.

Why pilots stall before production

The pilot runs on borrowed access. Demos use an admin connection and a hand-picked folder of documents, because that’s the fastest way to show the agent working. Production needs the agent to act as the person asking, which means identity checks the pilot never had.

Every connection is built for one agent. The first agent gets its own Salesforce connection, its own approval step and its own logging. The second agent gets new ones, and by the fifth nobody knows which of them still work.

Nobody owns the agent after launch. The team that built the pilot moves on. The instructions, the tools and the model behind it keep changing, and nobody checks whether each change made it better or worse.

The handoff is an afterthought. The pilot answered what it could and told people to raise a ticket for the rest. In production, that sentence is where users stop trusting it.

What to fix first

1. Connect the agent as the person asking. Run each tool with the user’s own permissions, so the agent can never do more than they could.
2. Give it tools built for its job. A tool that issues a credit up to a set limit behaves more predictably than open access to the ERP.
3. Build the test set before the second release. Use real requests from the pilot, including the ones it must refuse, and run them on every change.
4. Decide where it stops, and who picks up. Set the actions that need a person, route each case with its context, and tell the user who has it.
5. Log every step from the first day. Record the request, each tool call and its result, so any answer can be traced back.
6. Put the second agent on the same foundation. Reuse the connections, access checks and logging instead of building them again.

What it looks like at Zuora

Zuora is rolling out agents across the company through MCP, with each one acting with the permissions of the person using it.

“As we roll MCP out, I don't want end users thinking about which system they're authenticating to or what tools they have access to. Tray's Dynamic Authentication lets us scale agents across the enterprise without compromising on user-level permissions.”

After the first agent

The first agent is a project. By the fifth, the company has a portfolio of overlapping agents with unclear owners, and nobody is sure what each one can touch. Why the fifth agent breaks covers that stage. For governing every AI tool and agent in the company, including the ones nobody registered, see AI and MCP governance.