How to govern AI agents in production: an enterprise checklist

An agent is about to email four hundred customers, and the only thing between it and them is somebody noticing. That is what running AI in production without governance looks like. The agent works, the demo went well, and nobody decided what it is allowed to do on its own.

The scale of the problem is already visible. 82% of organizations have found AI agents running in their environment that they did not know about (Cloud Security Alliance, 2026), and 97% of organizations breached through their AI apps had no AI access controls in place (IBM, Cost of a Data Breach 2025).

The short answer. Running agents safely comes down to eight controls: an inventory, a narrow identity per agent, approvals set by blast radius, a full audit trail, customer-data protection, treating outside content as untrusted, release gates, and an off switch. The checklist below maps each one to the NIST AI Risk Management Framework and the OWASP Top 10 for Agentic Applications, so you can show security and audit where each control comes from.

The checklist

1. Know every agent that is running

You cannot govern an agent you have not found. Keep one register of every agent and every tool it can call, with an owner for each. Then go looking for the ones nobody registered: agents built in someone’s AI assistant, connected with a personal token, calling production systems.

Maps to: NIST AI RMF Govern and Map. How to build it: shadow AI discovery.

2. Give each agent its own identity and only the tools its job needs

A helpdesk agent connected with an admin’s credentials, because that was the account that worked in the demo, can still delete any record in the CRM six months later. Give each agent the few tools its job needs. Have it act as the person asking, or as a narrow service account, and split read access from write access. Review access on a schedule, the way you would for a person.

Maps to: OWASP ASI02 Tool Misuse and ASI03 Identity & Privilege Abuse. How to build it: scope an agent’s tool access.

3. Decide what needs a human by blast radius

Not every action needs approval, and approving everything trains people to click yes without reading. Set the threshold by how much damage an action could do. Reading a record or drafting a reply can run on its own. Emailing every customer, issuing a refund or deleting data waits for a person. Show the approver exactly what will happen, and let a request that nobody answers expire without acting.

Maps to: OWASP ASI09 Human-Agent Trust Exploitation, which warns that confident, polished explanations can lead people to approve harmful actions. How to build it: human approval for agent actions.

4. Log every action: who asked, what ran, what came back

When security asks what an agent changed last Tuesday, “a service account called the API” is not an answer. Every action needs a record with the person behind it, the agent, the tool, what went in and what came back. Send those records to the tools your security team already searches, so agent activity sits next to everything else.

Maps to: NIST AI RMF Manage. How to build it: an MCP audit trail.

5. Protect customer data in flight and in logs

Agents read customer data to do their job, and the risk is less what they read than where it ends up. Mask personal and financial fields as data moves through each step, and before anything is written to a log or sent to a model. Keep data in the region your customers agreed to, and make sure you can answer, for any customer, what an agent saw and what it changed.

Maps to: the data privacy risk in NIST’s Generative AI Profile (AI 600-1).

6. Treat outside content as untrusted

An email, a web page or a support ticket can carry instructions written for the agent rather than the reader. Keep content the agent reads separate from the instructions it follows. Limit which tools an agent can reach after it has read untrusted input, and do not let it write to its own long-term memory unchecked.

Maps to: OWASP ASI01 Agent Goal Hijack and ASI06 Memory & Context Poisoning.

7. Promote agents through dev, staging and production

An agent is software, and it should ship like software. Build in dev, test against realistic data in staging, and promote to production only with sign-off from someone other than the builder. Keep the previous version ready to restore.

Maps to: NIST AI RMF Measure and Manage.

8. Keep an off switch, and limit what one run can do

Agents fail differently from scripts. One bad step can trigger the next, and a loop can repeat a mistake a thousand times before anyone looks. Cap how many actions a single run can take, alert on unusual volume, and make sure whoever is on call can stop an agent in one step.

Maps to: OWASP ASI08 Cascading Failures and ASI10 Rogue Agents.

Agents that run without human approval

Most agents in production will run unattended most of the time, and that is the point of them. Governing them is less about adding approvals and more about deciding, in advance, what they can do alone:

• Draw the line by blast radius. Write down which actions run alone and which wait for a person, and review the list when the agent gains a new tool.
• Cap the damage per run. A limit on records changed, emails sent or money moved turns a runaway loop into a small incident.
• Make the record complete. An unattended agent should leave a trail good enough to replay what it did and why.
• Default to not acting. When an approval times out or a check fails, the safe outcome is that nothing happens.

Agents that touch customer data

Customer data raises the bar on every control above. Three matter most. Scope each agent to the customer records its job needs, not the whole CRM. Mask sensitive fields before they reach a log or a model. And log every read as well as every write, because “what did the agent see?” is usually the first question after an incident.

Where Tray.ai fits

Tray.ai is where many teams build and run their agents, alongside the integrations those agents call, so the controls above apply to both in one place:

• One audit log. Every agent action, MCP tool call and workflow run is logged with who triggered it, what ran and what came back, and you can stream the logs to Splunk, Datadog and other SIEMs.
• One access model. The same role-based access control covers workflows, agents, MCP tools and APIs. Calls through Agent Gateway are tied to a named user and can run on their own credentials.
• Customer data protection. Built-in data masking covers PII, PHI and financial fields inside workflows, with regional hosting in the US, EU and APAC.
• Release gates. Dev, staging and production are separated by approval gates.
• Compliance. SOC 1 and SOC 2 Type 2 audited, plus HIPAA and GDPR, platform-wide.

IDC named Tray.ai a Major Player in the IDC MarketScape: Worldwide Agentic AI Platforms 2026 Vendor Assessment (#US54140926, 12 October 2026), and Gartner named it a Pioneer in the 2026 Emerging Market Quadrant for No-Code Agent Builders.

J.W. Pepper put this to work on MCP, replacing 500+ ungoverned MCP tools with about 20 governed workflows and no raw database access.

What Tray.ai does not do is model governance: bias testing, model risk registers and AI regulatory documentation. For that, dedicated AI governance tools such as IBM watsonx.governance and Credo AI go further, and most enterprises will want one alongside the controls above. See how Tray.ai handles the agent side on the governance and trust page.

Sources

• NIST, AI Risk Management Framework (AI 100-1)
• NIST, Generative AI Profile (AI 600-1), July 2024
• OWASP GenAI Security Project, Top 10 for Agentic Applications, December 2025
• Cloud Security Alliance, 2026: 82% of organizations found AI agents in their environment they did not know about
• IBM, Cost of a Data Breach Report 2025: 97% of organizations breached through AI apps had no AI access controls
• IDC MarketScape: Worldwide Agentic AI Platforms 2026 Vendor Assessment, #US54140926, 12 October 2026