AI operations integrations and automations
Running agents and models in production, with the cost attributed, the tools scoped, and every run joined to the work it actually did.
Running a model in production is mostly not a model problem. It is knowing what an agent did, what it was allowed to do, what it cost, what it was allowed to read, and whether the answer it gave was right. The recurring design decision is what to gate on: blast radius rather than model confidence, coverage rather than a score, and the outcome rather than whether the run threw an error.
Built with Tray Headless
How to expose an internal system as an MCP tool
Scope the tool to a job rather than an API, resolve identity per call, make writes idempotent, and log every invocation.
Integration
How to build human approval for agent actions
Decide what needs approving by blast radius, show the approver what will happen, expire cleanly, and keep the record.
Automation
How to build LLM cost and usage tracking
Attribute spend to a team and a feature, alert on the rate rather than the total, and catch the loop before the invoice does.
Automation
How to build a knowledge base to vector sync
Chunk on structure, carry permissions into the index, delete on delete, and re-embed only what changed.
Integration
How to build shadow AI discovery
Find the AI tools nobody registered, from expenses, SSO and DNS, then route to a path rather than a ban.
Automation
How to build AI support deflection
Answer only what the knowledge base actually covers, escalate early with the context, and measure resolution rather than deflection.
Automation
How to build an agent observability pipeline
Capture whole agent runs instead of single calls, join each one to the outcome it produced, and alert on the failures that return an answer anyway.
Integration
How to build a document intelligence pipeline
Score confidence per field rather than per document, validate against a system of record, and route only the uncertain fields to a person.
Automation
Guides for other teams
- Revenue operations (17)
- Finance (8)
- Customer success (5)
- People operations (6)
- IT and security (7)
- Marketing (3)
- Data operations (5)
- Platform engineering (4)
- Legal and compliance (5)