Skip to content

Blog / Agent development and automation

Best AI agent observability tools (2026)

Eight AI agent observability tools compared: tracing, evals, and run records, with hosting and published pricing, and where each fits.

The best AI agent observability tools split by where they look. Langfuse, Arize Phoenix, and Opik are open-source tracing and evaluation tools you can self-host. LangSmith and Braintrust lead with evaluations. Datadog and Splunk add agent observability to monitoring platforms many teams already run. Tray records what each agent run changed in your business systems and streams those logs into the monitoring tools above.

The full picture. AI agent deployment, end to end covers every stage, the systems it runs on and how to build each one.

An agent makes nine tool calls, retries twice, and returns a confident wrong answer. Traced one call at a time, every step looks healthy. Observability for agents has to follow the whole run, score whether it did the right thing, and connect it to what it changed.

This list sorts eight tools by which of those jobs each does best. Most teams pair a tracing or eval tool with the monitoring platform they already run. For governance beyond observability, see the wider AI governance tools market.

AI agent observability tools at a glance

Product What it records Hosting Pricing Best for
Agent platforms with run records
Tray Each agent action, tool call, and workflow run, with who triggered it Managed cloud; regional hosting on Enterprise Add-on · Tray starts at $50 a month A record of what agents changed in business systems
Monitoring platforms with agent observability
Datadog Agent traces next to APM, logs, and infrastructure SaaS From $160 a month for the first 100K LLM spans Teams already on Datadog
Splunk Agent Observability Traces, evaluations, and guardrails on all traffic Observability Cloud or self-hosted From $100 a month per 1.2 million spans, billed annually Splunk shops that want evals and guardrails in one place
Open-source tracing and evaluation
Langfuse Traces, evals, cost, and prompts for every AI call Cloud or self-hosted, free Free tier; Core from $29 a month Teams that want open source with a managed option
Arize Phoenix and AX Traces and evals, built on OpenTelemetry Phoenix runs locally or self-hosted; AX in the cloud Phoenix is free; AX Pro from $50 a month OpenTelemetry-first teams
Opik Agent traces, evals, and guardrails Self-hosted open source, or Comet cloud Free; Pro Cloud from $19 a month The lowest-cost managed start
Evaluation-led platforms
LangSmith Traces, online evals, and monitoring for any agent stack Cloud; hybrid or self-hosted on Enterprise Free for one seat; Plus at $39 a seat a month Teams building with LangChain or LangGraph
Braintrust Traces turned into evals that gate releases SaaS; on-premises on Enterprise Free tier; Pro at $249 a month Teams that block releases on eval scores

Pricing, hosting, and capabilities checked against each vendor’s own pages on 17 November 2026. Helicone and Portkey, which appear on other lists, were left off: both changed ownership in 2026 and their products are in transition.

Agent platforms with run records

Tray

Best for: teams that need a record of what each agent run changed in business systems, delivered to the monitoring tools they already use.

Tray records every agent action, MCP tool call, and workflow run with who triggered it, what ran, and what came back. It keeps step-level execution logs, records platform changes as audit events, shows trends in its Insights Hub, and can mask sensitive values. Logs stream to Datadog, Splunk, or another SIEM, so Tray’s run records sit next to the traces from the tools below. Teams building their own pipeline can follow how to build an agent observability pipeline, which joins each run to the business outcome it produced.

Trade-offs: Tray does not score output quality with evaluations or build test datasets; pair it with an eval tool from this list. It records agents that run on Tray or call tools through it. Log retention and streaming depend on your plan, as what an integration platform should record explains.

Monitoring platforms with agent observability

Datadog

Best for: teams already running Datadog for APM and logs.

Datadog Agent Observability traces agents with the same tracer used for the rest of the stack, so one trace runs from a backend service through the agent’s reasoning to the end user. Teams can evaluate agents in production, build golden datasets from annotated traces, and test changes against them before shipping. Role-based access, sensitive data protection, and HIPAA compliance come with the platform.

Trade-offs: pricing is per LLM span, starting at $160 a month for the first 100K spans, on top of the Datadog products it connects to. The case is strongest when the rest of your telemetry is already there.

Splunk Agent Observability

Best for: Splunk customers who want tracing, evaluation, and runtime guardrails in one product.

Cisco completed its acquisition of Galileo in May 2026, and Galileo is now Splunk Agent Observability. It traces agent workflows from request to response and shows quality and security signals, such as hallucination, prompt injection, and PII leakage, next to latency and errors. Its own evaluation models score every interaction rather than a sample, and guardrails act on live traffic.

Trade-offs: the product is being folded into Splunk Observability Cloud, so expect packaging to keep changing. Pricing starts at $100 a month per 1.2 million spans, billed annually.

Open-source tracing and evaluation

Langfuse

Best for: teams that want an open-source platform with a managed option.

Langfuse traces every AI call and agent run, with evaluations and cost tracking in the same place. Teams instrument with its SDKs or any OpenTelemetry exporter, or route model calls through its AI gateway without changing code, and can export the data to a warehouse. Self-hosting is free; Langfuse Cloud starts free and Core costs $29 a month.

Trade-offs: self-hosting means running and upgrading it yourself. On Langfuse Cloud, enterprise SSO comes with a $300-a-month add-on.

Arize Phoenix and Arize AX

Best for: teams standardizing on OpenTelemetry who want to start free and local.

Phoenix is Arize’s open-source tool for tracing and evaluating agents. It runs on a laptop or self-hosted, built on OpenTelemetry instrumentation. Arize AX is the managed platform, adding agent trajectory views, custom dashboards and monitors, and unlimited online evals, with AX Pro at $50 a month.

Trade-offs: Phoenix and AX are separate products, so moving from one to the other is a migration. Self-hosted AX, enterprise SSO, and audit logs sit in the top tier.

Opik

Best for: teams that want the cheapest managed starting point.

Opik, from Comet, logs every step an agent takes, from retrieval to tool calls, with LLM-as-a-judge evaluations, prompt management, and production monitoring. The open-source version is free to self-host. Comet’s cloud has a free plan for up to 25K spans a month, and Pro Cloud costs $19 a month for 100K spans.

Trade-offs: Opik’s guardrails for PII and topics are only available in self-hosted deployments.

Evaluation-led platforms

LangSmith

Best for: teams building with LangChain or LangGraph, or any stack that wants tracing and online evals together.

LangSmith traces agents step by step through native framework support and SDKs for Python, TypeScript, Go, and Java, with OpenTelemetry. Monitoring scores production quality with online evaluations. The Developer plan is free for one seat with 5,000 base traces a month; Plus costs $39 a seat a month plus usage.

Trade-offs: pricing combines seats and usage. Hybrid and self-hosted deployment come only with Enterprise.

Braintrust

Best for: teams that gate releases on evaluation scores.

Braintrust records prompts, responses, and tool calls in real time, scores outputs with LLMs, code, or human review, and turns production patterns into evaluations that can block a bad release. The Starter plan is free; Pro costs $249 a month.

Trade-offs: pricing combines a platform fee, processed data, and model credits, so estimate your volume before choosing. On-premises deployment comes with Enterprise.

How to choose

  • Your telemetry already lives in Datadog or Splunk. Start with their agent observability, so agent traces sit next to everything else.
  • You want open source you can self-host. Look at Langfuse, Arize Phoenix, or Opik.
  • Releases should stop when quality drops. Braintrust and LangSmith are built around evaluations that gate a release.
  • You need to know what agents changed, not just what they said. Tray records each run’s actions in your business systems and streams them to your monitoring tool. For the cost side, see tracking LLM cost and usage per run.
  • You are deciding what to measure first. See how to measure AI agent performance before choosing a tool.

Agents that reach business systems through MCP need their own audit trail; monitoring and auditing MCP traffic covers what to record.

Sources

Every product claim was read on 17 November 2026 on the vendor’s own pages:

Frequently asked questions

What is AI agent observability?

+

AI agent observability records what an agent did across a whole run: its inputs, each model and tool call, retries, and the final output. Good observability also scores whether the run did the right thing and links it to what it changed, so a confident wrong answer is as visible as an error.

How is agent observability different from LLM observability?

+

LLM observability tracks individual model calls: latency, tokens, cost, and output quality. Agent observability follows the whole run, because an agent can make many healthy calls that add up to a wrong result. It also records tool calls and the actions an agent took in other systems.

What is the difference between observability and evaluation?

+

Observability records what happened in production. Evaluation scores whether it was right, using automated judges, code checks, or human review. Most tools on this list do both, and evaluation-led tools such as Braintrust turn production traces into tests that run before each release.

How much do AI agent observability tools cost?

+

Several tools start free, including Langfuse, Arize Phoenix, Opik, LangSmith, and Braintrust. Paid plans start at $19 a month for Opik Pro Cloud and $29 a month for Langfuse Core. Datadog charges per LLM span from $160 a month for the first 100,000, and Splunk Agent Observability starts at $100 a month per 1.2 million spans, billed annually.

Do I need a separate tool if I already use Datadog or Splunk?

+

Both now include agent observability, so many teams can start there. A separate tool adds value when you want open-source self-hosting, evaluation workflows that gate releases, or a record of what agents changed in business systems.