Automation  ·  AI operations

How to build LLM cost and usage tracking

The model bill triples in a month and nobody can say which feature did it. Here is how attribution that answers that actually works, the prompts that build it, and what running it demands.

Built with Tray Headless

  1. System AI gateway
  2. Step Stamp attribution
  3. Step Aggregate by owner
  4. Step Rate alert
  5. System Snowflake
Also Slack

Attribution is stamped at call time rather than reconstructed from the invoice, and the alert watches the rate instead of the running total.

The short answer

What is LLM cost tracking?

LLM cost tracking breaks into four parts: attribution stamped at call time instead of reconstructed from an invoice, aggregation by team and feature instead of by model, alerting on the rate of spend instead of the running total, and a per-owner budget somebody actually owns. The common failure is reconstructing attribution afterwards. A provider invoice tells you the total and nothing about which feature caused it, and by the time it arrives the month is over.

What matters here

  • Stamp attribution when the call is made. An invoice tells you the total and nothing about the cause.
  • Attribute to a team and a feature, not a model. Nobody can act on "GPT spend rose".
  • Alert on the rate, not the total. A runaway loop is visible in an hour and invisible in a monthly figure.
  • Track cost per outcome, not per token. A feature costing more per call may be cheaper per resolved ticket.
  • Cache and measure the hit rate. It is usually the largest single reduction available and nobody measures it.

Who this is for

You run AI operations or engineering leadership. Model spend is real money now, it grows unpredictably, and the invoice is the only visibility anybody has.

How it works in practice

The sequence, from a model call to somebody being able to act on its cost.

  1. 1

    Every call is stamped at the point it is made

    Team, feature, environment, user and request id. Reconstruction later is guesswork.

  2. 2

    Tokens and cost are recorded per call

    Input, output, cached, and the model, because prices change and the record has to survive that.

  3. 3

    Spend is aggregated by owner and feature

    A team and a feature can act. A model name cannot.

  4. 4

    Alerts watch the rate of spend

    Hourly against a baseline, so a loop is caught in an hour instead of in an invoice.

  5. 5

    Cost per outcome is derived

    Per resolved ticket, per generated draft. Cost per token flatters the wrong model.

  6. 6

    Cache hit rate is reported alongside

    Usually the largest available saving, and usually unmeasured.

What cost tracking is made of

Four parts, and the first cannot be added retrospectively.

Attribution at call time

Team, feature, environment and user stamped on the request. This is the part that cannot be recovered later at any price.

Aggregation by owner

Spend rolled to teams and features, each with a named owner. A cost centre nobody owns is a cost nobody reduces.

Rate-based alerting

Spend per hour against a baseline. A runaway retry loop is obvious within an hour and invisible in a monthly total.

Cost per outcome

Per ticket resolved or draft accepted. Token cost alone makes the cheap model look better when it needs three attempts.

The Tray Headless prompts

Paste these into Claude Code or Codex with the Tray Headless plugin installed. Each stage runs on its own. The systems named in them are the worked example rather than a requirement, and every prompt says so.

Once per project, run /tray-workflows:set-workspace to pick the workspace these build in. Point it at a sandbox first.

  1. 1

    Start here: stamp attribution at the source

    This is the part that cannot be added afterwards.

    Headless skills build-workflow 

    Use build-workflow.

Every model call must carry, at the moment it is made:

  Team and feature, from a registry instead of a free text string
  Environment: production, staging or development
  The calling user or service
  A request id that ties it to the business event that caused it
  The model and its version

Reject calls with no attribution rather than accepting them into an
unknown bucket. An unattributed bucket only ever grows, and within a
quarter it is the largest line and nobody can decompose it.

None of this can be reconstructed later. The provider invoice knows the
total and nothing about why.

    Rejecting unattributed calls feels harsh in week one and is the only thing that keeps the data useful by month three.

  2. 2

    Record tokens and cost per call

    Prices change and the record has to survive that.

    Headless skills tray-patterns 

    Record for every call: input tokens, output tokens, cached tokens where
the provider reports them, the model, latency, and whether it succeeded.

Compute cost at call time from a price table you hold, and store the
computed cost alongside the price version used. Provider prices change,
and a historical cost recomputed at today's price is wrong in a way that
makes trends meaningless.

Store the cost even when a call fails. Failed calls are frequently
charged, and retry loops are made of them, which is exactly the pattern
you most need visible.
  3. 3

    Aggregate by owner, not by model

    Nobody can act on "GPT spend rose".

    Roll spend up to teams and features, each with a named owner:

  Cost per team per day and per month
  Cost per feature, ranked
  Cost per environment, because development spend on production models
  is a common and quiet leak
  Cost per user for anything user-facing, to catch the outlier

Report the top features by spend and by growth rate separately. The
fastest growing feature is usually more interesting than the largest one,
because the largest was budgeted and the fastest growing was not.

Nobody can act on model-level reporting. A team lead can act on their own
feature being third on the list.
  4. 4

    Alert on the rate, and catch the loop

    A runaway loop is visible in an hour and invisible in an invoice.

    Headless skills tray-gotchas tray-patterns 

    Use tray-gotchas, then alert on rate rather than total:

  Hourly spend more than a multiple above the same hour last week
  A single feature exceeding its daily budget before the day ends
  Any single request id generating more than a handful of calls, which
  is a retry loop
  Development environment spend above a small ceiling
  A single user above a threshold, which is either a bug or misuse

Alert to the feature owner, not to a central AI channel where it becomes
somebody else's problem.

The retry loop is the one that matters most. An agent stuck retrying a
failing call can spend a month of budget overnight, and a monthly total
shows it four weeks later.
  5. 5

    Measure cost per outcome and cache hit rate

    Per-token cost flatters the model that needs three attempts.

    Derive cost per outcome for anything with a countable outcome: cost per
resolved support ticket, per accepted draft, per completed enrichment.

That is the number worth optimising. A more expensive model that
succeeds first time is frequently cheaper per outcome than a cheap one
that needs three attempts and a human correction.

Then measure cache hit rate and the spend avoided. Caching is usually the
largest single reduction available and almost nobody measures whether
theirs is working.

Report both monthly alongside spend, so a cost conversation is about
efficiency, not only about volume.
  6. 6

    Prove it works, then hand the budgets to the owners

    Because a budget nobody owns is a cost nobody reduces.

    Run the per-step schema checks and the whole-workflow audit before this
touches production.

Then open the same workflow in Tray Build so each team can see and adjust
their own budget and alert thresholds in the visual canvas. Central
tracking with no owner produces a monthly report that everybody reads and
nobody acts on.

What it connects to

Calls happen everywhere and the cost has to land somewhere somebody owns.

Snowflake

Land every call with its attribution and computed cost. The record everything else is derived from.

Writes

Datadog

Emit spend rate and latency as metrics, so cost sits next to the performance data engineers already watch.

Writes

Slack

Alert the feature owner on a rate breach or a retry loop, rather than a central channel nobody owns.

Writes

Okta

Resolve the calling user and their team, so attribution comes from the directory rather than a string somebody typed.

Reads

Looker

Publish spend by team and feature where budget holders already look.

Writes

Same build, other stacks

The design does not change if you run something else in one of these seats. The same prompts build it against Google BigQuery, Microsoft Teams, Azure Active Directory, Power BI, Databricks or Google Chat.

Named systems are the ones most teams run, not the only ones that work. Each is an authentication in your Tray workspace, referenced by name, so the workflow never holds a credential. Where we have a connector page, the name links to it.

Running it in production

This tracks real money that grows unpredictably. The value is in catching things early.

It runs where production runs, not on a laptop

Per-call recording and hourly rate checks run continuously on the same engine, because a loop at 2am is exactly the case that matters.

Cost is stored with its price version

A historical cost recomputed at today price is wrong, and trends built on that are worse than no trend at all.

Credentials are held by the platform, never hardcoded

Provider and gateway credentials sit in your workspace, so a model provider can be added or rotated without touching the tracking.

Feature owners own their budgets

Budgets and thresholds open in Tray Build. Central tracking with no owner produces a report everybody reads and nobody acts on.

Unattributed calls are rejected

It feels harsh in week one and it is the only thing that keeps the data decomposable by month three.

Questions people ask

Why stamp attribution at call time?

Because it cannot be reconstructed. A provider invoice knows the total and nothing about which feature caused it, and by the time it arrives the month is over.

Why alert on rate, not total?

Because a runaway retry loop can spend a month of budget overnight. Hourly spend against a baseline catches it in an hour; a monthly total catches it four weeks later.

Why measure cost per outcome?

Because per-token cost flatters the cheap model that needs three attempts. A more expensive model succeeding first time is often cheaper per resolved ticket.

What should happen to unattributed calls?

Reject them. An unknown bucket only grows, and within a quarter it is the largest line on the report and nobody can decompose it.

What is the most overlooked saving?

Cache hit rate. It is usually the largest single reduction available and almost nobody measures whether their caching is actually working.

Further reading

Background on the same subject, for the case rather than the build.

Related guides

AI operations

How to expose an internal system as an MCP tool

Scope the tool to a job rather than an API, resolve identity per call, make writes idempotent, and log every invocation. The Headless prompts that build it.

AI operations

How to build human approval for agent actions

Decide what needs approving by blast radius, show the approver what will happen, expire cleanly, and keep the record. The Headless prompts that build it.

AI operations

How to build an agent observability pipeline

Capture whole agent runs instead of single calls, join each one to the outcome it produced, and alert on the failures that return an answer anyway. The prompts.

Last reviewed September 2026.