How to build AI support deflection

What matters here

• Never answer from the model's own knowledge. If it is not in the knowledge base, the honest answer is that a person will help.
• Escalate on thin coverage rather than guessing. A confident wrong answer costs more than the ticket it prevented.
• Carry the whole attempted conversation into the escalation. A customer repeating themselves is worse than never having tried.
• Measure resolution, not deflection. Deflection counts the people who gave up.
• Never let it answer on billing, security or contractual questions. Those escalate regardless of confidence.

Who this is for

You run support operations. Volume is rising, the knowledge base is decent, and the concern is an assistant confidently telling a customer something untrue.

How it works in practice

What happens between a customer asking and either being helped or reaching a person.

1. 1

   The question is classified and checked against topic rules

   Billing, security and contractual questions escalate immediately regardless of confidence.

2. 2

   Content is retrieved before anything is generated

   If retrieval returns nothing relevant, that is the answer: escalate.

3. 3

   Coverage is assessed, not only confidence

   How well the retrieved content actually addresses this question.

4. 4

   An answer cites its sources

   So the customer can check it and the agent can see what was said.

5. 5

   Escalation carries the whole attempt

   What was asked, what was answered, what was retrieved, and why it escalated.

6. 6

   Outcome is recorded against resolution, not deflection

   Did the customer come back, and were they satisfied.

Paste these into Claude Code or Codex with the Tray Headless plugin installed. Each stage runs on its own. The systems named in them are the worked example rather than a requirement, and every prompt says so.

Once per project, run /tray-workflows:set-workspace to pick the workspace these build in. Point it at a sandbox first.

1. 1

   First, decide what it must never answer

   The restricted list is the first control, before any retrieval.

   Headless skills build-workflow

   Copy

   Use build-workflow. The systems in play are Zendesk and our knowledge
   base, or whatever we run in those seats.
   
   Before anything else, define what the assistant must never answer
   regardless of confidence:
   
     Billing amounts, refunds and credits
     Contractual terms, entitlements and SLA claims
     Security questions and anything about an incident
     Anything about an account other than the requester's own
     Legal, compliance or data protection questions
   
   Those escalate immediately. Being wrong about a bug is embarrassing.
   Being wrong about a refund or an SLA is a commercial commitment somebody
   has to honour or retract.

2. 2

   Answer only from what you retrieved

   A model answering from its own knowledge will be confidently wrong.

   Headless skills build-workflow

   Copy

   Use build-workflow. Retrieve before generating, and generate only from
   what came back.
   
   Instruct explicitly: answer only from the provided content. If the
   content does not address the question, say so and escalate. Do not infer,
   do not generalise from similar products, and do not fill a gap.
   
   Cite the source documents in the answer with links, so the customer can
   verify and the agent can see what the assistant was working from.
   
   A model answering about your product from its training data will be
   plausible and wrong, and the customer will not be able to tell the
   difference until it costs them something.

3. 3

   Escalate on coverage, not just confidence

   Coverage and confidence are different, and coverage is the useful one.

   Copy

   Assess whether the retrieved content actually addresses the question,
   separately from how confident the generation is.
   
   Escalate when:
   
     Retrieval returned nothing above a relevance threshold
     The content is adjacent but does not answer the specific question
     The question spans several documents that disagree
     The customer has already asked once and the answer did not help
     Sentiment suggests frustration, which should escalate regardless
   
   Two exchanges is the limit. A customer on a third attempt with an
   assistant is a customer writing a complaint about the assistant, and no
   deflection statistic is worth that.

   Coverage is the better signal because a model can be confident about content that does not answer the question. Checking the retrieval is checking the right thing.

4. 4

   Hand over everything on escalation

   A customer repeating themselves is worse than never having tried.

   Headless skills tray-gotchas

   Copy

   Use tray-gotchas, then make the handover complete.
   
   The agent receives: the full conversation, what the assistant answered,
   the documents it retrieved, why it escalated, and the account context.
   
   The customer must never be asked to explain again. Being handed to a
   person who starts with "how can I help" after five minutes with an
   assistant is the single worst outcome of the whole design, and it is more
   annoying than never offering the assistant at all.
   
   Set expectations honestly in the handover: tell the customer a person is
   taking over and roughly when. Do not let an escalated conversation sit
   silently in a queue looking to the customer exactly like the assistant
   still thinking.

5. 5

   Measure resolution, never deflection

   Deflection counts the people who gave up.

   Copy

   Record an outcome per conversation: resolved by the assistant with no
   follow-up, escalated, or abandoned.
   
   Then report:
   
     Resolution rate, meaning the customer did not come back within 72 hours
     Escalation rate and the reasons, which shows where the knowledge base
     is thin
     Abandonment, which deflection rate silently counts as a success
     Satisfaction split between assistant-resolved and agent-resolved
     Questions where retrieval returned nothing, which is the content
     backlog and the most valuable output of the whole thing
   
   Never report deflection rate as the headline. It rises when customers
   give up, and optimising it makes the experience worse while the number
   improves.

6. 6

   Prove it works, then hand the topic rules over

   Because what is safe to answer changes as the product does.

   Copy

   Run the per-step schema checks and the whole-workflow audit before this
   touches production. Run it shadow first: generate answers, show nobody,
   and have support read a week of them against what agents actually
   replied.
   
   Then open the same workflow in Tray Build so support operations can
   change the restricted topics, the coverage thresholds and the escalation
   rules in the visual canvas.

What it connects to

The question arrives in the support tool and the answer has to come from the documentation.

Named systems are the ones most teams run, not the only ones that work. Each is an authentication in your Tray workspace, referenced by name, so the workflow never holds a credential. Where we have a connector page, the name links to it.

Questions people ask

Related guides