How to build incident to resolution

What matters here

• Make declaring cheap. If declaring feels like a big deal people wait, and the waiting is the expensive part.
• Build the timeline from events as they happen. Reconstructing it later produces a story, and the times always flatter.
• Separate detect, acknowledge, mitigate and resolve. One duration hides which part is actually slow.
• Update customers on a cadence, even when there is nothing new. Silence is read as nothing being done.
• Track action items to closure. A postmortem whose actions are never done is a document, not a control.

Who this is for

You run engineering operations, SRE or IT. Incidents are handled well and recorded badly, and the postmortem is written from Slack scrollback by whoever has time.

How it works in practice

From something breaking to the record being trustworthy, in order.

1. 1

   Declaring is one action from anywhere

   A command in chat or a click from an alert. Anything heavier and people wait to see if it recovers.

2. 2

   The channel, roles and record are created together

   Incident commander, comms lead and scribe assigned from the rota, not volunteered for mid-incident.

3. 3

   The timeline builds itself from events

   Alerts, deploys, status changes and key messages, timestamped as they happen.

4. 4

   Customer updates run on a cadence

   Every thirty minutes for a severity one, whether or not there is news, because silence reads as inaction.

5. 5

   Resolution captures four separate times

   Detected, acknowledged, mitigated, resolved. One number hides which part was slow.

6. 6

   The postmortem is created automatically with the timeline in it

   Draft populated, owner assigned, due date set, and actions tracked to closure.

Paste these into Claude Code or Codex with the Tray Headless plugin installed. Each stage runs on its own. The systems named in them are the worked example rather than a requirement, and every prompt says so.

Once per project, run /tray-workflows:set-workspace to pick the workspace these build in. Point it at a sandbox first.

1. 1

   First, find the event sources

   The timeline is only as good as the events feeding it.

   Headless skills build-workflow

   Copy

   Use build-workflow. The systems in play are PagerDuty, Datadog, Slack,
   Statuspage, Jira, Zendesk and GitHub, or whatever we run in those
   seats. Before you plan anything, tell me which of them are already
   authenticated in the workspace, because I do not want a connector
   stubbed that I have not authenticated. I expect PagerDuty, Datadog,
   Slack, Jira and Statuspage.
   
   What I need from each of them: how to read alert fire and acknowledge
   times, how to read deploy events, how to create and update a status
   page incident, and how the on-call rota is queried.
   
   The rota matters most. Roles assigned from a rota happen; roles
   assigned by asking during an incident do not.

2. 2

   Make declaring one action

   Every gate you add is time spent deciding whether this counts.

   Headless skills build-workflow

   Copy

   Use build-workflow. Declaring should be a single action: a slash command
   in chat, or one click from an alert.
   
   Ask for exactly two things: a one-line description and a severity. Nothing
   else. Everything else is derived or filled in later.
   
   On declare, in one step:
   
     Create the incident channel, named consistently so it is findable
     Assign incident commander, comms lead and scribe from the on-call rota
     Open the bridge and put the link in the channel topic
     Create the incident record with a stable id
     For severity one and two, open a status page incident in investigating
   
   Do not ask who wants to be commander. Assign from the rota and let people
   swap. Asking wastes the first ten minutes, which are the ones that
   matter.

   Severity can be corrected later and usually is. Making people get it right at declaration time is the most common reason declaring gets delayed.

3. 3

   Build the timeline from events, not memory

   A reconstructed timeline is a story, and the times always flatter.

   Copy

   Assemble the timeline continuously, as things happen:
   
     Alerts firing, with their original timestamps
     Who was paged and when they acknowledged
     Deploys and feature flag changes in the window, which is the first
     question anybody asks
     Status page updates
     Messages in the channel marked as significant, by reaction or command
     Severity changes, with who changed it and why
   
   Never rely on somebody scrolling back afterwards. A week later the
   scrollback is incomplete, the timestamps get rounded, and the version
   that gets written is the one everybody remembers instead of the one that
   happened.

4. 4

   Update customers on a cadence

   Silence is read as nothing being done.

   Headless skills tray-gotchas tray-patterns

   Copy

   Use tray-gotchas, then run customer communications on a timer:
   
     Severity one: every 30 minutes
     Severity two: every hour
     Regardless of whether there is news. "Still investigating, next update
     at 14:30" is an update.
   
   Prompt the comms lead when an update is due and post it to the status
   page. Do not auto-publish text nobody approved, and do not let the timer
   lapse silently.
   
   Also handle: linking affected support tickets so the volume is visible,
   suppressing routine automated customer email during a severity one, and
   recording exactly what was told to customers and when, because that
   becomes the contractual record.

5. 5

   Capture four times, and make the postmortem unavoidable

   One duration hides which part was actually slow.

   Copy

   On resolution, capture detected, acknowledged, mitigated and resolved as
   four separate timestamps. Time to acknowledge and time to mitigate are
   different problems with different fixes, and a single duration hides
   which one you have.
   
   Then create the postmortem automatically for anything at severity two or
   above: the draft with the timeline already in it, an owner, and a due
   date. Do not wait for somebody to volunteer.
   
   Track action items in the tracker the team already uses, with owners and
   dates, and report on them until they close. An overdue postmortem action
   from an incident three months ago is the most useful number in this whole
   workflow, and it is the one nobody produces.

6. 6

   Test it, then hand the severity matrix over

   Because what counts as a severity one is an operational judgement.

   Copy

   Run the per-step schema checks and the whole-workflow audit before this
   touches production, and run a game day against it before trusting it in
   anger.
   
   Then open the same workflow in Tray Build so engineering operations can
   adjust the severity matrix, the update cadences and the role assignments
   in the visual canvas.

What it connects to

Incidents touch monitoring, people, customers and the record, all at once.

Named systems are the ones most teams run, not the only ones that work. Each is an authentication in your Tray workspace, referenced by name, so the workflow never holds a credential. Where we have a connector page, the name links to it.

Questions people ask

Further reading

Background on the same subject, for the case rather than the build.

• MCP governance in the enterprise

Related guides