Skip to content

Blog / Operations in practice

Security alert triage: the alert closed as noise

The one alert that mattered was closed as noise by an analyst four hours into the queue. Why security alert triage fails, and what to fix first.

An analyst starts the day shift with 400 alerts in the queue. Most are the same few rules firing on the same few machines, so she closes them in batches. Around noon, a rule that usually fires on a test server fires on a finance laptop instead. It looks like the others, so it goes with the batch.

Two weeks later, the incident review finds it. The detection had worked, and the alert carried no sign that this time was different.

Alert triage is the part of security operations where a person decides what each alert means. When every alert looks the same, that decision is a guess.

What is security alert triage?

Security alert triage decides which alerts need a person, in what order, and with what context. It attaches the owner, the asset, recent activity and past alerts before an analyst opens the alert. It holds back known noise with a written reason, and sends the rest to the right queue by severity.

Why the real alert gets closed

  1. Alerts arrive bare. The analyst looks up the user, the device and its history in several tools before deciding anything.
  2. Noisy rules are never tuned. A rule that fires a hundred times a day on a test server trains everyone to ignore it.
  3. Suppression leaves no record. Alerts are closed in bulk with no reason saved, so nobody can check the decision later.
  4. Severity comes from the rule alone. The same alert gets the same priority on a test server and on a finance laptop.
  5. Duplicates aren’t grouped. One event raises ten alerts across three tools, and each is triaged on its own.

What it costs

Analysts spend the day on lookups, and the queue grows faster than they can clear it. Noise teaches them to close quickly, so the alert that mattered gets the same few seconds as the rest.

What to fix first

  1. Enrich every alert before it reaches a person. Owner, asset, sign-ins and past alerts are on the alert when it opens.
  2. Set severity from the asset as well as the rule. A finance laptop or a production server raises it.
  3. Suppress with a written reason. Each suppression rule says why it exists, who approved it and when it will be reviewed.
  4. Group alerts from the same event. Make one case, with every related alert attached.
  5. Report the noisiest rules every week. Tune or retire the rules that fire most and lead nowhere.

The guide to building security alert triage has the prompts for each step.

Where it fits

Triage depends on SIEM logging delivering complete events. It feeds account takeover response and endpoint isolation, which act on the alerts that turn out to be real. The service desk faces the same problem with tickets, covered in IT ticket triage.

For every part of security operations, from first alert to shipped fix, see SOC automation.