AI-Driven Incident Response: Can Machine Learning Beat Human Intuition?

Balancing automation with analyst expertise

22-05-2025

AI-Driven Incident Response: Can Machine Learning Beat Human Intuition?

Contents

At 03:14 one night my monitoring stack lit up: a burst of failed SSH logins against a box that has no business accepting SSH from outside, followed by one success. A correlation rule had already fired, opened a ticket, and — because I’d wired it that way — pulled the host’s recent auth log into the alert. The “AI” part flagged it as a credential-stuffing pattern with 0.91 confidence. The human part (me, bleary, on the sofa) took one look and realised it was my own laptop on a flaky VPN reconnecting forty times before the password manager filled in correctly. The model was right about the pattern and wrong about the world. That gap is the whole subject of this post: machine learning is genuinely good at parts of incident response and confidently useless at others, and the engineering question is where you draw the line. <h2 id="what-the-machine-is-actually-good-at">What the machine is actually good at</h2><div class="ad-unit ad-in-article" aria-label="Advertisement"> Advertisement <ins class="adsbygoogle" style="display:block;text-align:center" data-ad-client="ca-pub-3726833845844946" data-ad-slot="3291553914" data-ad-format="auto" data-full-width-responsive="true"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </div> Strip away the marketing and “AI-driven incident response” is two unglamorous things that work. The first is anomaly detection — a model trained on what normal looks like for a given host or user flags the statistically weird: a service account that suddenly reads ten thousand files, a login from a geography the user has never touched, an outbound connection to an address nothing else talks to. The second is correlation and triage — collapsing a thousand raw events into one ranked incident, deduplicating, and enriching it with context so a human sees a story instead of a firehose. Both are speed plays. A machine reads every line of every log every second; a human reads the ones the machine surfaces. On volume alone that’s not a contest, and the value is real: the slow, mechanical first pass over the haystack is exactly the work humans are worst at and burn out doing. What makes it tractable at all is that you don’t need a giant model. A modest scoring function over enriched events — the kind of thing you can run on your own hardware, the same way I run <a href="/story/running-gemma-3-locally-googles-small-model-on-consumer-hardware/">Gemma 3 locally for other tasks</a> — gets you most of the triage win without shipping your security telemetry to someone else’s cloud. <h2 id="the-unglamorous-prerequisite-your-data">The unglamorous prerequisite: your data</h2> Before any of the clever modelling matters, the boring part decides whether it works at all: the model is only as good as the telemetry feeding it, and most “the AI doesn’t catch anything” complaints are really “we never gave it the right logs.” A useful pipeline needs a few things in place, and none of them are AI. You need normalised events from enough sources — auth logs, network flows, DNS queries, process execution, cloud audit trails — landed somewhere queryable, with consistent timestamps and a shared notion of “which host” and “which user.” If your SSH logs call a machine by its short hostname and your firewall logs call it by IP, the correlation engine can’t join them, and no model fixes a join it can’t make. You need enrichment at ingest: GeoIP, ASN, asset criticality, and whether a user is privileged, attached to each event as it arrives, because a human analyst reasons with that context instantly and the model can only use what’s in the record. And you need enough history of normal to define abnormal — a model trained on a week of data hasn’t seen your monthly batch jobs and will scream every time payroll runs. Get this plumbing right and even a simple scoring function performs well. Get it wrong and the fanciest model in the world is pattern-matching noise. I’d spend the first month on the pipeline and the boring normalisation, not on model selection — the leverage is overwhelmingly there. <h2 id="a-concrete-playbook">A concrete playbook</h2><div class="ad-unit ad-in-article" aria-label="Advertisement"> Advertisement <ins class="adsbygoogle" style="display:block;text-align:center" data-ad-client="ca-pub-3726833845844946" data-ad-slot="3291553914" data-ad-format="auto" data-full-width-responsive="true"></ins> <script>(adsbygoogle = window.adsbygoogle || []).push({});</script> </div> The unit of automation worth building is the playbook: a deterministic response that fires when a detection crosses a confidence threshold, does the cheap safe things automatically, and hands the expensive irreversible things to a human. Here’s the shape of one, trimmed to the essentials, for a suspected compromised credential: <div class="highlight"><div class="chroma"> <table class="lntable"><tr><td class="lntd"> <pre tabindex="0" class="chroma"><code> 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 </code></pre></td> <td class="lntd"> <pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"># soar-playbook: suspected-credential-compromise.yml trigger: detection: impossible-travel OR credential-stuffing min_confidence: 0.85 # things the machine may do unattended — cheap and reversible auto_actions: - enrich: { add: [geoip, asn, recent_auth_log, user_risk_score] } - notify: { channel: secops-alerts, include: enrichment } - rate_limit: { source_ip: $event.src_ip, duration: 15m } # things that require a human to click — expensive or irreversible gated_actions: - disable_account: { user: $event.user } - revoke_sessions: { user: $event.user } - isolate_host: { host: $event.host } escalate_if: - confidence: ">= 0.95" - asset_tier: "crown-jewel" </code></pre></td></tr></table> </div> </div>The dividing line in that file — <code>auto_actions</code> versus <code>gated_actions</code> — is the design. Rate-limiting an IP for fifteen minutes is reversible and low-blast-radius, so let the machine do it at 03:14. Disabling the CFO’s account during a board meeting because of a 0.86-confidence anomaly is not something you want a model doing unattended, no matter how clean the maths looks. Every action you automate, you should be able to answer: “what’s the worst case if the model is wrong, and how fast can I undo it?” <h2 id="where-the-model-confidently-misleads-you">Where the model confidently misleads you</h2> The failure mode that bites is not the model missing things — it’s the model being plausibly wrong. My 03:14 VPN incident is the benign version. The dangerous version is the inverse: a real intrusion that looks normal because the attacker moved slowly, used valid credentials, and stayed inside the statistical envelope the model considers fine. Anomaly detection is, by construction, blind to attacks that don’t look anomalous, and a patient adversary’s entire job is to not look anomalous. The other trap is alert fatigue working in reverse. Tune the model loose and you drown analysts in false positives until they reflexively dismiss everything, including the true one. Tune it tight and you miss the subtle real attack. There is no setting that escapes the trade-off; there is only choosing where on the curve you sit and revisiting it as the environment changes. A model is also only as honest as its training data — feed it six months of “normal” that quietly included an undetected breach, and it will learn to consider that breach normal. Context is the thing the machine simply does not have. It does not know that the database the alert touches holds invoices, that the maintenance window started an hour ago, or that the “suspicious” admin is the on-call engineer doing exactly what the runbook says. That business context lives in human heads and is exactly what turns a ranked list of anomalies into a prioritised response. There’s a second-order risk too: automation breeds complacency. Once a team trusts the machine to surface what matters, they stop reading the things it filters out — which is fine until the day the model is wrong about what matters. The healthiest teams I’ve seen treat the model’s output as a prioritised reading list, not a complete one, and deliberately spot-check the low-priority bucket so the muscle of human judgement doesn’t atrophy. The point of automating the haystack search is to free humans for the needles, not to convince them there are no needles the machine missed. My 03:14 VPN false positive is the cheap, funny version of this. The expensive version is a security team that’s been lulled by months of clean dashboards into assuming green means safe, and a patient attacker who has spent those same months learning to stay green. <h2 id="a-short-history-because-it-explains-the-hype-cycle">A short history, because it explains the hype cycle</h2> It helps to know that none of this is new, only newly capable. Signature-based intrusion-detection systems date to the early 1990s — Snort, the open-source workhorse most people met first, arrived in 1998 — and they worked by pattern-matching known-bad traffic against a rule set. They caught what they had a signature for and were blind to everything else, and they were noisy: a busy network could generate more alerts than any team could read, which is the original alert-fatigue problem we’re still fighting. The 2000s brought correlation engines and the first “SIEM” products that stitched logs together; the 2010s brought the machine-learning anomaly-detection wave, fuelled less by algorithmic breakthroughs than by cloud providers suddenly having enormous labelled data sets to train on. The current generation layers large language models on top for summarising incidents and drafting response steps in plain language. Each wave was sold as the one that would finally replace the analyst, and each turned out to be a better assistant rather than a replacement. That pattern — capability climbs, the “human is obsolete” pitch returns, reality settles on “force multiplier” — is worth remembering when you read the latest vendor deck. The honest read of three decades is that the machine keeps getting better at the volume problem and no better at the judgement problem. <h2 id="how-id-actually-wire-it">How I’d actually wire it</h2> Treat the model as a force multiplier on the boring half, never as the decision-maker on the consequential half. Concretely: <ol> <li>Automate enrichment and triage; gate remediation. Let the machine collapse, rank, and annotate. Let a human pull the trigger on anything that disables, isolates, or deletes.</li> <li>Pilot on low-stakes assets first. Point the automation at non-critical systems, watch its false-positive and false-negative rates for a few weeks, and only widen scope once you trust the numbers — the same caution as introducing any tool that can change state in your infrastructure.</li> <li>Measure the boring metrics. Track mean time to detect and mean time to respond before and after. If the automation isn’t moving those numbers, it’s theatre.</li> <li>Feed corrections back. Every “this was a false positive” verdict from an analyst is a training signal. Capture it deliberately rather than letting it evaporate in a chat thread.</li> <li>Keep it auditable. You want a Git-style record of what fired, what the model scored, and who approved the gated actions — the same “who changed this and why” discipline that makes <a href="/story/gitops-with-flux-letting-git-be-your-clusters-source-of-truth/">GitOps with Flux</a> worth the effort applies to security automation too.</li> </ol> <h2 id="verdict-who-should-bother">Verdict: who should bother</h2> If you’re triaging more alerts than humans can read — which is almost anyone running real infrastructure — then yes, ML-driven triage and enrichment earn their place, and the payoff shows up fast in mean-time-to-respond. The honest caveat is that it’s an assistant, not an analyst: it wins the volume game and loses the judgement game, and wiring it as though it could win both is how organisations automate themselves into a confidently-wrong incident. For a solo homelab, a handful of correlation rules and a local model doing triage is plenty and genuinely useful. For a team defending crown-jewel assets, the model belongs firmly on the enrichment side of the line, with a human’s finger on every irreversible button. Can machine learning beat human intuition? On speed, easily. On judgement, not yet — and the engineering win is in knowing which question you’re asking.

Written by Smarc

Founder and editor of vo.rs. A lifelong tinkerer who self-hosts far more than is sensible, hardens Linux boxes for fun, and prods the latest AI tools to see what they can really do. The how-to guides here are the notes Smarc wishes had existed the first time round.

Tagged#incident response #machine learning #cybersecurity

AI-Driven Incident Response: Can Machine Learning Beat Human Intuition?

Balancing automation with analyst expertise

Related Content

Cybersecurity by Design: Embedding Zero-Trust into Your Product Roadmap

Fine-Tuning vs Prompting vs RAG: Picking the Right Tool Without Wasting GPU Hours

Talking to Your Documents: A Practical RAG Pipeline with Open-Source Tools

Prompt Injection: The SQL Injection of the AI Era