Seventy-one percent of organizations say AI has access to core business systems. Only 16% govern that access effectively, according to the 2026 CISO AI Risk Report. Ask your IR team three questions: Where is your AI system inventory? What happens if a production model starts generating harmful outputs? Who has the authority to take it offline?
I’ve spent 14 years in security — energy, banking, telecom, manufacturing. Red team work, detection programs and the last several years focused on AI risk and ShadowAI. What I see consistently: Organizations have AI in production, they have an IR playbook and they think those two things are connected. They’re not.
The CISO who thinks their IR playbook covers AI incidents probably hasn’t tested it. The ones who have tested it know it doesn’t.
Two kinds of AI incident — and why that split matters more than the list
AI incidents surged 56.4% from 2023 to 2024, reaching 233 documented cases. Most IR frameworks — including NIST SP 800-61, MITRE ATLAS and the GLACIS AI Incident Response Playbook — provide you with a taxonomy of six incident types and stop there. While useful, it misses the more important split: Failures the model causes on its own, versus failures caused by a human. Your detection approach, your containment logic and your legal exposure are very different between those two groups.
Model-originated failures — degradation, bias, hallucinations — happen when the system does exactly what it was built to do, just badly. The Epic Sepsis Model, deployed across hundreds of US hospitals, had a sensitivity of only 33% at external validation. It missed two-thirds of actual sepsis cases and flooded physicians with false alerts, as a 2021 JAMA Internal Medicine study found. No one attacked it. It just quietly stopped working while every dashboard stayed green.
Externally induced failures — adversarial attacks, data poisoning, privacy breaches — happen when someone corrupts the inputs or the training environment. Tesla’s Autopilot phantom braking cases, investigated by NHTSA across hundreds of thousands of vehicles, show what adversarial input failures look like in a safety-critical system. These two groups need different primary defenses and their own playbooks.
Then there is the hybrid case, which carries the most legal exposure right now. Hallucinations are model-originated but they land in court like human errors. When Air Canada’s chatbot invented a bereavement fare policy, the airline was held liable. When a US federal court let Mobley v. Workday proceed, it accepted that an AI hiring platform could be directly liable as an ‘agent’ of the employers using it. Neither failure looked like a security incident. Both ended up as legal ones. If your legal team is not on your IR call tree, your playbook is already incomplete.
The CIA triad doesn’t cover a hallucination
The CIA triad — confidentiality, integrity, availability — does not apply to most AI incidents. When Air Canada’s chatbot made up a policy, nothing was unavailable, nothing was changed without authorization, nothing was disclosed. The framework simply doesn’t reach it. When the Epic Sepsis Model missed two-thirds of cases, there was no breach, no intrusion, no indicator of compromise. By every traditional IR metric, the system looked fine.
This is not an edge case. Classical IR frameworks assume deterministic failures with static indicators of compromise — an assumption that breaks down against probabilistic systems. Microsoft’s Security Blog said it well in April 2026: A model may produce harmful output today and something completely different from the same prompt tomorrow. The root cause is not a line of code. It is a probability distribution, and as Microsoft’s Security Blog put it, you cannot patch a probability distribution.
The numbers confirm the gap. Average AI incident detection time is 4.5 days. Sixty-seven percent of AI incidents come from model errors, not adversarial attacks — yet security budgets keep funding perimeter tools built for the latter. We are looking for the wrong signals, with the wrong tools, for the wrong failure modes.
What a mature AI IR capability looks like
I get asked this at every conference I speak at. Here is the short answer: Three things that mature teams have in place before any incident occurs.
First, an AI Bill of Materials (AIBOM) for every production system. Think of it like a software SBOM, but for AI: It documents the base model, training datasets, third-party dependencies and the full component stack. Without it, you don’t know what your AI is made of — and you can’t investigate a data poisoning incident or a supply chain compromise without that baseline. The OWASP GenAI Security Project released an open-source AIBOM generator in December 2025 that produces output in CycloneDX format aligned with SPDX standards. It is practical to implement now.
Second, a model card for every production AI system — not a document in a shared drive nobody opens, but something your IR team can pull up in the first ten minutes of a response. Training data provenance. Model version. Known performance limits, including which subpopulations showed weaker accuracy in testing. Access controls. Blast radius if it fails. Most organizations I work with have model documentation written for data scientists that no one in security can use at 2am. That is not documentation. That is liability.
Third, a named data scientist on the IR call tree. Not someone to brief after the incident — someone with authority to interrogate model behavior in real time. Traditional IR has a network engineer on call. AI IR needs the same logic applied to the people who understand how the failing system works.
A fourth thing that very few teams have: A documented rollback threshold for each deployed model. A pre-agreed definition of what anomaly rate, drift metric or fairness deviation triggers containment or a fallback switch. Teams without this spend the first hours of an AI incident debating whether what they are seeing is actually a problem. Teams with a threshold spend those hours responding.
Four things to do before the next incident
Rewrite your detection triggers. Output anomaly scoring, data distribution monitoring for drift and behavioral tracking of model API usage need to be in your detection layer. They will not come from your SIEM. This is instrumentation work at the AI system level.
Redefine containment. For most AI incidents, ‘isolate the system’ is the wrong first move. Switching to a rule-based fallback while keeping the service running may cause less harm than taking the system offline and triggering a business escalation. Each deployed model needs pre-defined rollback criteria and a named fallback. Write those down now.
Get legal in the room before the incident. Mobley v. Workday means both the AI vendor and the deploying organization can carry liability for bias incidents. Air Canada means you cannot disclaim what your AI says to a customer. If your legal team is learning about an AI incident from a press inquiry, something has already gone wrong.
Build your AI inventory and treat it like your asset register. Start with the AIBOM for your highest-risk systems — those with access to customer data, financial decisions or clinical workflows. The GenAI-IRF framework gives you a structured taxonomy for this work and the GLACIS AI Incident Response Playbook maps it to NIST SP 800-61 and MITRE ATLAS procedures your team can adapt without starting from scratch.
Forty-two percent of organizations have already had a suspicious or confirmed AI incident, and more than half say their security posture is catching up, inconsistent or reactive. Updating your playbook isn’t optional. Fix it before you need it.
This article is published as part of the Foundry Expert Contributor Network.
Want to join?
No Responses