{"id":10095,"date":"2026-10-21T01:09:32","date_gmt":"2026-10-21T01:09:32","guid":{"rendered":"https:\/\/cybersecurityinfocus.com\/?p=10095"},"modified":"2026-10-21T01:09:32","modified_gmt":"2026-10-21T01:09:32","slug":"uk-fca-exec-wants-to-require-disclosure-of-significant-ai-failures-and-near-misses","status":"publish","type":"post","link":"https:\/\/cybersecurityinfocus.com\/?p=10095","title":{"rendered":"UK FCA exec wants to require disclosure of \u2018significant AI failures and near-misses\u2019"},"content":{"rendered":"
\n
\n
\n
\n
<\/div>\n

When UK Financial Conduct Authority (FCA) Chief Data Officer Jessica Rusu gave a speech on Monday advocating that UK companies be required to report \u201csignificant AI failures and near-misses,\u201d she added to the drumbeat of demands for some kind of regulatory control over enterprise AI deployments.\u00a0<\/p>\n

But her speech also highlighted regulator frustration over futile attempts to slow down the seemingly unstoppable agentic and genAI rollouts, as reports of agent misbehavior and the lack of meaningful consequences<\/a> continue to emerge with alarming frequency.<\/p>\n

Rusu<\/a>, speaking to a Bloomberg UK Policy Series audience, said that firms need to invest more in cyber and operational resilience.<\/p>\n

\u201cOur rules already require firms to report major incidents, and that must extend to significant AI failures and near-misses. As the AI Minister has recognized, near-miss reporting for agentic AI is becoming a national security priority. It is a regulatory priority too,\u201d Rusu said<\/a>. \u201cIt also means thinking differently about assurance. A system that is safe at deployment may behave differently at scale, under stress, or as the models beneath it change. Agentic systems therefore require continuous, conditional assurance.\u201d<\/p>\n

She added, \u201cthe FCA will work with industry, fellow regulators, and frontier labs to develop a trusted framework for agentic finance, including clear approaches to agent identity, delegated authority, accountability and execution.\u201d<\/p>\n

One of the questions that came out of the speech is what precisely was meant by \u201csignificant AI failures and near-misses.\u201d\u00a0Questions to FCA to clarify their terminology were not answered by publication time.<\/p>\n

AI failure vs near miss<\/h2>\n

But Pieter Arntz<\/a>, a malware intelligence researcher at security firm Malwarebytes, offered his own take on what those terms likely mean.<\/p>\n

\u201cA significant AI failure is one with serious consequences, not simply an incorrect answer. In financial services, that could mean an AI system authorizing substantial fraudulent payments, exposing sensitive customer information, denying customers access to essential financial services, or triggering transactions that threaten market integrity,\u201d Arntz said. \u201cThe test should be the scale and severity of the impact, rather than whether the failure looks technically dramatic.\u201d <\/p>\n

On the other hand, he added, \u201cAn AI near-miss is an event that could realistically have caused serious harm, but was caught before that harm occurred. For example, an AI agent might attempt an unauthorized transfer that a separate approval control blocks, or a prompt-injection attack might induce it to disclose sensitive information, only for an outbound security control to stop the disclosure.\u201d<\/p>\n

However, Arntz warned of the risk to Rusu\u2019s approach, should the final penalties be underwhelming. \u201cThere is a risk of this becoming a paper tiger if compliance amounts to filing a report and absorbing a predictable fine,\u201d he said.\u00a0<\/p>\n

Warnings from multiple directions<\/h2>\n

UK-based Gartner VP analyst Nader Henein<\/a> said he is skeptical about the significance of the FCA\u2019s proposal, in part because so many UK enterprises already have to comply with EU regulations, which have started asking companies to make such disclosures.<\/p>\n

\u201cFinancial watchdogs including the Bank of England (BoE), the UAE Central Bank, and the Monetary Authority of Singapore have all issued warnings,\u201d he said. \u201cA few months ago, the European central banks, together with the European Systemic Risk Board, escalated the cyber risk assessment level associated with frontier models to \u2018severe,\u2019 warning that simultaneous cyber incidents could impact the financial sector. They acknowledged that AI-driven cyber incidents now threaten core financial sector stability.\u201d<\/p>\n

Henein said that he is especially skeptical that meaningful changes will occur in the financial sector.<\/p>\n

\u201cFinancial institutions tend to heed the warnings of the regulator that decides whether they can continue to operate, but without clear laws, regulatory oversight, and the obligatory fines, these disjointed statements are not likely to make a difference,\u201d he said.\u00a0<\/p>\n

Could \u2018incentivize the wrong behavior\u2019<\/h2>\n

Noah Kenney<\/a>, principal consultant at Digital 520, added that one risk of this increased visibility requirement is that it could inadvertently incentivize the wrong behavior.\u00a0<\/p>\n

\u201cA bank with good monitoring may catch hundreds of attempts by AI agents to act outside their authority,\u201d he said. \u201cA bank with poor monitoring may report zero, because it never saw them. On paper, the safer bank looks worse. A model failing a red-team test is the test working, not a regulatory incident.\u201d<\/p>\n

However, Frank Dickson<\/a>, principal analyst at Dickson Research, said that the near-miss approach has the strongest potential to change the conversation.<\/p>\n

\u201cI love the near-miss idea. It is the most important thing in the speech. It is the part financial firms are least ready for,\u201d he said.\u00a0<\/p>\n

But, he stressed, \u201c[Rusu] also warns about reliance on a single model. An outage at one AI supplier that takes down a customer service channel counts, too.\u201d<\/p>\n

Another concern about reliance on a specific model is the ever-present risk that an AI vendor could change the model without telling all of its customers.\u00a0<\/p>\n

\u201cA model can pass testing in March and misbehave in May without anyone at the firm changing a line of code, because the provider changed the model underneath,\u201d Dickson said. \u201cA model version change should get the same scrutiny as any other production change.\u201d<\/p>\n

This article originally appeared on CIO.com<\/a>.<\/em><\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>","protected":false},"excerpt":{"rendered":"

When UK Financial Conduct Authority (FCA) Chief Data Officer Jessica Rusu gave a speech on Monday advocating that UK companies be required to report \u201csignificant AI failures and near-misses,\u201d she added to the drumbeat of demands for some kind of regulatory control over enterprise AI deployments.\u00a0 But her speech also highlighted regulator frustration over futile […]<\/p>\n","protected":false},"author":0,"featured_media":10096,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-10095","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-education"],"_links":{"self":[{"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/posts\/10095"}],"collection":[{"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=10095"}],"version-history":[{"count":0,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/posts\/10095\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/media\/10096"}],"wp:attachment":[{"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=10095"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=10095"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=10095"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}