I recently gave a presentation at SecTor on proactive threat hunting, which sparked some meaty conversations afterward on the show floor. On the expo floor, surrounded by “AI-first” security vendors, the CISOs and threat hunters I spoke with were worried. They’re worried because AI can elevate script kiddies into elite hackers with advanced capabilities and legions of adversarial AI bots and we’re not prepared for that — at least, not yet.
While there’s no doubt AI holds great potential for cybersecurity, in practice, it’s mainly being used to automate what we’re already doing. For companies to stand a chance, we need new approaches to AI-powered defense, not optimized ones.
The asymmetry problem
Attackers already have systemic advantages that AI amplifies dramatically. While there are some great examples of how AI can be used for defense, these methods, if used against us, could be devastating. For example, XBOW is an autonomous pen-testing bot, created by a startup of the same name. It’s a security product and an impressive one at that. This summer, for the first time in bug bounty history, XBOW’s autonomous penetration tester reached the top spot on the HackerOne leaderboard for several months running.
It’s important to note that while its pen-testing bots were completely autonomous, there were still humans in the loop. Aside from HackerOne requiring human review of findings before submission, XBOW built specialized infrastructure to help it prioritize scans. Even so, its findings were impressive — including discovering a previously unknown vulnerability in Palo Alto Networks’ GlobalProtect VPN solution, affecting over 2,000 hosts. I encourage you to read up on how XBOW accomplished this, as well as how practitioners responded.
Clearly, AI can absolutely turbocharge pen testing, completing in minutes what typically takes humans hours or days. But companies still need to patch. And as we know, the more tools you have, the more alerts you get. AI can help with prioritization, but the reality is that most companies don’t have the visibility or context they need to fully automate remediation.
This has been an issue since the days of IDS vs. IPS and still persists today. It’s not just about detecting and validating threats — it’s about how fast companies can respond to them, especially in highly complex environments.
I once worked at a company with one million endpoints that spun 50,000 servers up and down daily. Every one of those 50k events had a ripple effect on the environment. At another company, we had 2,000 instances of Log4j to patch. Like many of my peers, lacking context about which instances posed the biggest threat, we started patching Internet-facing systems and then moved on to internal ones after. I’m sure that many of those instances were dead ends. But because we had no way to tell, we had to fix them all.
It’s hard to gain context at that scale. Most companies have multiple defensive layers — and they all have flaws. Using weaknesses in those layers, attackers weave through them and create attack paths. The question is: How are we finding those paths before they do?
With that in mind, think about how a bad actor could leverage a tool like XBOW. Or Hexstrike-AI — another agent-based defense tool that acts as an orchestration and abstraction layer for coordinating large numbers of specialized AI agents to launch complex operations at scale. It was co-opted by hackers to exploit three zero-day vulnerabilities affecting NetScaler ADC and NetScaler Gateway appliances within 12 hours of disclosure.
Now do you see why we’re so concerned?
The identity problem
Another huge problem with adversarial AI bots is that they are often hard to spot. Most attackers don’t break in — they log in. Credential abuse remains the primary way attackers breach their targets because humans are still susceptible to phishing and least-privilege access remains a hard problem to solve. And just when we start to make some progress, major tech innovations such as the move from on-premises to cloud-native IT add technical debt, additional complexity and new risks. It’s truly a game of whack-a-mole.
Bots further complicate identity security because humans consent to having agents act on their behalf. But agents, like humans, can be over-permissioned. That opens the door for hackers to hijack legitimate agents — ones that have consent to act on behalf of a person or company — and hijack their intent.
Furthermore, bots operate 24/7, weaving through defenses without fatigue. Defensive bots must meet them on this terrain. They need to be just as smart as adversarial bots and able to determine, in real time, whether an agent’s actions align with its intended purpose. Given that most cyber exposures are caused by human error — accidental insider threats — we also need defensive bots to save users from themselves. According to the 2025 Verizon DBIR, 60% of breaches involved human error.
The case for digital twins in security
Hopefully, by now I’ve built the case for why incremental improvements won’t cut it. I’m not saying that they aren’t helpful or welcome, I just don’t believe they’ll level the playing field. However, one area of promise I’m excited about is the use of digital twins for real-time threat modeling.
Digital twins started out as physical twins, developed by NASA to assess and simulate conditions on board Apollo 13 after its oxygen tanks exploded early in the mission. They played an essential role in troubleshooting the technical issues the astronauts 200,000 miles away were experiencing and were widely credited with helping bring them home safely.
The “twin” concept slowly evolved from physical to digital, but got a boost in 2020 when IoT devices matured enough to serve as sensor technology capable of replicating complex environments. This opened the door to their use in robotics, manufacturing and healthcare — from simulating surgeries to optimizing cancer care — and, of course, IT.
Large enterprise defense involves endless mundane tasks (patching, backups, etc.). Automation helps, but every environment change — even a positive one — can create new attack paths to critical assets that are invisible to defenders. Digital twins help teams quickly understand which attack paths are riskiest and prioritize remediation much more effectively than existing tools can.
AI bots and digital twins equal 24/7/365 attention on security
I often say that security practitioners can’t solve problems that we don’t know about, which is why proactive threat hunting is so important — if we’re not hunting for threats, we won’t find them. Certainly not as easily as XBOW showed us agentic AI bots can. Human-based threat hunting is limited by human ability, time and friction in production. The use of AI bots within a digital twin enables continuous, multi-threaded threat hunting and attack path validation without impacting production environments. This addresses the prioritization challenges that security and IT teams struggle with in a meaningful way.
Really, digital twins offer the same benefits to security teams as physical twins provided to NASA scientists more than 55 years ago: accurate simulations of how a given change might impact large, complex and highly dynamic attack surfaces. Plus, it’s exciting to imagine how the UX might evolve to help defenders visualize what’s happening in unprecedented ways.
Think big
AI is a truly transformational technology and it’s exciting to think about how AI defense can evolve over the next few years. I encourage product builders to think big. Why not draw inspiration from science fiction? From Philip K. Dick, William Gibson, Isaac Asimov and Neal Stephenson to the century-ahead works of Jules Verne, we can seed our collective imagination with insights from artists and futurists.
In the fight against malicious AI, I’m optimistic that our humanity will be our biggest asset. If we can conceptualize it, AI can help us build it.
This article is published as part of the Foundry Expert Contributor Network.
Want to join?
No Responses