{"id":3512,"date":"2025-06-11T01:28:12","date_gmt":"2025-06-11T01:28:12","guid":{"rendered":"https:\/\/cybersecurityinfocus.com\/?p=3512"},"modified":"2025-06-11T01:28:12","modified_gmt":"2025-06-11T01:28:12","slug":"is-attacker-laziness-enabled-by-genai-shortcuts-making-them-easier-to-catch","status":"publish","type":"post","link":"https:\/\/cybersecurityinfocus.com\/?p=3512","title":{"rendered":"Is attacker laziness enabled by genAI shortcuts making them easier to catch?"},"content":{"rendered":"
\n
\n
\n
\n
<\/div>\n

OpenAI\u2019s recent report detailing various defenses it has deployed to fight fraudsters, especially those leveraging its LLM to impersonate people on social media, has met with mixed reactions from experts.<\/p>\n

One prominent analyst group, Gartner, sees it as more of a PR stunt than actually making a case that OpenAI is delivering a cybersecurity differentiator.\u00a0<\/p>\n

\u201cOpenAI\u2019s current measures (for example, banning accounts, monitoring, collaborating) to help people defend against cyberattacks are reactive and very limited,\u201d said Avivah Litan, a Gartner distinguished VP analyst currently focusing on AI strategies. \u201cThey don\u2019t directly and fully address user needs, and come off as a PR marketing move to act like they do.\u201d<\/p>\n

Another analyst, though, saw something very positive in the report.\u00a0<\/p>\n

\u2018Flips the usual script\u2019<\/h2>\n

Jeremy Roberts, senior director of research at Info-Tech Research Group, said he thought the report was interesting because it illustrated how the nature of genAI made it so much easier to catch the crooks.<\/p>\n

\u201cOpenAI\u2019s threat intel dump flips the usual script. Yes, it shows the expected laundry list of abuses, but the interesting headline is how often the attackers\u2019 use of ChatGPT made them easier to catch,\u201d Roberts said. \u201cBecause the threat actors kept plugging entire workflows into the model, OpenAI could see everything from brute force scripts to social media playbooks in near real-time, and tip off platforms or hosting providers before the campaigns broke out of Category 2 impact.\u201d<\/p>\n

That information can help enterprise CISOs in two ways, Roberts said.\u00a0<\/p>\n

\u201cFirst, large language model telemetry is becoming a bona fide threat intelligence feed. You\u2019ll want a way to ingest hashes, domains, and TTPs [tactics, techniques, and procedures] that model providers surface,\u201d Roberts said. \u201cSecond, AI misuse today is mostly efficiency gain, not capability gain. OpenAI found no evidence that its models gave nation state actors tools they couldn\u2019t already script, just a speed boost that also widened their digital footprint. That means classic controls still work: monitor for script offloading, insist on human validation of resum\u00e9 pipelines, and treat sudden spikes in polarizing social content as IO [Influence Operations<\/a>] smoke signals.\u201d<\/p>\n

That might just make life easier for SOCs looking to detect and block these attacks.<\/p>\n

\u201cIn short, AI driven offense is real but still somewhat clumsy, and transparency from model providers turns that clumsiness into a detection advantage,\u201d Roberts said. \u201cSecurity teams should press vendors for similar reporting and wire those indicators into their SOC before the next [genAI-fueled attack] shows up.\u201d<\/p>\n

Tactics of attackers<\/h2>\n

The OpenAI report, published in June<\/a>, detailed a variety of defenses the company has deployed against fraudsters. One, for example, involved bogus job applications.<\/p>\n

\u201cWe identified and banned ChatGPT accounts associated with what appeared to be multiple suspected deceptive employment campaigns. These threat actors used OpenAI\u2019s models to develop materials supporting what may be fraudulent attempts to apply for IT, software engineering, and other remote jobs around the world,\u201d the report said. \u201cAlthough we cannot determine the locations or nationalities of the threat actors, their behaviors were consistent with activity publicly attributed to IT worker schemes connected to North Korea (DPRK). Some of the actors linked to these recent campaigns may have been employed as contractors by the core group of potential DPRK-linked threat actors to perform application tasks and operate hardware, including within the US.\u201d<\/p>\n

Another tactic involved a traditional cyberattack with malware.<\/p>\n

\u201cWe banned a cluster of ChatGPT accounts that appeared to be operated by a Russian-speaking threat actor. This actor used our models to assist with developing and refining Windows malware, debugging code across multiple languages, and setting up their command-and-control infrastructure,\u201d the report said. \u201cThe actor demonstrated knowledge of Windows internals and exhibited some operational security behaviors. Based on the operation\u2019s focus on using a trojanized crosshair gaming tool and its stealthy tactics, we have dubbed it ScopeCreep.\u201d\u00a0\u00a0<\/p>\n

Perhaps the most interesting part of the report dealt with some tweaks of fraud attacks that CISO teams can watch for.<\/p>\n

\u201cThis threat actor had a notable approach to operational security. They utilized temporary email addresses to sign up for ChatGPT accounts, limiting each ChatGPT account to one conversation about making one incremental improvement to their code. They then abandoned the original account and created a new one,\u201d the report noted. \u201cThe actor distributed the ScopeCreep malware through a publicly available code repository that impersonated a legitimate and popular crosshair overlay tool (Crosshair-X) for video games.\u201d <\/p>\n

The report said that unsuspecting users who downloaded and ran the malicious version would have additional malicious files downloaded from attacker infrastructure and executed. Then the malware would initiate a multi-stage process to escalate privileges, establish stealthy persistence, notify the threat actor, and exfiltrate sensitive data while evading detection.\u00a0\u201cThe threat actor utilized our model to assist in developing the malware iteratively, by continually requesting ChatGPT to implement further specific features,\u201d OpenAI said.<\/p>\n

Will Townsend, a VP and principal analyst with Moor Insights & Strategy, was more charitable than Gartner.<\/p>\n

\u201cIt clearly demonstrates the depth that OpenAI is taking to secure models and mitigate poisoning that can lead to hallucinations and GPU workload disruption,\u201d Townsend said.<\/p>\n

Detection \u2018easy to sidestep\u2019<\/h2>\n

However, Gartner\u2019s Litan detailed several of her concerns about the OpenAI report that colored her opinion of it.<\/p>\n

\u201cIt is reactive and measures [attacks] after<\/em> misuse is detected\u201d such as after malware is created, Litan said. She also saw the proposed defense techniques as \u201cresource-intense monitoring that relies on heavy-handed human resources for detection. Not scalable.\u201d<\/p>\n

She also observed that OpenAI did the obvious, in that it \u201conly focuses on OpenAI models and not other AI platforms or open source models.\u201d<\/p>\n

Litan called the techniques that OpenAI described as relatively easy for attackers to sidestep.\u00a0\u201cThere is a risk of attacker evasion [because] their reactive detection can\u2019t keep up with fast evolving tactics,\u201d she said.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>","protected":false},"excerpt":{"rendered":"

OpenAI\u2019s recent report detailing various defenses it has deployed to fight fraudsters, especially those leveraging its LLM to impersonate people on social media, has met with mixed reactions from experts. One prominent analyst group, Gartner, sees it as more of a PR stunt than actually making a case that OpenAI is delivering a cybersecurity differentiator.\u00a0 […]<\/p>\n","protected":false},"author":0,"featured_media":3513,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-3512","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-education"],"_links":{"self":[{"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/posts\/3512"}],"collection":[{"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3512"}],"version-history":[{"count":0,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/posts\/3512\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/media\/3513"}],"wp:attachment":[{"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3512"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3512"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3512"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}