{"id":2744,"date":"2025-04-14T11:29:25","date_gmt":"2025-04-14T11:29:25","guid":{"rendered":"https:\/\/cybersecurityinfocus.com\/?p=2744"},"modified":"2025-04-14T11:29:25","modified_gmt":"2025-04-14T11:29:25","slug":"ai-hallucinations-lead-to-a-new-cyber-threat-slopsquatting","status":"publish","type":"post","link":"https:\/\/cybersecurityinfocus.com\/?p=2744","title":{"rendered":"AI hallucinations lead to a new cyber threat: Slopsquatting"},"content":{"rendered":"
\n
\n
\n
\n
<\/div>\n

Cybersecurity researchers are warning of a new type of supply chain attack, Slopsquatting, induced by a hallucinating generative AI model recommending non-existent dependencies.<\/p>\n

According to research by a team from the University of Texas at San Antonio, Virginia Tech, and the University of Oklahama, package hallucination is a common thing with Large Language Models (LLM)-generated code which threat actors can take advantage of.<\/p>\n

\u201cThe reliance of popular programming languages such as Python and JavaScript on centralized package repositories and open-source software, combined with the emergence of code-generating LLMs, has created a new type of threat to the software supply chain: package hallucinations,\u201d the researchers said in a paper<\/a>.<\/p>\n

From the analysis of 16 code-generation models, including GPT-4, GPT-3.5, CodeLlama, DeepSeek, and Mistral, researchers observed approximately a fifth of the packages recommended to be fakes.<\/p>\n

Threat actors can exploit hallucinated names<\/h2>\n

According to the researchers, threat actors can register hallucinated packages and distribute malicious codes using them.<\/p>\n

\u201cIf a single hallucinated package becomes widely recommended by AI tools, and an attacker has registered that name, the potential for widespread compromise is real,\u201d according to a Socket analysis of the research. \u201cAnd given that many developers trust the output of AI tools without rigorous validation, the window of opportunity is wide open.\u201d<\/p>\n

Slopsquatting, as researchers are calling it, is a term first coined by Seth Larson, a security developer-in-residence at Python Software Foundation (PSF), for its resemblance to the typosquatting<\/a> technique. Instead of relying on a user\u2019s mistake, as in typosquats, threat actors rely on an AI model\u2019s mistake.<\/p>\n

A significant number of packages, amounting to 19.7% (205,000 packages), recommended in test samples were found to be fakes. Open-source models \u2013like DeepSeek and WizardCoder\u2013 hallucinated more frequently, at 21.7% on average, compared to the commercial ones (5.2%) like GPT 4.<\/p>\n

Researchers found CodeLlama ( hallucinating over a third of the outputs) to be the worst offender, and GPT-4 Turbo ( just 3.59% hallucinations) to be the best performer.<\/p>\n

These hallucinations are bad news<\/strong><\/h2>\n

These package hallucinations are particularly dangerous as they were found to be persistent, repetitive, and believable.<\/p>\n

When researchers reran 500 prompts that had previously produced hallucinated packages, 43% of hallucinations reappeared every time in 10 successive re-runs, with 58% of them appearing in more than one run.<\/p>\n

The study concluded that this persistence indicates \u201cthat the majority of hallucinations are not just random noise, but repeatable artifacts of how the models respond to certain prompts.\u201d This increases their value to attackers, it added.<\/p>\n

Additionally, these hallucinated package names were observed to be \u201csemantically convincing\u201d. Thirty-eight percent of them had moderate string similarity to real packages, suggesting a similar naming structure. \u201cOnly 13% of hallucinations were simple off-by-one typos,\u201d Socket added.<\/p>\n

While neither the Socket analysis nor the research paper mentioned any in-the-wild Slopsquatting instances, both cautioned protective measures. Socket recommended developers >install dependency scanners\u00a0before production<\/a>\u00a0and\u00a0runtime<\/a> to fish out malicious packages. Rushing through security testing is one of the reasons AI models succumb to hallucinations. Recently, OpenAI was blamed<\/a> for slashing its models\u2019 testing time and resources significantly, exposing its usage to significant threats.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>","protected":false},"excerpt":{"rendered":"

Cybersecurity researchers are warning of a new type of supply chain attack, Slopsquatting, induced by a hallucinating generative AI model recommending non-existent dependencies. According to research by a team from the University of Texas at San Antonio, Virginia Tech, and the University of Oklahama, package hallucination is a common thing with Large Language Models (LLM)-generated […]<\/p>\n","protected":false},"author":0,"featured_media":2745,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-2744","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-education"],"_links":{"self":[{"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/posts\/2744"}],"collection":[{"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=2744"}],"version-history":[{"count":0,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/posts\/2744\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/media\/2745"}],"wp:attachment":[{"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2744"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=2744"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=2744"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}