{"id":10443,"date":"2026-11-18T08:25:00","date_gmt":"2026-11-18T08:25:00","guid":{"rendered":"https:\/\/cybersecurityinfocus.com\/?p=10443"},"modified":"2026-11-18T08:25:00","modified_gmt":"2026-11-18T08:25:00","slug":"how-organizations-can-secure-their-ai-generated-code","status":"publish","type":"post","link":"https:\/\/cybersecurityinfocus.com\/?p=10443","title":{"rendered":"How organizations can secure their AI-generated code"},"content":{"rendered":"<div>\n<div class=\"grid grid--cols-10@md grid--cols-8@lg article-column \">\n<div class=\"col-12 col-10@md col-6@lg col-start-3@lg\">\n<div class=\"article-column__content\">\n<div class=\"container\"><\/div>\n<p class=\"wp-block-paragraph\">The IT manager didn\u2019t know the company hired a \u201cvibe coder\u201d straight out of school. By the time they met a few months later, the new hire had already built a web application that operated with customer data and private contracts.<\/p>\n<p class=\"wp-block-paragraph\">The app ran under a domain the new hire controlled, stored data on Supabase\u2019s free plan, and deployed directly from his GitHub repository to Cloudflare\u2019s serverless infrastructure.<\/p>\n<p class=\"wp-block-paragraph\">\u201cHe vibe-codes everything straight into production, on servers all over the world,\u201d the <a href=\"https:\/\/www.reddit.com\/r\/sysadmin\/comments\/1u5jtub\/shadow_vibe_coder_in_my_department\/\">IT manager wrote on Reddit<\/a>. \u201cWe\u2019re EU based.\u201d<\/p>\n<p class=\"wp-block-paragraph\">When asked whether he had read the AI-generated code or taken any security precautions, the new hire said, \u201cMy GitHub repo is private.\u201d<\/p>\n<p class=\"wp-block-paragraph\">The IT manager took the concern to the CEO, who brushed it aside: \u201cLet it go.\u201d<\/p>\n<p class=\"wp-block-paragraph\">In many organizations, vibe coders build software without thinking much about security. Their apps may get the job done, but they are ticking bombs.<\/p>\n<p class=\"wp-block-paragraph\">In <a href=\"https:\/\/arxiv.org\/pdf\/2512.03262\">SusVibes<\/a>, a benchmark of 186 real-world feature requests from open-source projects, agents given feature descriptions without security guidance produced working code up to 57% of the time. But only 11.8% of their solutions were secure.<\/p>\n<p class=\"wp-block-paragraph\">\u201cThe failures we saw were not careless errors,\u201d says Danqing Wang of Carnegie Mellon University\u2019s Language Technologies Institute, author of the paper describing SusVibes. \u201cSpotting those requires reasoning about what an attacker can do, not just what the feature should do.\u201d<\/p>\n<p class=\"wp-block-paragraph\">What has changed, especially this year, is that AI code-generating tools have gotten so good at producing working code that people are just trusting them, says Dr. Katie Paxton-Fear, staff security advocate at Semgrep. \u201cDevelopers simply aren\u2019t being as critical now as they were when AI was worse at writing code.\u201d<\/p>\n<p class=\"wp-block-paragraph\">Whether it\u2019s the vibe-coder in marketing or the seasoned engineering team boosting its productivity, AI-generated code is an increasing security concern for CISOs. Much of that is a function of volume, but even as AI has proved highly capable of finding flaws in code bases, it is <a href=\"https:\/\/www.csoonline.com\/article\/4210735\/ai-can-find-zero-days-but-still-cant-reliably-write-secure-code.html\">not yet adept at contributing code without introducing its own<\/a>.<\/p>\n<h2 class=\"wp-block-heading\">Treating AI-generated code as \u2018untrusted input\u2019<\/h2>\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/www.infoworld.com\/article\/4024327\/12-ai-coding-agents-at-the-cutting-edge.html\">Leading AI coding agents<\/a> such as Claude Code, Cursor, GitHub Copilot, and Amazon Q Developer have become developer go-to tools, and CIOs are increasingly <a href=\"https:\/\/www.cio.com\/article\/4176062\/cios-are-enlisting-business-users-to-vibe-code-their-own-apps.html\">enlisting business users to vibe code their own apps<\/a>.<\/p>\n<p class=\"wp-block-paragraph\">But as AI-generated code surfaces everywhere, CISOs have only one option: to treat it \u201cas untrusted input that has to earn its way into the codebase,\u201d says Bianca Lankford, VP of engineering at Datadog.<\/p>\n<p class=\"wp-block-paragraph\">To do so, security leaders are moving security checks closer to where code is written: static analysis, secrets scanning, and dependency checks run on every pull request and, increasingly, in the developer\u2019s editor.<\/p>\n<p class=\"wp-block-paragraph\">\u201cCompanies are also approving specific AI tools, limiting what assistants can access, and training reviewers to spot common AI mistakes like outdated libraries, weak input validation, and overly permissive defaults,\u201d Lankford adds.<\/p>\n<p class=\"wp-block-paragraph\">At Mimecast, using AI to write code doesn\u2019t remove the developer\u2019s responsibility for it, says Leslie Nielson, CISO at the information security vendor. \u201cA named person owns every AI-assisted change that ships, and that\u2019s recorded in the pull request, not assumed,\u201d he adds. \u201cAI can propose. Only people can approve.\u201d<\/p>\n<p class=\"wp-block-paragraph\">The volume of code organizations need to review is only growing, and reviewing everything manually can be impossible. That\u2019s why Nielsen recommends security leaders combine automated checks with human judgment.<\/p>\n<p class=\"wp-block-paragraph\">\u201cWe use automation for the known and repetitive, like dependency checks, secrets detection, and standard vulnerability patterns,\u201d he says. \u201cWe keep a person accountable for anything involving access, authentication, or sensitive data. Those categories always go to a specialist reviewer, regardless of how clean the scan looks.\u201d<\/p>\n<p class=\"wp-block-paragraph\">Datadog\u2019s Lankford says the decision to bring in a specialist should depend on what the code could put at risk, not on what entity \u2014 human or AI \u2014\u00a0wrote it. \u201cAnything involving authentication, cryptography, payments, personal data, or infrastructure goes to a specialist, and code owner rules can route those changes automatically,\u201d she says. \u201cOther useful signals include diff size, new dependencies, and changes to permissions or network exposure.\u201d<\/p>\n<p class=\"wp-block-paragraph\">The goal is to protect the human reviewers\u2019 attention. \u201cA reviewer buried in low-risk changes isn\u2019t reviewing,\u201d Nielsen adds. \u201cThey\u2019re approving.\u201d<\/p>\n<p class=\"wp-block-paragraph\">Another priority should be preserving the knowledge behind the code. \u201cGenerative AI may produce software, but it cannot produce understanding,\u201d notes Alex \u0218tef\u0103nescu, security expert and tech activist. \u201cDocumentation, a solid test suite, and a written record of every significant architecture decision ever made are going to save developers and organizations countless hours when a security incident hits or when they are debugging complex issues.\u201d<\/p>\n<p class=\"wp-block-paragraph\">All of this should be created and maintained by people and intended for human use, not externalized to generative algorithms, \u0218tef\u0103nescu adds.<\/p>\n<h2 class=\"wp-block-heading\">What\u2019s at risk with AI-generated code<\/h2>\n<p class=\"wp-block-paragraph\">The inclusion of third-party libraries or phantom dependencies \u2014 dependencies that are not explicitly declared in a manifest file \u2014 is a common concern with AI-generated code, as these unreported software packages might not be identified during a scan and can potentially hide vulnerabilities.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">One way to address this is to use software composition analysis (SCA) and software supply chain security tools to help identify the libraries in use, as well as the vulnerabilities and potential legal and compliance issues that might bring.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">While not unique to AI-generated code, organizations also expose themselves to risks when developers download machine learning (ML) models or datasets from platforms such as Hugging Face.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">\u201cIn spite of security checks on both ends, it may still happen that the model contains a backdoor that becomes active once the model is integrated,\u201d says \u0218tef\u0103nescu. \u201cThis could ultimately lead to data being leaked from the company that used the malicious models.\u201d<\/p>\n<p class=\"wp-block-paragraph\">When it comes to coding agents and code completion functionality in tools like GitHub Copilot, \u0218tef\u0103nescu worries about hallucinations. \u201cAn LLM will always generate the most statistically probable continuation of a given prompt, so there are no real guarantees in place that it will generate a real package from PyPI, for example, after the word \u2018import\u2019,\u201d they say. \u201cSome attackers are aware of this and register package names on platforms like npm and PyPI, filling in some functionality that code completion tools suggest in order to make the packages seem legitimate.\u201d<\/p>\n<p class=\"wp-block-paragraph\">If these packages are imported into real applications, they can do real damage.<\/p>\n<p class=\"wp-block-paragraph\">Sensitive information inadvertently leaking into codebases because of developers using AI-powered coding tools is another key concern.<\/p>\n<p class=\"wp-block-paragraph\">To address these risks, CISOs can establish protocols for downloading and integrating ML models or datasets from external platforms such as Hugging Face and ensure those protocols are incorporated in AI code agents\u2019 instruction files where applicable. Such protocols can include implementing automated scanning tools to detect malicious code or backdoors, having a policy that only allows the use of models from verified publishers, or conducting internal testing in isolated environments.<\/p>\n<p class=\"wp-block-paragraph\">Not all AI-based code comes from teams full of software engineers. \u201cWe see a lot of adoption being driven by data analysts, marketing teams, researchers, etc., within organizations,\u201d says Darren Meyer, staff research engineer at Endor Labs.<\/p>\n<p class=\"wp-block-paragraph\">These teams aren\u2019t traditionally developing their own software but are increasingly writing simple tools that adopt AI libraries and models, so they\u2019re often not aware of the risks involved. \u201cThis combination of shadow engineering with lower-than-average application security awareness can be a breeding ground for risk,\u201d he adds.<\/p>\n<p class=\"wp-block-paragraph\">To make sure these teams are working safely, CISOs must consider forming relationships with these teams early in the process. Cybersecurity leaders might also want to set up training programs tailored to non-traditional development teams to educate data analysts, marketing professionals, and researchers on the potential risks associated with AI-based tools and libraries.<\/p>\n<h2 class=\"wp-block-heading\">Investing in defense<\/h2>\n<p class=\"wp-block-paragraph\">\u201cSecurity budgets don\u2019t generally grow at the same pace that software development accelerates, and AI adoption is only widening that gap,\u201d Meyer says.\u00a0Application security is often underfunded in most organizations, yet allocating sufficient time and resources to it is essential, as AI adoption and AI-assisted coding accelerate the pace of software development.<\/p>\n<p class=\"wp-block-paragraph\">\u201cA portfolio of high-quality security tools that can help address this gap is no longer optional,\u201d Meyer says. \u201cAnd while tools are critical to closing the gap, so are AppSec and ProdSec staff that can effectively partner with developers \u2014 even non-traditional developers \u2014 and understand the technical, compliance, and security implications of AI.\u201d<\/p>\n<p class=\"wp-block-paragraph\">When it comes to securing enough resources to protect AI systems, some stakeholders might hesitate, viewing it as an optional expense rather than a critical investment. \u201cAI adoption is a divisive topic in many organizations, with some leaders and teams being \u2018all-in\u2019 on adoption and some being strongly resistant,\u201d Meyer says. \u201cThis tension can present challenges for insightful CISOs and business information security officers (BISOs).\u201d<\/p>\n<p class=\"wp-block-paragraph\">CISOs that are aware of both the advantages and disadvantages of this might try to set controls to manage risks effectively, but this might create the perception of holding the organization back from innovating if they don\u2019t explain what they do properly.\u00a0\u201cOrganizations need to develop comprehensive strategies that balance the productivity benefits of AI tools with robust security practices,\u201d says Thomas Segura, senior technical writer at GitGuardian.<\/p>\n<h2 class=\"wp-block-heading\">Cost-effective changes CISOs should implement<\/h2>\n<p class=\"wp-block-paragraph\">But if a security leader has 90 days and no budget to spare, which changes would have the biggest impact?<\/p>\n<p class=\"wp-block-paragraph\">Datadog\u2019s Lankford says a short, plain-language AI tool policy can make expectations clear. At a minimum, such a policy should spell out which tools employees can use, what code and data they can share, and which changes need specialist review. The aim is to give people answers they can use while they work, with a clear point of contact when they\u2019re unsure.<\/p>\n<p class=\"wp-block-paragraph\">It\u2019s also important to understand that employees don\u2019t want to circumvent policies. They just want to get their work done without friction and without waiting weeks for approval. Giving them the right tools makes the safer route easier to take.<\/p>\n<p class=\"wp-block-paragraph\">By contrast, banning tools doesn\u2019t work, because employees will find a way to still use them. A ban can push AI use out of sight, leaving security teams with less visibility into the code being written and the data being shared, Mimecast\u2019s Nielsen says.<\/p>\n<p class=\"wp-block-paragraph\">CISOs should look across the development process to identify where they can intervene and which risks they can address at each stage. \u201cUnderstand what layers you can influence and what vulnerabilities you can influence at each layer: in the design phase, in the CLAUDE.md file, as the agent is writing, during a pull request, as part of an AI pentest,\u201d Semgrep\u2019s Paxton-Fead says.<\/p>\n<p class=\"wp-block-paragraph\">It\u2019s also important to explain to employees who are using AI code-generating tools that security lives in an implicit context: the threat model, the trust boundaries, what the attacker controls. \u201cAgents cannot reliably plan and reason on these, so they optimize for the only signal they have, which is to satisfy the given functionality description,\u201d Carnegie Mellon\u2019s Wang says.<\/p>\n<p class=\"wp-block-paragraph\">When things go wrong, people in the organization need to feel safe telling security what happened, including which unapproved tools they used and what data they shared. Fear of punishment can delay that conversation, leaving the team trying to contain a problem without the full picture.<\/p>\n<p class=\"wp-block-paragraph\">\u201cOffer amnesty instead of a crackdown, so people tell you the truth about what they\u2019re using,\u201d Nielsen advises.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>","protected":false},"excerpt":{"rendered":"<p>The IT manager didn\u2019t know the company hired a \u201cvibe coder\u201d straight out of school. By the time they met a few months later, the new hire had already built a web application that operated with customer data and private contracts. The app ran under a domain the new hire controlled, stored data on Supabase\u2019s [&hellip;]<\/p>\n","protected":false},"author":0,"featured_media":10444,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-10443","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-education"],"_links":{"self":[{"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/posts\/10443"}],"collection":[{"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=10443"}],"version-history":[{"count":0,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/posts\/10443\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/media\/10444"}],"wp:attachment":[{"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=10443"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=10443"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=10443"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}