AI startups leak sensitive credentials on GitHub, exposing models and training data

Tags:

Nearly two-thirds of the world’s top private AI companies have exposed API keys and access tokens on GitHub, according to new research from cloud security firm Wiz, raising concerns that rapid growth is outpacing security discipline.

Wiz found verified secret leaks in 65% of the Forbes AI 50 companies, representing a combined valuation of more than $400 billion.

Despite the severity of the issue, nearly half of Wiz’s disclosure attempts either failed to reach the affected companies or received no response, suggesting limited preparedness to handle security reports.

The leaks included credentials that could have exposed private AI models, training data, and internal organizational details, suggesting how speed-to-market pressures continue to outpace secure development practices in the AI sector.

“Think API keys, tokens, and sensitive credentials, often buried deep in deleted forks, gists, and developer repos most scanners never touch,” Wiz said in a blog. “Some of these leaks could have exposed organizational structures, training data, or even private models.”

Some of the leaked credentials belonged to major AI platforms such as Hugging Face, Weights & Biases, and LangChain, which could have granted access to private models or sensitive training datasets, according to Wiz.

Wider implications

Analysts have said that misconfigured cloud storage has been a recurring problem for more than a decade, citing past incidents such as AWS S3 leaks.

But while the pattern is familiar, the potential damage has grown as exposed assets now include AI models, training data, and complex development pipelines.

“Speed vs security seems to be the underlying root cause leading to cloud misconfigurations, inadequate secret management, lack of security or privacy by design, and tooling gaps,” said Sunil Varkey, a cybersecurity analyst. “The impacts can be severe, extending well beyond typical data exposure, as an AI leak can disrupt multiple levels of an organization at once, including technology, business, legal, ethical, and strategic competitiveness.”

The scale of exposure points to “a glaring DevSecOps chasm” between AI startups and more mature SaaS or cloud firms, according to Chandrasekhar Bilugu, CTO of SureShield. “AI teams, racing to prototype, often store secrets like configuration files in public repositories, with many missing even basic scanning of deleted forks or gists,” he said.

“With companies exposing API keys and tokens on GitHub, the real-world risk is catastrophic: attackers can hijack proprietary models for competitive sabotage, siphon customer PII for identity theft (affecting billions in potential GDPR fines), or pivot into supply chain chaos,” Bilugu added. “In AI, where training data is a precious commodity, a single leaked token grants access to thousands of private models with IP theft or model poisoning that follows.”

The findings suggest that as AI adoption accelerates, developers and CISOs alike will need to tighten oversight of development pipelines and secret storage practices.

Compliance and governance

The Wiz findings highlight how exposed API keys can escalate into full-scale compromises across AI ecosystems, according to Sakshi Grover, senior research manager for IDC Asia Pacific Cybersecurity Services. “Stolen credentials can be used to manipulate model behavior or extract training data, undermining trust in deployed systems.”

Grover noted that such exposures are often linked to the way AI development environments operate. “AI projects often operate in loosely governed, experimentation-driven environments where notebooks, pre-trained models, and repositories are shared frequently, leaving secrets unscanned or unrotated,” Grover added.

She pointed to data from IDC’s Asia/Pacific Security Study, which showed that 50% of enterprises in APAC alone plan to invest in API security when selecting CNAPP vendors, reflecting how exposed APIs have become a major attack vector.

With regulators sharpening their focus on AI safety and data protection, secret management and API governance are likely to become auditable elements of emerging AI compliance frameworks, Grover said.

Categories

No Responses

Leave a Reply

Your email address will not be published. Required fields are marked *