Researchers have developed a tool that they say can make stolen high-value proprietary data used in AI systems useless, a solution that CSOs may have to adopt to protect their sophisticated large language models (LLMs).
The technique, created by researchers from universities in China and Singapore, is to inject plausible but false data into what’s known as a knowledge graph (KG) created by an AI operator. A knowledge graph holds the proprietary data used by the LLM.
Injecting poisoned or adulterated data into a data system for protection against theft isn’t new. What’s new in this tool – dubbed AURA (Active Utility Reduction via Adulteration)– is that authorized users have a secret key that filters out the fake data so the LLM’s answer to a query is usable. If the knowledge graph is stolen, however, it’s unusable by the attacker unless they know the key, because the adulterants will be retrieved as context, causing deterioration in the LLM’s reasoning and leading to factually incorrect responses.
The researchers say AURA degrades the performance of unauthorized systems to an accuracy of just 5.3%, while maintaining 100% fidelity for authorized users, with “negligible overhead,” defined as a maximum query latency increase of under 14%. They also say AURA is robust against various sanitization attempts by an attacker, retaining 80.2% of the adulterants injected for defense, and the fake data it creates is hard to detect.
Why is all this important? Because KGs often contain an organization’s highly sensitive intellectual property (IP), they are a valuable target.
Mixed reactions from experts
However, the proposal has been greeted with skepticism by one expert and with caution by another.
“Data poisoning has never really worked well,” said Bruce Schneier, chief of security architecture at Inrupt Inc., and a fellow and lecturer at Harvard’s Kennedy School. “Honeypots, no better. This is a clever idea, but I don’t see it as being anything but an ancillary security system.”
Joseph Steinberg, a US-based cybersecurity and AI consultant, disagreed, saying, “in general this could work for all sorts of AI and non-AI systems.”
“This is not a new concept,” he pointed out. “Some parties have been doing this [injecting bad data for defense] with databases for many years.” For example, he noted, a database can be watermarked so if it is stolen and some of its contents are later used – a fake credit card number, for example — investigators knows where that piece of data came from. Unlike watermarking, however, which puts one bad record into a database, AURA poisons the entire database, so if it’s stolen, it’s useless.
AURA may not be needed in some AI models, he added, if the data in the KG isn’t sensitive. The real unanswered question is what the real-world trade-off between application performance and security would be if AURA is used.
He also noted that AURA doesn’t solve the problem of an undetected attacker interfering with the AI system’s knowledge graph, or even its data.
“The worst case may not be that your data gets stolen, but that a hacker puts bad data into your system so your AI produces bad results and you don’t know it,” Steinberg said. “Not only that, you now don’t know which data is bad, or which knowledge the AI has learned is bad. Even if you can identify that a hacker has come in and done something six months ago, can you unwind all the learning of the last six months?”
This is why Cybersecurity 101 – defense in depth – is vital for AI and non-AI systems, he said. AURA “reduces the consequences if someone steals a model,” he noted, but whether it can jump from a lab to the enterprise has yet to be determined.
Knowledge graphs 101
A bit of background about knowledge graphs: LLMs use a technique called Retrieval-Augmented Generation (RAG) to search for information based on a user query and provide the results as additional reference for the AI system’s answer generation. In 2024, Microsoft introduced GraphRAG to help LLMs answer queries needing information beyond the data on which they have been trained. GraphRAG uses LLM-generated knowledge graphs to improve performance and lower the odds of hallucinations in answers when performing discovery on private datasets such as an enterprise’s proprietary research, business documents, or communications.
The proprietary knowledge graphs within GraphRAGs make them “a prime target for IP theft,” just like any other proprietary data, says the research paper. “An attacker might steal the KG through external cyber intrusions or by leveraging malicious insiders.”
Once an attacker has successfully stolen a KG, they can deploy it in a private GraphRAG system to replicate the originating system’s powerful capabilities, avoiding costly investments, the research paper notes.
Unfortunately, the low-latency requirements of interactive GraphRAG make strong cryptographic solutions, such homomorphic encryption of a KG, impractical. “Fully encrypting the text and embeddings would require decrypting large portions of the graph for every query,” the researchers note. “This process introduces prohibitive computational overhead and latency, making it unsuitable for real-world use.”
AURA, they say, addresses these issues, making stolen KGs useless to attackers.
AI is moving faster than AI security
As the use of AI spreads, CSOs have to remember that artificial intelligence and everything needed to make it work also make it much harder to recover from bad data being put into a system, Steinberg noted.
“AI is progressing far faster than the security for AI,” Steinberg warned. “For now, many AI systems are being protected in similar manners to the ways we protected non-AI systems. That doesn’t yield the same level of protection, because if something goes wrong, it’s much harder to know if something bad has happened, and its harder to get rid of the implications of an attack.”
The industry is trying to address these issues, as the researchers observe in their paper. One useful reference, they note, is the US National Institute for Standards and Technology (NIST) AI Risk Management Framework that emphasizes the need for robust data security and resilience, including the importance of developing effective KG protection.
No Responses