Key Takeaways:

AI has pushed security to the forefront, helping attackers find vulnerabilities in chips and the chip industry to defend against them.
Security needs to extend throughout a chip’s lifetime, but it’s not clear how AI will age with a chip.
AI pushes visibility outside a chip, which changes the dynamics for how to secure it.

AI is making it both easier and more difficult to secure electronic devices, amassing an enormous list of known and potential system vulnerabilities that can help prevent cyberattacks, while making it more difficult to secure both hardware and software.

On the plus side, AI can scrape the Internet for all available research and reports, involving every known component and architecture, and guide engineering teams on the best way to assemble them. It can flag compromised protocols and IP, find hidden keys and side-channel weaknesses that might be overlooked using traditional tools, and adjust for LLM drift and AI agent aging through continuous monitoring.

The downside is that nearly all data is fair game for AI. A fundamental benefit of AI is its greatest liability, enabling it to access data anywhere within a system, or between systems. But it’s not always clear where or when AI obtains its data, or how that data can be combined with other data. AI is a black box that must be observed from outside a chip or system, rather than using traditional approaches such as monitoring traffic, scanning for viruses and Trojans, and ensuring that passwords and keys are updated regularly.

As a result, it’s difficult to prevent problems or control AI’s behavior, no matter how AI-savvy the engineering team. In July, OpenAI models escaped from a sandbox using a zero-day vulnerability and exposed data from Hugging Face, which is basically the equivalent of GitHub for AI/ML. Days later, Anthropic revealed that its Claude AI assistant was able to access the Internet “from within testing environments that should have been sealed off.”

On top of that, Anthropic’s Mythos frontier model can be used to quickly find hardware vulnerabilities that would take the best security experts years to identify, if ever. “This is technology that is very difficult to put back in the box,” observed Marc Witteman, senior director at Keysight Technologies. “And there are many organizations working on this, so even if Anthropic is stopped, others will continue.”

What’s particularly dangerous about Mythos, and other models like it, is the breadth of knowledge about potential security risks and the speed at which they can adapt to find new security holes. “A friend of mine runs a small security consulting testing company, and he got into Mythos and took some of the penetration scripts that he had trained the tool on, the usage of the scripts, and worked with the tool to find what other zero days already exist,” said Scott Best, senior director for silicon security products at Rambus. “There are very few actual proprietary attacks. But when you train a chatbot on the entirety of the Internet, and all the human knowledge that has ever been online, and then you strip-mine it for the parts, you get a lot of obscure information that you didn’t know about. There is outstanding research out there that nobody knows about, and these tools have been trained on all of it. So my friend had the tool investigate other zero days and compiled it together, and he instrumented a fake network that was completely virtualized, put a bunch of virtual machines on the network running various commercial operating systems, and set this tool loose. And we just watched as the tool ran these scripts and thousands of cybersecurity vulnerabilities and probed every one of them, and something like 20% of those systems were now compromised. This all happened within 10 minutes while sitting in a pub. Mythos is an incredible force multiplier.”

Validation
There is no quick fix to this problem, and no one-size-fits-all solution. But there is widespread concern around the globe about how to close security holes and build more resilient hardware. What’s changed is that in an AI-driven world, security is no longer a separate discipline. It is an integral part of every aspect of chip design through manufacturing and beyond that changes the criticality of different steps and processes, and the level at which any potential problems need to be addressed.

Validation is a good example. In the past, as long as a device met or exceeded the power and performance specs, and was fully functional at time zero, it was considered a good chip — even if it didn’t turn out exactly as the device’s architect(s) intended. But in an AI world, that’s no longer the case. For security reasons, what comes out of manufacturing increasingly needs to look much more like it was supposed to, and it needs to be validated against different workloads, and at different points in its lifecycle.

“AI introduces some new concepts, like non-deterministic type of aging, meaning there is some accuracy degradation that may become a security vulnerability, not just performance loss,” said Dana Neustadter, senior director of product management for Security IP and Solutions at Synopsys. “AI systems make it more important than ever to move from protecting at design time to adapting over time — over a lifetime, actually — because for the AI system it’s not about just making sure that the model works correctly. The behavior needs to be consistent over time, so you need some form of more complex cross-layer trust correlation that needs to happen. AI introduces some new concepts that need to be taken into account, specifically for security.”

AI agents add yet another dimension. “Agentic AI changes the hardware problem because workloads become more dynamic, more distributed, and less predictable,” said Noam Brousard, vice president of solutions engineering at proteanTecs. “CPUs, accelerators, memory subsystems, fabrics, and interfaces may be exercised in patterns that are hard to model in advance. That has a direct impact on data analytics because the value is no longer just in knowing what the chip looked like at manufacturing test. The value is in correlating real operating behavior with real electrical margin over time.”

In effect, validation becomes a reference point, margin shrinks, and tighter integration is needed between design teams, fabs, and OSATs. That, in turn, requires sharing more data so it can be analyzed over time, because a chip or chiplet or compute cluster is part of a larger system, and AI-driven attacks can find vulnerabilities anywhere in a system.

Sharing data isn’t something that comes easily for the chip industry. Still, the industry may have no choice as penalties are attached to breaches. Under the European Cyber Resilience Act (CRA), for example, the EU can impose fines of up to 2.5% of a company’s total revenue, or up to €15 million, whichever is greater, for failing to comply with regulations involving hardware and software.

“There was a big lag in terms of security awareness because there was no one date,” said Yan-Taro Clochard, product marketing group director at Secure-IC, a Cadence Company. “Nobody said, ‘You have to do it like this. You have to take care of security in a mandatory manner.’ Having regulations pushing players in the industry to think that through is very important. CRA makes it mandatory for the customer to know what their security problem might be, to anticipate the supply chain, to anticipate their operations, and to think about security during a chip’s lifetime. For the security industry at large, this is a good thing, and overall, security for the entire end-to-end supply chain is very good because it forces everybody to think about how sensitive their assets are.”

Achieving that goal may be painful, however. With AI, understanding what went wrong and who’s responsible is a huge challenge when access to data is limited, or when AI agents begin aging in unanticipated ways.

“Agentic CPUs pose a bigger challenge to reliability than classical CPUs,” said proteanTecs’ Brousard. “In classic CPUs, when a wrong value results, the entire process would be stopped by a built-in boundary such as a memory violation, a checksum error, or a null result, and the system crashes (ie. blue screen, glitch). The system architecture naturally limits how long a wrong value can remain silent before something catches it or breaks. That is why SDC, or silent data corruption, was not always an obvious, easy-to-sell use case for CPU customers. In many CPU environments, customers may say, ‘If something goes wrong, we usually know pretty quickly.’”

What caused it to go wrong in an AI chip may or may not be a cyberattack, but a failure could well open the door for a side-channel attack. At the same time, closing that door may be significantly easier if what gets manufactured exactly matches what’s in the initial architecture. The architecture provides a baseline for determining how the AI models and agents, which are otherwise opaque, may have drifted.

“This is about removing hallucinations and giving results based upon the ground truth of the tools themselves,” said Amit Gupta, senior vice president and general manager of the Solido Custom IC and Central AI divisions at Siemens EDA. “And then, it’s very important to have skills files that are optimized for the tools so that the agent knows, ‘Okay, the user is asking for this kind of task. These are the tools that I’m going to call to perform this. And the agents aren’t always going to be right, so it’s important to give ‘verbose mode’ to the engineers to see what the agent is doing. Then the engineer can say, ‘Hey, wait a second, this isn’t quite what I wanted. You’re going in the wrong direction. Change it over to this and pause things. Tell the agent to take on this new way of working, and then it can go ahead and interrupt.’”

Verbose mode is an option in many programming languages to provide log or reasoning details, such as which tool is called, what steps are being executed, and what queries are being run. “It’s not a black box that’s just going off and doing something,” Gupta said. “The engineer can see what’s happening and run the characterization.”

Visibility is a relative term where AI is involved, but the more data that can be provided by and for design teams, the better.

“An AI model may be smarter than anyone, but it doesn’t necessarily have all the tools,” said Keysight’s Witteman. “In addition to models, you also need agents, and agents are AI functions that understand how to use a tool. Typically, you combine an AI model with several AI agents that reach out to the physical world. If you want to test something, you connect it to some test equipment, and that test equipment will be connected to AI agents that know how to run those tests, that are connected to an AI model that can generate tests and execute them. So it’s a network of functions with an AI model at the core and AI agents that talk to the physical world. One team tried to build this whole setup with a model and several agents, and they actually ended up with two models. One model does the testing work, and the other model is the supervisor. So I asked them, ‘What does the supervisor do?’ And the guy who worked on it said, ‘The supervisor just keeps testing or overseeing the health of the situation. If a product that you’re testing stops responding, then it doesn’t make sense to keep testing it because it doesn’t do anything. So this supervisor is looking at, ‘Are there still LEDs blinking? Is there still communication happening? Because if it’s not, then something probably has crashed and you need to reboot everything.’”

Attestation
However, determining what caused a failure isn’t always obvious, particularly in the AI world, so some expert sleuthing is needed. Was some anomalous behavior a security breach, or an orchestration failure in a complex system of systems? To find out, engineers may need to start at the beginning. Where and when were the die, IP, AI agents and models, and various interconnects created? What were the initial specs? Was it manufactured correctly, or was it binned as underperforming or running too hot?

“If you can measure something, you can store a measurement,” said Erik Wood, senior director of cryptography and product security at Infineon Technologies. “And then you can have another party measure it again in real-time, authenticate it, and then sign that authentication. And so it’s cryptographically verified that, ‘Yes, it is the right measurement. And yes, I’m a trusted source who just signed that.’”

This is particularly important when it comes to the AI language models and agents that run on the physical hardware. “For everything in ML/AI data centers, and ML/AI at the edge, all the customers are requiring a concept called ‘authenticate at time of use.’ That is the most important thing that they’re all asking for,” Wood said. “We want to be able to authenticate the machine learning model at the time of use. If we can’t trust it at the time of use, because it’s been sitting idle and it could be messed with, something could have happened. Somebody could have access to it. We don’t know how long the part’s been powered off. And as soon as it’s powered on, we want to be able to authenticate it at time of use so that every time we’re using that model or using that application, we know it’s trusted.”

Insulation
Another option is to basically insulate data from the hardware with a virtualization layer. Virtual machine (VM) technology has been proven for decades in data centers for load balancing and adding more compute resources in large data centers and in automotive applications. Using VMs for orchestration allows for a quick fix if something goes awry.

“A lot of this has been coming out of the automotive space, where the automotive companies are trying to move from many systems into a single central processing unit,” said Kristof Beets, vice president of product management at Imagination Technologies. “But you’ve got all these different processing requirements. Some of them are graphics, some are compute, some are functional safety, and some are just generic stuff that you don’t care about. So how do you deal with all of those and give them the right protection? How do you give them the right priorities in the system?”

Automotive has a different problem, as well, which makes virtualization compelling. The design and development cycles are slow for hardware, which is why OEMs and their suppliers are shifting to a software-defined approach. But from a security standpoint, insulating the hardware from a potential hack using a virtualization layer is a way to combine benefits from both worlds — the robustness and good enough speed for hardware and the flexibility of software to add new features and close security holes as they arise.

“With ADAS and basic assisted features, it wasn’t critical. It couldn’t crash the car,” Beets said. But now, increasingly, we see both in regulations and in use cases where you have to make sure what you are doing is accurate. Lots of people look at their little screen, and you want to make sure what you’re seeing in that surround view is correct and updating and providing you with the correct warnings where necessary. The most extreme form of this is autonomous vehicles, where it’s a combination of CPUs, dedicated neural processing units, and an array of GPUs. How you execute is critical, and virtualization can help with all of those different domains and associations.”

Conclusion
AI will change security in fundamental ways. AI-based tools will find vulnerabilities, and AI-based tools will close them. From that perspective, nothing has changed. But what is changing is the speed at which vulnerabilities will be found and closed. It’s not clear yet if this game of winners and losers will actually shift the balance. But how and where the tools on both sides are used will clearly become more sophisticated, and designs will become more robust, more accurate, and much easier to trace.

AI moves at blazing speed, armed with all the knowledge that has ever been published. The big questions are how that knowledge will evolve, what it will be used for, and for what purposes. At this point in time, no one really knows.