Now, some believe AI could eventually automate one of the industry’s hardest jobs: building the software that powers AI itself.<\/p>\n
The industry is at an important threshold, says Jeremy Nixon<\/a>, a former Google Brain researcher and the founder of AI software startup Infinity. He told Business Insider his startup used AI coding agents to recreate CUDA-like software for the chip startup D-Matrix <\/a>in 10 hours \u2014 evidence, he said, that one of Nvidia’s biggest moats is being crossed.<\/p>\n Infinity founder and CEO Jeremy Nixon.\u00a0<\/p>\n Infinity.inc<\/p>\n The pressure doesn’t only come from startups. Cloud giants like Google, Amazon, and Microsoft have spent years building software around their own AI chips, while OpenAI and Anthropic have recently demonstrated AI models capable of generating system software.<\/p>\n DeepSeek founder Liang Wenfeng<\/a> recently said that coding agents<\/a>, along with his startup’s own programming language TileLang, have made AI software substantially easier to build.<\/p>\n Coding agents aren’t just helping challengers.<\/p>\n Nvidia said developers increasingly use CUDA’s code libraries to build AI applications, and that it also “uses AI coding agents to develop CUDA faster and validate at greater scale,” said Ankit Patel, Nvidia’s vice president of developer ecosystem.<\/p>\n Inference could change the CUDA equation<\/p>\n If CUDA’s first advantage was software, the second is everything built on top of it. Millions of lines of code and internal workflows have been developed by companies, creating a powerful lock-in effect that makes switching to alternatives costly and cumbersome.<\/p>\n Internal documents at Amazon identified CUDA <\/a>as a major roadblock to adoption of its Trainium and Inferentia AI chips, Business Insider previously reported.<\/p>\n CUDA’s age is both an advantage and a constraint, said Chris Lattner<\/a>, cofounder and CEO of Qualcomm-owned AI software startup Modular<\/a>. Originally built for gaming long before the AI boom, CUDA carries layers of legacy technology, “like Microsoft Windows trying to fit onto a phone,” he said.<\/p>\n Modular cofounder and CEO Chris Lattner.\u00a0<\/p>\n Modular<\/p>\n Others say AI’s shift from training toward inference \u2014 where models answer requests and draw conclusions \u2014 creates another threat.<\/p>\n With this evolution, companies care less about maximizing performance with the most powerful chips and more about running AI profitably, said Marshall Choy, chief business officer of Korean AI chip startup Rebellions.<\/p>\n This could result in greater demand not only for specialized hardware but for software that can run across different chips. If companies can switch between chips without rewriting software, that reduces one of CUDA’s biggest lock-ins.<\/p>\n “That’s where the CUDA moat from Nvidia gets broken because CUDA is no longer a factor in the inference side,” Choy said. “It’s an open source play.”<\/p>\n Nvidia said that its tightly integrated hardware and software offerings have become more valuable as AI models get put to work.<\/p>\n “As AI shifts toward inference and agentic workloads, the need for deep, full-stack optimization only grows,” Patel said.<\/p>\n<\/p>\n
<\/p>\n