{"id":107700,"date":"2026-07-16T04:17:07","date_gmt":"2026-07-16T04:17:07","guid":{"rendered":"https:\/\/www.europesays.com\/ai\/107700\/"},"modified":"2026-07-16T04:17:07","modified_gmt":"2026-07-16T04:17:07","slug":"agentic-inference-reshapes-ai-infrastructure-at-raise-summit","status":"publish","type":"post","link":"https:\/\/www.europesays.com\/ai\/107700\/","title":{"rendered":"Agentic inference reshapes AI infrastructure at RAISE Summit"},"content":{"rendered":"

Agentic inference is reshaping the center of gravity in AI infrastructure<\/a>. What began as a race to scale training has shifted into a phase defined by expanding context windows, memory\u2011augmented reasoning and the need to keep graphics processing units continuously fed with data.<\/p>\n

As enterprises push deeper into agentic systems, storage has moved into the critical path of AI performance. Agentic inference is exposing new bottlenecks, accelerating the adoption of design patterns and pushing organizations to rethink how intelligence is staged, retrieved and delivered to GPUs, according to Greg Matson<\/a>\u00a0(pictured), senior vice president and head of marketing and products at Solidigm, a trademark of SK Hynix NAND Product Solutions Corp.<\/p>\n

\u201cIt started a couple of years ago with training, where the need for high-capacity, high-performance storage very adjacent to the GPUs was all of a sudden center stage,\u201d Matson told theCUBE. \u201cBut now, as we go from last year to this year, inference phase into agentic inference, it\u2019s exploding even more. Storage is actually a whole new storage tier that\u2019s being created to extend the memory for the system.\u201d<\/p>\n

Matson spoke with theCUBE at RAISE Summit<\/a>, during an exclusive broadcast on theCUBE, SiliconANGLE Media\u2019s livestreaming studio. Conversations with cross-industry leaders showed how agentic inference is expanding infrastructure design beyond raw compute to encompass specialized architectures, memory and storage, capital deployment and sovereign control of enterprise data. (* Disclosure below.)<\/p>\n

Here\u2019s theCUBE\u2019s complete video interview with Greg Matson:<\/p>\n

Here are three insights you may have missed from theCUBE\u2019s coverage of RAISE Summit<\/a>:<\/p>\n

Insight #1: Agentic inference drives specialization across the AI stack.<\/p>\n

Advanced Micro Devices Inc. is responding to varied AI workloads by optimizing across CPUs, GPUs, adaptive computing and networking rather than focusing on individual chips. Its ROCm software<\/a> stack is designed to provide a consistent layer across data center clusters, edge deployments and AI-enabled PCs, according to Mark Papermaster<\/a>, chief technology officer and executive vice president of AMD.<\/p>\n

\u201cThe workloads are so complex because people are looking at what they do end to end,\u201d he said during the event. \u201cThey\u2019re looking at whole processes, not just one bespoke task. That means you need different computing engines, and they need to work together at scale<\/a>. We\u2019re talking across massive clusters of racks.\u201d<\/p>\n

Tensordyne Inc. is addressing power constraints by changing the math inside the silicon. Its Napier inference chip<\/a> uses the proprietary Pareto logarithmic number system, replacing multiplications with additions to reduce reliance on large, power-intensive multiplier circuits. A 72-chip Napier pod draws 30 kilowatts, compared with 150 kilowatts for a comparable Nvidia Corp. system, according to Gilles Backhus<\/a>, co-founder of Tensordyne.<\/p>\n

\u201cOur logarithmic math \u2014 it\u2019s completely under the hood,\u201d he told theCUBE. \u201cFrom a user point of view, from a [software development kit] point of view, you don\u2019t even notice it. It just looks like normal floating-point math. It\u2019s just that the engine under the hood is more efficient<\/a>.\u201d<\/p>\n

Purpose-built accelerators are also moving into production alongside GPUs. A Parasail deployment<\/a> pairs d-Matrix Corsair accelerators<\/a> with Nvidia Hopper and Blackwell GPUs to serve the different requirements of compute-heavy prefill and latency-sensitive token generation, according to d-Matrix Corp. co-founders Sudeep Bhoja<\/a>, chief technology officer, and Sid Sheth<\/a>, president and chief executive officer. It represents an early commercial-scale example of heterogeneous inference in production.<\/p>\n

\u201cLow latency is the name of the game today,\u201d Bhoja said during the event. \u201cAgents are running for a long time; users don\u2019t want to wait. So, there\u2019s a demand on trying to get the latency down, and that means disaggregated inference<\/a>.\u201d<\/p>\n

Here\u2019s the complete video interview with Sudeep Bhoja and Sid Sheth:<\/p>\n

Insight #2: Storage becomes an active extension of AI memory.<\/p>\n

As agentic inference expands from individual prompts to longer-running sessions, the volume of context data<\/a> can exceed GPU memory capacity. Hyperscalers are replacing legacy infrastructure with high-capacity solid-state storage positioned near accelerators, according to Matson.<\/p>\n

\u201cWhile the GPU is the most expensive part of your infrastructure, you want that thing to be humming 100% of the time generating tokens,\u201d he told theCUBE. \u201cAnd if it\u2019s down, waiting for data, then you\u2019re wasting your money on GPUs<\/a>.\u201d<\/p>\n

Solidigm is testing storage as part of complete AI systems rather than as an isolated component. Its AI Central Lab<\/a> runs actual workloads across accelerator hardware and partner software. At the same time, high-density solid-state drive configurations show how greater capacity in less rack space can reduce storage power demands, according to Avi Shetty<\/a>, vice president of AI ecosystem, solutions and market enablement at Solidigm.<\/p>\n

\u201cNobody cares about random read, random write on a data sheet right now,\u201d Shetty said. \u201cWhat people care about is how does it operate in an AI data center<\/a> and an AI workload.\u201d<\/p>\n

Here\u2019s theCUBE\u2019s complete video interview with Avi Shetty:<\/p>\n

Insight #3: Capital and sovereignty become part of the AI infrastructure stack.<\/p>\n

Agentic inference projects require more than power and GPUs; they can stall when developers can\u2019t assemble financing quickly enough. Argentum AI Inc.\u2019s demand-first model secures customers before committing capital<\/a>, using contracted revenue to support construction while remaining neutral regarding silicon and original equipment manufacturers, according to Andrew Sobko<\/a>, founder and chief executive officer of Argentum AI Inc.<\/p>\n

\u201cWe formed the view that the biggest bottleneck in the focus on the speed of deployment is a capital stack,\u201d he told theCUBE. \u201cHow do you get the projects financed as fast as possible? That sort of became one of our core products, where we call it bringing power, compute and capital.\u201d<\/p>\n

Data sovereignty, meanwhile, is moving from compliance into infrastructure architecture<\/a> as agentic inference draws on proprietary enterprise context. The concept spans territorial, operational, stack, legal and unit economics concerns, according to Amit Eyal Govrin<\/a>, chief executive officer of Agentcy Labs Inc., and Philip Rathle<\/a>, chief technology officer of Neo4j Inc.<\/p>\n

\u201cSovereignty is exerting agency and control over your AI,\u201d Govrin said during the event. \u201cYou have to be free and clear of state, economic and threat actors overtaking any level of control over your stack. You\u2019re not paying rent to somebody else.\u201d<\/p>\n

Knowledge graphs add another form of control by allowing some decisions to run deterministically rather than relying entirely on probabilistic models. That can give enterprises consistent business rules, explainability and governance alongside the flexibility of large language models, Rathle noted.<\/p>\n

\u201cHaving the capacity to do AI with a full brain, both hemispheres, is ultra important,\u201d he told theCUBE. \u201cLLMs are spontaneous, creative \u2014 they make mistakes, you don\u2019t know why. Having the graph as the left brain to the LLM right brain is really at the core of where graphs fit in.\u201d<\/p>\n

Here\u2019s theCUBE\u2019s complete video interview with Amit Eyal Govrin and Philip Rathle:<\/p>\n

To watch more of theCUBE\u2019s coverage of RAISE Summit, here\u2019s our complete video playlist<\/a>:<\/p>\n

(* Disclosure: TheCUBE is a paid media partner for the RAISE Summit event. Neither Solidigm, the headline sponsor of theCUBE\u2019s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)<\/p>\n

Photo: SiliconANGLE<\/p>\n

Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE\u2019s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.<\/p>\n

15M+ viewers of theCUBE videos, powering conversations across AI, cloud, cybersecurity and more
\n11.4k+ theCUBE alumni \u2014 Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network.<\/p>\n

About SiliconANGLE Media<\/p>\n

SiliconANGLE Media is a recognized leader in digital media innovation, uniting breakthrough technology, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE<\/a>, theCUBE Network<\/a>, theCUBE Research<\/a>, CUBE365<\/a>, theCUBE AI<\/a> and theCUBE SuperStudios \u2014 with flagship locations in Silicon Valley and the New York Stock Exchange \u2014 SiliconANGLE Media operates at the intersection of media, technology and AI.<\/p>\n

Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.<\/p>\n","protected":false},"excerpt":{"rendered":"Agentic inference is reshaping the center of gravity in AI infrastructure. What began as a race to scale…\n","protected":false},"author":2,"featured_media":107701,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[179,7493,396,55710,45672],"class_list":["post-107700","post","type-post","status-publish","format-standard","has-post-thumbnail","category-agentic-ai","tag-agentic-ai","tag-agentic-artificial-intelligence","tag-siliconangle","tag-three-insights-you-may-have-missed-from-thecubes-coverage-of-raise-summit","tag-victoria-gayton"],"_links":{"self":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/107700","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/comments?post=107700"}],"version-history":[{"count":0,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/107700\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media\/107701"}],"wp:attachment":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media?parent=107700"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/categories?post=107700"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/tags?post=107700"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}