Supermicro.com
Every enterprise AI deployment that runs into cost ceilings, sluggish response times, or hard limits on how many simultaneous conversations an agent can sustain is hitting the same invisible wall: the GPU is being asked to be both the brain and the filing cabinet, and it cannot do both well at the same time. The real AI infrastructure bottleneck in 2026 is not compute — it is memory architecture. The Supermicro Open Storage Summit — a free virtual event that will open tomorrow, August 11, and run through September 3 — is structured around exactly that problem, bringing 38 industry experts from 21 companies together to map how enterprise AI infrastructure needs to be rebuilt for the agentic era. The full summit agenda and registration are available via Supermicro’s official press release.
The core technical shift the summit addresses is straightforward to state and difficult to engineer: as large language models generate output, they accumulate a key-value (KV) cache — a memory structure that stores the intermediate attention states for every token already processed in a conversation. Without that cache, the model would have to recompute its full attention over the conversation history for each new word it generates, turning a linear operation into a quadratic one. With the cache stored on GPU high-bandwidth memory (HBM), inference stays fast — but only up to a point. At 128,000-token contexts and beyond, that cache can claim the majority of an H100 GPU’s 80 gigabytes of HBM, leaving inadequate room for model weights and concurrent user sessions. Agentic AI, which maintains growing task histories across many reasoning steps, makes this pressure structural rather than occasional.
The solution the industry is converging on is offloading the KV cache from GPU HBM to a dedicated storage tier — what the summit’s Session 7 calls a “context memory” layer, positioned between the GPU’s local NVMe SSDs and network-attached storage arrays. Nvidia formalized this architecture in January 2026 with its Inference Context Memory Storage Platform, now known as CMX, managed by the BlueField-4 DPU. The CMX partner ecosystem includes nearly every company presenting at this summit: VAST Data, DDN, IBM, Nutanix, WEKA, Supermicro, and Cloudian are all on Nvidia’s published partner list. The summit is, in significant measure, this ecosystem convening to show enterprise practitioners how to put CMX-aligned architectures into production before their AI deployments run out of room to grow.
“Enterprise AI success depends on a data infrastructure that can scale with growing model complexity and inference demands,” said Michael McNerney, Senior Vice President of Marketing and Network Security at Supermicro. Rob Strechay, Principal Analyst at theCUBE, framed the commercial stakes: “As enterprises look to become token producers rather than perpetual token consumers, storage becomes a strategic platform — not simply for capacity, but for feeding GPUs, managing context, accelerating inference, and enabling trusted AI at scale.”
The timing is pointed. Supermicro will hold its Q4 fiscal year 2026 Q4 earnings call August 11, the same day Session 1 airs. In July, the company disclosed a preliminary update: Q4 revenue is expected near the low end of its $11.0 billion to $12.5 billion guidance range, but gross margins — estimated at 15 to 17 percent — will come in dramatically above prior guidance of 8.2 to 8.4 percent. Most significantly, Supermicro disclosed that its backlog reached record levels after receiving more than $60 billion in new orders during Q4 alone — orders that will need to be deployed into working AI infrastructure over the quarters ahead. The summit’s 12 sessions represent, in part, Supermicro’s effort to help customers turn that backlog into functioning, scalable AI deployments rather than expensive hardware waiting for an architectural decision.
All 12 sessions will air live at 10:00 AM PDT (1 PM ET) on their scheduled dates and will be available on-demand through theCUBE immediately following each broadcast. Registration is free for data center architects, storage engineers, and enterprise IT professionals. Register and view the schedule at the theCUBE event portal.
What the Context Memory Gap Actually Is — and Why It Matters Now
In a standard transformer inference run, the model allocates HBM on the GPU to store the KV cache for the entire active conversation. Every new token of output extends the cache, which must remain in memory for the decode phase to proceed without recomputation. For a chatbot handling a 4,000-token conversation, this is manageable. For an enterprise AI agent working through a lengthy legal document, debugging a complex codebase, or maintaining a customer service history across multiple sessions, the context can easily exceed 100,000 tokens — and at that scale, the cache’s memory footprint competes directly with the model’s weights for the same fixed pool of HBM. The KV cache memory exhaustion problem is the structural reason long-context agentic AI underperforms at scale.
The result is a pattern that infrastructure teams are increasingly recognizing: GPU clusters that look fully occupied in monitoring dashboards but are actually experiencing throughput degradation from memory pressure, not from compute saturation. A cluster throttled by KV cache memory pressure delivers higher cost-per-token and lower requests-per-second than the GPU specifications imply.
Offloading KV cache to a dedicated external storage tier breaks that constraint. In testing on four NVIDIA H100 GPUs, storage systems specifically architected for KV cache offload delivered a 19x Time to First Token improvement at a 131,000-token context compared to GPU-only KV cache management. The practical meaning: AI agents can handle far longer tasks and maintain far richer context without GPU exhaustion forcing truncation or recomputation.
Nvidia’s CMX platform, announced at CES in January, standardizes how this offload works: a four-tier memory hierarchy (GPU HBM → CPU DRAM → local NVMe SSDs → networked external storage arrays) managed by the BlueField-4 DPU and accessed by Nvidia’s Dynamo inference framework. Nearly every storage company at this summit has either already shipped or is actively developing a CMX-compatible storage implementation — making the summit sessions as close to a live industry field guide for this architecture as currently exists.
How Do Enterprises Choose Storage for AI Inference?
The answer the summit gives across its first three sessions is: it depends on where in the pipeline the data lives, how fast it needs to move, and how much it costs to keep it there.
Session 1 (August 11, with WEKA, Scality, Samsung, WD, and Supermicro) presents the canonical multi-tier approach: a high-performance parallel file system built on all-flash media for latency-sensitive inference serving, layered against S3-compatible object storage primarily backed by high-capacity HDDs for cold data, long-tail model weights, and archived inference logs. The economics follow directly from physics: all-flash storage delivers microsecond-latency random reads at a dollar-per-gigabyte cost roughly 10 to 20 times higher than HDDs; object storage on HDDs trades latency for cost and scale. Matching each data type to the right tier — rather than storing everything on the most expensive medium — is where meaningful cost optimization occurs.
Session 2 (August 12, with DDN, KIOXIA, and WD) examines this tradeoff through two vertical use cases that represent the extremes: financial trading systems, which demand sub-millisecond response and audit-grade data integrity, and genomic sequencing pipelines, which require petabyte-scale unstructured sequence data to be ingested under research compliance constraints. The infrastructure design is not the same for both, and the session is specifically organized to show how the same core technology components — enterprise SSDs from KIOXIA, high-capacity HDDs from WD, and DDN’s parallel file system — are configured differently for each workload.
Session 3 (August 13, with Nutanix, MinIO, and PEAK:AIO) steps back from the media question to address what separates AI deployments that make it to production from those that stall: cost and token economics, infrastructure that can scale from workgroup to enterprise, data readiness and governance requirements, and the access controls that security teams demand before they approve a production rollout.
Breaking the Context Wall: What Session Seven Will Explain
Session 7 (August 25, with VAST Data, Solidigm, and Supermicro) carries the technical argument to its logical conclusion. The session introduces “context memory” as a named storage tier — something the AI industry did not have a common vocabulary for as recently as 18 months ago — and walks through implementation architectures for systems that persist KV cache data in a large-scale storage array rather than allowing it to be evicted from GPU memory between conversation turns or reasoning steps.
The Solidigm contribution is not incidental. Solidigm manufactures enterprise NVMe SSDs — the physical medium that makes the context memory tier economically viable. The performance characteristics of enterprise NVMe storage (sustained sequential read bandwidth measured in gigabytes per second, queue depth for handling simultaneous KV cache read requests, endurance ratings for write-intensive cache workloads) determine whether the offload tier performs at the latency required for inference. An SSD optimized for enterprise database workloads is not automatically appropriate for KV cache serving — the access pattern is different.
VAST Data’s contribution is its context-aware KV cache routing software, which implements routing logic that determines which storage node holds the relevant cached attention states for an incoming inference request and retrieves them with minimum latency. Supermicro provides the all-flash storage arrays that serve as the physical hardware layer for both companies’ software implementations.
Neocloud Infrastructure: How Crusoe Builds Storage for AI at GPU Scale
Session 4 (August 18, with Crusoe, VAST Data, and KIOXIA) examines a question the hyperscaler model cannot fully answer: how do AI-native cloud providers differentiate their storage architecture from AWS and Azure when they are competing for workloads that AWS and Azure are not optimized to serve?
Crusoe — which built OpenAI’s Stargate campus in Abilene, Texas, and was valued at roughly $10 billion in an October 2025 funding round — is one of the defining examples of what the storage industry calls a neocloud: a GPU-as-a-service provider built entirely around AI compute, competing on AI-native design rather than general-purpose cloud services. Neocloud revenue as a category is projected to reach roughly $20 billion in 2026, growing toward $180 billion by 2030, driven by the fact that hyperscalers cannot build AI data centers fast enough to meet demand.
The storage design for a neocloud is not simply a hyperscaler storage design at smaller scale. The workload profile is more homogeneous (AI training and inference rather than general-purpose cloud), the network topology is different (InfiniBand-heavy rather than Ethernet-primary), and the ratio of flash storage to spinning disk is inverted from what most enterprise storage teams work with. Session 4 will explore the specific architectural choices — flash media selection, data management platform design, KV cache handling — that define how neoclouds differentiate their AI-serving economics from hyperscalers.
IBM, KIOXIA, and the Time-to-First-Token Equation
Session 5 (August 19, with IBM, KIOXIA, and Supermicro) focuses on a metric that has become the primary user-visible benchmark for AI inference quality: Time to First Token, or TTFT — the interval between when a user submits a query and when the first word of the model’s response appears.
TTFT is governed not just by the model’s computational throughput but by how quickly the KV cache for the user’s conversation context can be loaded from wherever it is stored into the GPU’s working memory for the decode phase. IBM’s AI Data Platform architecture addresses this by managing KV cache placement across Supermicro all-flash storage systems, using KIOXIA enterprise NVMe flash as the physical medium. The session will explain how thoughtful placement of cached attention states — keeping frequently accessed context closer to the GPU and less-accessed context on higher-capacity, slightly slower tiers — can reduce TTFT and cut recomputation overhead for long-context inference workloads.
Data Lakes Are Becoming AI Infrastructure
Session 8 (August 26, with MinIO, AMD, and Supermicro) and Session 9 (August 27, with DDN, Solidigm, and Supermicro) together address a practical question facing most enterprise data teams: they already have a data lake. What do they need to change about it to serve AI workloads?
Data lakes built for analytics and data warehousing rest on object storage and open formats — Apache Parquet, Delta Lake, Apache Iceberg — for interoperability. What makes these formats attractive for analytics (immutability, columnar layout, schema-on-read) creates friction for AI workflows that need low-latency random access, frequent writes, and transactional capabilities for managing pipeline state. The lakehouse transactional layer approach — adding a transactional layer on top of object storage using open table formats — is the industry’s current answer.
Session 8 will examine how MinIO’s S3-compatible object storage and AMD’s compute platforms combine with Supermicro systems to extend existing lake investments into AI pipelines rather than requiring greenfield replacement. Session 9 addresses AI data platforms as a distinct product category: pre-integrated appliances that handle ingestion, normalization, vectorization, and hand-off to AI inference environments, with built-in workflows for retrieval-augmented generation (RAG), visual search, and document processing. Unstructured data — documents, images, video, logs, and sensor streams — constitutes an estimated 80 to 90 percent of the unique data an enterprise generates, yet most organizations retain only a small fraction of it. AI changes the economics of that retention decision, and Sessions 8 and 9 address what the infrastructure implications of keeping more of it actually look like.
Edge, Hybrid, and the Remaining Three Sessions
Session 6 (August 20, with Hammerspace, SanDisk, and Supermicro) addresses the reality that most enterprise data centers are not greenfield: they operate existing storage systems, often from multiple vendors and spanning multiple generations, that predate AI by years or decades. Hammerspace’s global namespace technology provides one approach to bridging legacy and new storage systems without forcing a complete rip-and-replace.
Session 10 (September 1, with Iron Mountain, Scality, and Supermicro) will feature CSP Iron Mountain walking through the next phase of its Iron Cloud offering — specifically, an on-premises edge storage node providing low-latency local backup and retrieval while tiering data to Iron Mountain’s cloud backend for long-term retention. The solution runs on Scality object storage and Supermicro edge systems, representing a model for hybrid storage-as-a-service that satisfies enterprise SLA requirements for local performance while delivering cloud economics for retention.
Session 11 (September 2, with Cloudian, Hammerspace, Seagate, and Supermicro) will return to unstructured data management at scale — specifically the policy and tiering architectures that make economically viable long-term retention possible. Session 12 (September 3, with Nutanix, AMD, and Supermicro) will close the summit with a broader examination of how enterprises can move from AI-curious to AI-operational across the full stack: data readiness, hardware and software selection, on-premises versus hybrid trade-offs, and the operational and economic realities of running agentic AI in production.
Registration and Schedule
All 12 sessions are free to attend and require no enterprise affiliation. Sessions air live at 10:00 AM PDT (1:00 PM ET) on their respective dates and will be available on-demand through theCUBE immediately following each broadcast. Registration is available through the Supermicro event page or the theCUBE event portal.
The complete schedule: Session 1 (Aug. 11), Session 2 (Aug. 12), Session 3 (Aug. 13), Session 4 (Aug. 18), Session 5 (Aug. 19), Session 6 (Aug. 20), Session 7 (Aug. 25), Session 8 (Aug. 26), Session 9 (Aug. 27), Session 10 (Sept. 1), Session 11 (Sept. 2), Session 12 (Sept. 3).
Frequently Asked QuestionsWhat is the “context memory tier” and why is it the focus of this summit?
The context memory tier is a new storage architecture layer designed to hold a large language model’s key-value (KV) cache — the record of everything the model has processed in a conversation or task — in networked NVMe storage rather than in GPU high-bandwidth memory. GPU HBM is fast but finite: at long context lengths (100,000 tokens or more), the KV cache can consume the majority of available GPU memory, crowding out model weights and reducing the number of concurrent users a cluster can serve. By moving the KV cache to a dedicated external storage tier, enterprises can run longer AI agent tasks without GPU exhaustion, improve cost-per-token economics, and scale to more simultaneous sessions. Nvidia formalized this architecture in January 2026 with its CMX context memory platform; the summit’s Session 7 (August 25) is specifically organized around implementing it in production.
Why does enterprise storage now directly constrain what AI agents can do?
When an AI agent maintains context across many reasoning steps — tracking a long document review, managing a multi-turn customer support case, or executing a complex debugging workflow — each step adds to an accumulating KV cache that must stay available for the model to continue without recomputing from scratch. If GPU memory cannot hold the full cache, the agent either truncates its context (losing information it needs) or recomputes (adding latency and cost). Storage that can hold and serve that cache at the speed inference requires effectively extends the agent’s working memory. This is why several analysts and infrastructure teams describe AI storage not as a place data lives but as the medium through which AI agents think — and why storage latency shapes AI agent capability to AI agent capability.
How is a neocloud like Crusoe different from AWS or Azure for AI workloads?
A neocloud is a cloud provider built entirely around GPU compute for AI, rather than a general-purpose cloud platform that also offers GPU instances. Neoclouds differentiate on AI-native architecture choices — network topology designed for GPU cluster interconnect (InfiniBand-heavy), storage systems optimized for AI access patterns rather than general-purpose cloud workloads, and pricing models built around GPU-hours rather than per-service metering. Crusoe, which built OpenAI’s Stargate campus in Abilene, Texas, also differentiates on sustainability, sourcing power from stranded and renewable energy. The neocloud revenue and market projections as a category is projected to reach roughly $20 billion in revenue in 2026. Session 4 (August 18) examines specifically how neocloud storage design enables economics that general-purpose clouds cannot match for AI-heavy workloads.
What should I look for in AI storage performance benchmarks?
The most relevant benchmark for AI inference storage is Time to First Token (TTFT) — the interval between submitting a query and receiving the model’s first output word — as a function of context length. Standard storage benchmarks (IOPS, sequential throughput, latency at queue depth 1) do not capture the specific access pattern of KV cache serving, which involves high-concurrency random reads at the granularity of individual attention head tensors. A meaningful AI storage benchmark will state the model size, the context window length, the number of concurrent users, and the hardware configuration (GPU count and type, NVMe tier capacity and bandwidth). Dell’s benchmarking of KV cache offload on H100 systems showed a 19x TTFT improvement at 131,000-token context — a specific, reproducible result that illustrates the magnitude of the gap between GPU-only and storage-tier-assisted inference.