![\"Chip](\"https:\/\/www.eetimes.com\/wp-content\/uploads\/image_7249ec.jpeg\")
<\/a>Chip timeline: OpenAI emphasized the speed of development of its Jalape\u00f1o chip. (Source: OpenAI)<\/p>\nTo quantify Jalape\u00f1o\u2019s advantages, Ho said in his briefing that OpenAI leaned on InferenceX, a public benchmark from SemiAnalysis that measures the full serving path for AI requests. On three models\u2014OpenAI\u2019s own GPT-OSS-120B, as well as external models like Kimi K2.5 1T and DeepSeek R1 670B\u2014OpenAI reports that the Jalape\u00f1o chip delivers:<\/p>\n
\t\t\t\t\t By Unikey Electronics Pte. Ltd.\u00a0 08.26.2026<\/p>\n \t\t\t\t\t By Zackary Glazewski, Founding AI Engineer, ChipAgents\u00a0\u00a0 08.26.2026<\/p>\n \t\t\t\t\t By Colin Duggan, CEO and co-founder, BG Networks\u00a0\u00a0 08.26.2026<\/p>\n 1.5 to 1.9\u00d7 more AI work per watt at peak throughput versus comparable systems listed on InferenceX.<\/p>\n 1.7 to 3.6\u00d7 lower end\u2011to\u2011end latency.<\/p>\n For highly interactive workloads, 2.1 to 4.1\u00d7 higher performance than comparable systems.<\/p>\n Ho also draws attention to a methodological nuance: OpenAI reports single\u2011token prediction (STP) performance, without speculative multi\u2011token prediction (MTP) decode, while many public numbers include that optimization.<\/p>\n \u201cOur numbers are without speculative decode. Something called STP, meaning doing single token prediction, versus some of the numbers you\u2019ll see out there are actually multiple token predictions, which in essence result in about a three to five times boost,\u201d Ho said. \u201cSo, our single token prediction is actually as good or beats some of the multi token prediction of the other models of the other chips.\u201d<\/p>\n When comparing architectures, he said this distinction matters. OpenAI suggests Jalape\u00f1o is competitive even before you turn on aggressive, potentially model\u2011sensitive inference optimizations.<\/p>\n Not just a repurposed GPU; it\u2019s a general-purpose processor for AI workloads<\/p>\n During this week\u2019s briefing, Richard Ho emphasized that the new chip is not a repurposed legacy GPU, but a ground-up AI accelerator tailored specifically for transformer workloads. Designed around tight memory-compute co-design, it anchors a multi-generation roadmap. He also reiterated, as in the June announcement, that they\u2019d used AI to accelerate the development of the Jalape\u00f1o chip.<\/p>\n \u00a0\u201cWe basically started with a blank sheet of paper,\u201d Ho said. \u201cWe came in and we looked at the LLM models, where the bottlenecks were, where the inner loops were, and what was going on, and I think we might be the first like large scale chip design team to really do that from scratch without a legacy architecture, without a legacy programming model they had to support.\u201d<\/p>\n The result is a general\u2011purpose accelerator for AI workloads, rather than something narrowly tuned to OpenAI\u2019s own models. In the benchmarks, he had emphasized that Jalape\u00f1o ran both internal and external\/open models, highlighting it as a flexible platform rather than a single\u2011model engine.<\/p>\n Data movement is a critical bottleneck<\/p>\n Ho added that the defining problem in LLM compute is not just peak TFLOPs, but how much data has to move, and how far.<\/p>\n \u201cData movement was super critical,\u201d he said, and became the starting point for the architecture. \u201cIf you\u2019re forced to move that data around to different cores, it just takes up a lot of time and energy. So, we designed an architecture which minimizes that. There is close affinity between HBM banks and cores, and so if you\u2019re able to take advantage of that in your programming model, then you eke out a lot of benefits.\u201d<\/p>\n This \u201caffinity\u201d between HBM banks and compute cores is a key design point. Rather than treating the memory system as a flat, uniform pool, Jalapeno binds memory regions closely to compute resources, with the software stack aware enough to exploit that locality.<\/p>\n That means model state, including the KV cache used while generating a response, is explicitly placed and kept local while the system activates the right combination of compute, memory, and networking for each inference phase. The network is integral to the architecture and allows the entire workload to remain within one connected system.<\/p>\n According to OpenAI, the result is a balanced, versatile accelerator designed to adapt to shifting model architectures. It excels across both prefill and decode phases and dynamically adjusts to the fluctuating demands characteristic of agentic workloads.<\/p>\n On using AI to build a chip that can be programmed by AI<\/p>\n The nine-month tape-out milestone cited by OpenAI was achieved with the use of AI to design the chip itself. AI-driven workflows accelerated implementation exploration, tightened design, measurement, and verification cycles, and enabled continuous iteration across model workloads.<\/p>\n In his briefing, Ho said that Jalape\u00f1o is an example of AI assisted hardware design. OpenAI used earlier generation models to assist with chip design and bring up, and is now using its latest models to accelerate software optimization and kernel tuning. According to Ho, AI contributed both to arithmetic circuit optimization and to packing more compute into the die.<\/p>\n He noted that the team was able to bring up three separate LLMs on Jalape\u00f1o in about two months\u2014a timeline he cited as evidence of both the architecture\u2019s programmability and the effectiveness of AI assisted workflows.<\/p>\n Getting to roofline performance without thermal throttling<\/p>\n In his briefing, Ho talked about balancing compute, memory bandwidth, and networking to get very close to the roofline performance without thermal throttling. He said thermals and power are clearly part of the balancing act, and the Jalape\u00f1o chip targets about 700 W per accelerator, which is aggressive but still within the envelope of modern data center cooling solutions.<\/p>\n Ho also stressed that the system is designed to avoid the kind of power\u2011driven throttling engineers have grown accustomed to in high\u2011end GPU deployments.<\/p>\n \u201cIt doesn\u2019t thermally throttle. There\u2019s almost no thermal throttling here because it is balanced. It\u2019s coming in at about 700 watts, so it\u2019s well within the range of most cooling solutions.\u201d<\/p>\n Roadmap, deployment<\/p>\n Ho re-iterated that Jalape\u00f1o isn\u2019t a one\u2011off project, and that it\u2019s the first in a multi\u2011generation roadmap. He said Gen 2 is already in deep development and Gen 3 is \u201ctaking shape.\u201d Each generation builds on what the team learns and aims to further advance both efficiency and speed.<\/p>\n On deployment, OpenAI plans to start small\u2011volume rollouts at the end of this year, with a more significant ramp in 2027. Jalape\u00f1o will coexist with Nvidia hardware and other devices within OpenAI\u2019s fleet. In the near term, Ho expects Jalape\u00f1o to be particularly attractive for low\u2011latency inference, such as ultra\u2011responsive code generation and other interactive workloads, where the combination of reduced data movement, memory affinity, and high bandwidth can be fully exploited.<\/p>\n Ho also emphasized that external sales aren\u2019t the focus for the Jalape\u00f1o chip. OpenAI\u2019s own compute demand is growing fast enough that simply meeting internal needs will consume capacity for the foreseeable future.<\/p>\n This was also underscored by Sarah Friar, CFO of OpenAI, in a blog<\/a> this week: she said that, while Microsoft\u2019s compute and Nvidia\u2019s chips were foundational to OpenAI\u2019s growth, its portfolio of chips and providers it relies on also includes AWS, AMD, Broadcom, Cerebras, CoreWeave, Oracle, SB Energy and SoftBank.<\/p>\n \u00a0\u201cWe actively manage this portfolio for both capability and economics. We use premium systems where capability matters most and optimize for efficiency where scale and cost matter more,\u201d she said in the post. \u201cPreserving credible choice across providers, hardware, and deployment models lets us direct demand toward the strongest performance per dollar, maintain pricing discipline as market conditions change, and move with the frontier as stronger technology emerges. Direct control adds leverage where tighter integration can improve the entire system. We partner where the ecosystem helps us move faster and build where co-design creates a meaningful advantage.\u201d<\/p>\n This is a balanced way of saying, as with the broader hyperscaler fraternity, they all need so much AI compute to satisfy demand that, in the short to medium term, all options are open and required right now. Hence, they won\u2019t be replacing the GPU incumbents just yet. But with everyone developing their own ASICs and custom chips, there may just come a point in time when the power\/performance tradeoffs push one solution over the tipping point into being the preferred approach.<\/p>\n Read also:<\/p>\n OpenAI Hardware Chief: \u2018AI Scaling Laws Will Continue\u2019<\/a><\/p>\n NXP Expands Industrial Endpoint Access with MCU Topology Discovery<\/a><\/p>\n Andes Condor Closure Came Amid Broader Cost-Cutting Effort<\/a><\/p>\n AI AND BIG DATA<\/a>, CHIPLETS<\/a>, SEMICONDUCTOR<\/a>, SOC<\/a>\n <\/p>\n<\/a><\/p>\n<\/a><\/p>\n<\/a><\/p>\n<\/a>OpenAI presented several benchmark performance results, for example, this one illustrating peak and matched throughput. (Source: OpenAI)<\/p>\n<\/a>The pros and cons of speculative decoding using STP vs. MTP (Source: OpenAI)<\/p>\n<\/a>The Jalape\u00f1o chip is a general-purpose AI accelerator designed for performance\/Watt. (Source: OpenAI)<\/p>\n<\/a>Minimizing data movement is key to optimizing performance. (Source: OpenAI)<\/p>\n<\/a>The entire workload remains within one connected system, minimizing data movement and helping the complete request to stay fast and efficient from beginning to end. (Source: OpenAI)<\/p>\n<\/a>The nine-month tape-out milestone was achieved using AI to design the chip itself. (Source: OpenAI)<\/p>\n