OpenAI says its first custom inference chip, Jalape\u00f1o, can deliver more AI work per watt while also reducing response times, pointing to a hardware design aimed at handling increasingly demanding model workloads more efficiently.<\/p>\n
The company tested Jalape\u00f1o across three public models: GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. Across the tests, the chip delivered 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the comparison systems.<\/p>\n
For highly interactive workloads, OpenAI said Jalape\u00f1o achieved 2.1 to 4.1 times higher performance. The company argues that combining throughput and low latency in one architecture could help reduce the hardware and power needed to serve AI models.<\/p>\n
The chip is rated at 700 watts, although OpenAI said its measured sustained power remained at or below 550 watts during the tested workloads. The company compared Jalape\u00f1o with commercially available accelerator systems using the public InferenceX benchmark from SemiAnalysis.<\/p>\n
One chip tackles both bottlenecks<\/p>\n
A major focus of the design is avoiding the usual tradeoff between throughput and latency. AI inference has different demands depending on what the system is doing. Processing a user\u2019s prompt, known as prefill, is generally compute-intensive, while generating the response token by token, known as decode, depends more heavily on memory bandwidth.<\/p>\n
Jalape\u00f1o was designed to handle both phases within the same architecture. OpenAI said the chip keeps model state, including the key-value cache used during generation, closer to the processing resources that need it.<\/p>\n
Its networking system is also integrated into the architecture to reduce the amount of data that needs to move between chips. That is intended to limit communication delays that can leave computing resources waiting for data.<\/p>\n
The approach becomes particularly relevant for AI agents, which may perform many inference steps in sequence. A small delay in each step can add up to a much longer overall task.<\/p>\n
AI helped build Jalape\u00f1o<\/p>\n
OpenAI also used its own models during the chip\u2019s development. The company said AI helped engineers explore implementations, shorten design and verification cycles, and optimize arithmetic circuits.<\/p>\n
The team moved from initial design to tapeout in nine months. OpenAI then used Codex with GPT-Astra to bring three open-weight models<\/a> that were not part of the original production plan to high performance on Jalape\u00f1o within two months.<\/p>\n For selected GPT-OSS attention and mixture-of-experts blocks, AI-generated implementations ran 1.5 to 1.8 times faster than existing implementations written by human experts. OpenAI stressed that these results applied to individual blocks rather than the complete models.<\/p>\n The company also tested Jalape\u00f1o against large models including Kimi K2.5 1T. On that model, it reported about 1.5 times higher peak performance per watt and 3.4 times lower end-to-end latency than the comparison system.<\/p>\n OpenAI plans to begin deploying Jalape\u00f1o in its compute infrastructure <\/a>by the end of 2026. The company said the chip is the first generation of a multigenerational roadmap, with second- and third-generation designs already in development.<\/p>\n