OpenAI says its first custom inference chip, Jalapeño, can deliver more AI work per watt while also reducing response times, pointing to a hardware design aimed at handling increasingly demanding model workloads more efficiently.
The company tested Jalapeño across three public models: GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. Across the tests, the chip delivered 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the comparison systems.
For highly interactive workloads, OpenAI said Jalapeño achieved 2.1 to 4.1 times higher performance. The company argues that combining throughput and low latency in one architecture could help reduce the hardware and power needed to serve AI models.
The chip is rated at 700 watts, although OpenAI said its measured sustained power remained at or below 550 watts during the tested workloads. The company compared Jalapeño with commercially available accelerator systems using the public InferenceX benchmark from SemiAnalysis.
One chip tackles both bottlenecks
A major focus of the design is avoiding the usual tradeoff between throughput and latency. AI inference has different demands depending on what the system is doing. Processing a user’s prompt, known as prefill, is generally compute-intensive, while generating the response token by token, known as decode, depends more heavily on memory bandwidth.
Jalapeño was designed to handle both phases within the same architecture. OpenAI said the chip keeps model state, including the key-value cache used during generation, closer to the processing resources that need it.
Its networking system is also integrated into the architecture to reduce the amount of data that needs to move between chips. That is intended to limit communication delays that can leave computing resources waiting for data.
The approach becomes particularly relevant for AI agents, which may perform many inference steps in sequence. A small delay in each step can add up to a much longer overall task.
AI helped build Jalapeño
OpenAI also used its own models during the chip’s development. The company said AI helped engineers explore implementations, shorten design and verification cycles, and optimize arithmetic circuits.
The team moved from initial design to tapeout in nine months. OpenAI then used Codex with GPT-Astra to bring three open-weight models that were not part of the original production plan to high performance on Jalapeño within two months.
For selected GPT-OSS attention and mixture-of-experts blocks, AI-generated implementations ran 1.5 to 1.8 times faster than existing implementations written by human experts. OpenAI stressed that these results applied to individual blocks rather than the complete models.
The company also tested Jalapeño against large models including Kimi K2.5 1T. On that model, it reported about 1.5 times higher peak performance per watt and 3.4 times lower end-to-end latency than the comparison system.
OpenAI plans to begin deploying Jalapeño in its compute infrastructure by the end of 2026. The company said the chip is the first generation of a multigenerational roadmap, with second- and third-generation designs already in development.
The broader goal is to make inference faster and more power-efficient as OpenAI expands its computing infrastructure and serves increasingly capable models.