Jalapeno chip tests prove the feasibility of breaking the traditional trade-off between latency and ultra-high throughput in processing massive inference models.
Article index:
- Benchmark results and a direct challenge to Nvidia’s dominance
- Rack-scale mega-processing architecture
- Development speed and AI adoption
- Deployment roadmap and future production
- Technical notes and widespread competition
- Frequently asked questions
Benchmark results and a direct challenge to Nvidia’s dominance
On Tuesday, OpenAI released detailed benchmark results showing that its dedicated inference chip, “Jalapeno,” outperforms products from Nvidia and other chipmakers in overall throughput and energy efficiency. This marks the clearest and most comprehensive public challenge launched by the leading company against Nvidia’s absolute dominance in the smart computing hardware market.
The proactive release of this data preceded OpenAI’s official presentation at the Hot Chips semiconductor conference at Stanford University. The figures showed that the Jalapeno chip executes AI workloads 1.5 to 1.9 times better per watt of energy than Nvidia’s GB200 and GB300 systems, while reducing overall latency by 1.7 to 3.6 times when running advanced models such as DeepSeek-R1, the one-trillion-parameter version of Kimi K2.5, and OpenAI’s open-source model GPT-OSS 120B.
Rack-scale mega-processing architecture
Developed in close partnership with global firm Broadcom, the chip was designed as an application-specific integrated circuit (ASIC) fully customized for large language model inference tasks. At the clustered server level, each server rack consists of 128 compute accelerators delivering a staggering compute capacity of 1.7 exaflops at 4-bit precision, supported by 27.5 terabytes of High Bandwidth Memory (HBM4) and a memory bandwidth approaching 2 petabytes per second.
Richard Ho, vice president of hardware at OpenAI, told reporters that the chip provides “the best of both worlds” by combining low latency and high throughput, whereas traditional AI systems are forced to sacrifice one for the other. According to independent research firm SemiAnalysis, which tested the chip using the InferencePerf benchmark suite inside OpenAI’s labs, Jalapeno achieved processing speeds exceeding 700 tokens per second per user at a single concurrency level on the DeepSeek-R1 model, without relying on speculative decoding techniques or disaggregating prefill from decoding, methods competitors use to boost their figures.
Development speed and AI adoption
OpenAI managed to transition from the initial hiring phase of the hardware team to completing the final tape-out design in just 16 months, delivering the design to factories in November 2025. The company explained that leveraging AI-assisted circuit design tools helped reduce the footprint of multi-instruction units by 8% and the matrix engine area by 10%.
Despite these achievements, the chip remains dedicated exclusively to inference and model execution and does not support training new models. Richard Ho commented on this by saying: “We are going to need massive amounts of compute capacity in the future, and the Jalapeno chip is part of that solution, as is the Cerberus project, Nvidia hardware, and AMD chips as well.”
Deployment roadmap and future production
The company plans to deploy the chip on a limited scale within its data centers by the end of 2026, with mass commercial production beginning in 2027. OpenAI also revealed that development on the second generation of the chip is already underway, with preliminary plans for a third generation. Ho emphasized that OpenAI does not plan to sell the chip to outside parties, stating: “We have such a dire internal need for it that we cannot imagine when we would be able to sell it to others.”
Technical notes and widespread competition
SemiAnalysis noted technical caveats, pointing out that the tests were based on processing scenarios comprising 8,000 input tokens and 1,000 output tokens, which differ from complex agent workloads that stress cache memory and routing. They also noted that a comparison with Nvidia’s newer Vera Rubin architecture would be more equitable. The chip joins similar efforts by tech giants such as Google TPUs, custom designs from Microsoft and Amazon, and Anthropic’s proprietary chip project.
Frequently asked questions
Q: What is the Jalapeno chip and who developed it?
A: It is a dedicated inference computing chip developed by OpenAI in cooperation with Broadcom to accelerate the execution of massive language models with exceptional energy efficiency.
Q: How does the chip outperform current Nvidia systems?
A: It delivers 1.5 to 1.9 times higher efficiency per watt and reduces latency by up to 3.6 times compared to Nvidia GB200 and GB300 systems.
Q: Will OpenAI sell the chip to external developers?
A: No, the company confirmed that all produced chips will be used exclusively to meet the massive internal computing needs of its data centers and intelligent systems.
