Internal Deployment and Hardware Ambitions
Following the reveal of OpenAI’s Jalapeño ASIC, questions emerged regarding the company's hardware ambitions. While OpenAI partnered with Broadcom to build custom silicon for its own compute needs, it also showcased benchmarks comparing Jalapeño to Nvidia’s Blackwell accelerators. Richard Ho, Head of Hardware at OpenAI, stated that while the chip is built for internal use, it could theoretically be used elsewhere, leaving the door open for a wider rollout.
“We have such a strong demand for compute within the company. It's going to take us a good long time to even fill our own demand, which is growing all the time,” Ho said. “I think that we're going to have our hands full just providing compute for OpenAI for a good long time. That's not to say that it can't be used elsewhere. I believe it could be, but I think our priority is to make sure that OpenAI's compute needs are met first and foremost.”
Benchmark Performance and Compatibility
The competitive positioning of Jalapeño stems from benchmarks shared during Hot Chips, run on SemiAnalysis’ InferenceX benchmark against Nvidia’s GB200 and GB300. OpenAI demonstrated the custom ASIC accelerating its own open-weight GPT-OSS model, as well as DeepSeek R1 and Kimi K2.5.
Originally, OpenAI had not planned to show benchmarks at Hot Chips. Engineers successfully got Kimi and DeepSeek running on Jalapeño within the two-month window between receiving the A0 sample and the presentation.
Although Ho confirmed Jalapeño is deployed internally, he emphasized its broad software compatibility. Speaking on the Hot Chips benchmarks, Ho noted that the goal was to dispel the notion that the custom inference chip is hard-coded solely for OpenAI models, proving it to be programmable and general-purpose.
Supply Constraints and Efficiency Goals
A primary factor limiting an external hardware release is supply capacity. Ho noted that OpenAI has established a new baseline for supply after extensive engagement with fabs, but supplying external customers would present a distinct challenge.
Design Philosophy and Codesign
While Jalapeño accommodates other models, raw competitive performance was secondary to power efficiency. Ho highlighted efficiency as a critical requirement for managing power-constrained modern AI data centers.
Ho also emphasized the advantages of hardware-software codesign with internal models, noting that sharing sensitive research intellectual property with third-party silicon merchants introduces security risks that standard non-disclosure agreements cannot fully mitigate.
Future Comparisons and Vera Rubin Platform
If Jalapeño were deployed commercially, it would likely compete against Nvidia’s newer Vera Rubin platform rather than Grace Blackwell. Ho explained that Blackwell was used for published comparisons because those were the best available results at the time, though internal testing extends to Vera Rubin and larger context windows.
The public InferenceX benchmarks evaluated fixed 8,000 input and 1,000 output token configurations. According to Ho, internal benchmarks indicate the ASIC performs even more favorably under broader conditions, with promising initial comparisons to Vera Rubin.
“Obviously, by the time we deploy, it’ll be Vera Rubin, maybe even VR Ultra in some parts of the deployment schedule. We’ve done our internal ones, but obviously we don’t publish those. Those have to come from Nvidia and other people who are able to do that.… yeah, we’re doing really well on those,” Ho said.
