OpenAI Jalapeno (custom inference accelerator)¶
Scope: OpenAI's first-party inference ASIC, announced with Broadcom on 2026-06-24 and detailed at Hot Chips on 2026-08-25. Covers what the part is, who fabs and packages it, its memory and interconnect, the two-rack system topology, how the published comparisons against NVIDIA parts are constructed, and the operator-facing consequences (software stack maturity, workload fit, supply risk, cost reasoning). NVIDIA's own specifications are not restated here: see Blackwell platform, the NVIDIA GPU roadmap, and GPU generations. Roofline theory lives in arithmetic intensity, the FP4 formats in NVFP4 quantization, and the prefill/decode split in disaggregated inference.
Source and paywall status. The primary analyst source (SemiAnalysis, 2026-08-25) is a paid newsletter. Its body is readable up to the heading "Direct Implications for NVIDIA, AMD and Cerebras"; everything from that heading onward, plus the production-volume and ASP figures in its subscriber models, is paywalled and is not represented on this page. Every chart in the readable portion is a rasterised image, so the plotted values could not be read; the quantitative comparisons below therefore come from OpenAI's own published appendix, not from those charts. Vendor primary sources (OpenAI, Broadcom, NVIDIA, JEDEC, Hugging Face) were fetched directly and are cited per claim.
Evidence. The numpy block under Comparison arithmetic was executed and its output pasted byte for byte; it asserts ridge points, three ranking flips, and reproduces OpenAI's own published multipliers from OpenAI's own operands. Its cost inputs are illustrative parameters, not vendor pricing: no list price for any of these parts has been published. Nothing on this page was measured on hardware. All Jalapeno performance figures are vendor-reported or analyst-reported and none has been independently reproduced. Note also that the measured results and the peak-FLOPS figure are different silicon: every benchmark below was run on the A0 stepping, while the 13.4 PFLOPS peak used in all roofline and TCO arithmetic is the B0 stepping, which is still in the fab.
What it is¶
Jalapeno is an application-specific accelerator designed by OpenAI for LLM inference only, implemented with Broadcom and integrated into racks by Celestica. Broadcom's press release calls it "OpenAI's first Intelligence Processor" and describes it as "the first AI accelerator in a multi-generation compute platform the companies are building together."4 OpenAI frames it as "a blank-slate design for modern LLM inference, not a general-purpose accelerator adapted from earlier AI workloads."3
Silicon and packaging, as reported by SemiAnalysis from the Hot Chips disclosure:1
| Property | Reported value | Confidence |
|---|---|---|
| Compute die | One reticle-sized die on TSMC N3P | Analyst, from Hot Chips |
| Peak compute | 13.4 PFLOPS MXFP4 (B0 stepping) | Analyst |
| Package TDP | 700 W | Analyst, corroborated by OpenAI2 |
| Measured sustained power | at or below 550 W on the tested workloads | OpenAI2 |
| Memory | HBM4, 15.4 TB/s per package at ~10 Gbps pin | Analyst |
| Memory capacity | not disclosed in the readable source text | Gap |
| I/O chiplet | N3E, 32 lanes of 800G SerDes | Analyst |
| Scale-up | 24 lanes (600 GB/s) local, 8 lanes (200 GB/s) global | Analyst |
| Host I/O | PCIe Gen 5 to an x86 host | Analyst |
| Tape-out | CoWoS tape-out November 2025 | Analyst |
Two steppings, and this page mixes them. SemiAnalysis states "All these results were gathered on the A0 stepping of Jalapeño" and, separately, that "the B0 stepping delivers 13.4 PFLOPs of MXFP4 on a single reticle-sized compute die," with B0 "currently in the fab" and roughly 25% better in performance per watt than A0.1 Every measured figure on this page (OpenAI's results table, all latencies and time-between-tokens values, the 700 tokens per second per user) is A0 silicon. The 13.4 PFLOPS peak that drives the ridge point, the roofline table, the arithmetic-intensity crossover and both TCO bars is B0. The roofline arithmetic below therefore credits the part with a peak the measured results did not have, and no B0 measurement has been published.
The matrix engine is a weight-stationary systolic array using MXFP numeric formats, alongside 64-bit scalar cores and FP32/INT32 vector cores. SemiAnalysis reports two design choices that separate it from TPU-class parts and from GPUs: support for small matrix dimensions, so awkwardly shaped matmuls do not hit the tiling cliffs that large systolic arrays impose, and an out-of-order core with an L1 cache rather than the software-managed scratchpad plus async DMA that other accelerators use.1 The stated aim is to remove fixed overheads (launch latency, barrier latency, memory-system latency) that force GPUs to amortise work over large batches.
The two memory-system numbers are not independent of one another. SemiAnalysis derives the pin rate from the bandwidth: "This results in 15.4TB/s of memory bandwidth per package ... The 15.4TB/s bandwidth shows its HBM4 can hit 10Gbps pin speeds".1 Solving that same relation for the remaining unknown, which is what the executed block below does, recovers only the stack count the analyst silently assumed: 15.4 TB/s at 10 Gbps fixes the package at six 2048-bit HBM4 stacks (15.36 TB/s). That is a consistency check on a single reported figure, not corroboration by two. No second source confirms the bandwidth or the pin rate, and the stack count is disclosed nowhere. JEDEC's HBM4 standard specifies a 2048-bit interface with transfer speeds "up to 8 Gb/s," so a 10 Gbps pin rate is above the JEDEC baseline; JEDEC has published no statement of a rate above 8 Gb/s.8
Timeline, and a conflict in the record¶
OpenAI and Broadcom both state the program went "from initial design to manufacturing tape-out in just nine months."43 SemiAnalysis states "Design work began in the middle of 2024, going from initial team hiring to manufacturing tape-out in ~16 months."1 These anchor on different events (initial design versus initial team hiring) and are not necessarily contradictory, but the two figures should not be quoted interchangeably. Neither has been independently verified.
Why use it¶
The design target is tokens per unit of power, not tokens per chip. OpenAI states the normalisation explicitly: "Although performance is sometimes reported per chip, we believe the more useful standard is performance per unit of power."2 SemiAnalysis gives the reason as a facility constraint rather than a silicon one: adding accelerators and adding grid capacity happen on different timescales, so a datacenter's utility interconnection, cooling capacity, and backup generation cap the fleet long before budget does.1
That framing decides the whole comparison. As the executed block shows, a buyer constrained by chip count and a buyer constrained by megawatts rank the same two parts differently over a band of realised-throughput values. Vendor perf-per-watt claims are only decision-relevant for the second buyer.
OpenAI's published results, measured on SemiAnalysis's InferenceX benchmark at a nominal 8k/1k input/output split with single-token prediction, normalised by package TDP:2
| Model | Baseline | Peak mixed TPS/kW | End-to-end latency | Min TBT | Throughput at the baseline's best TBT (mixed tokens/kW) |
|---|---|---|---|---|---|
| GPT-OSS 120B | GB200 (1,200 W) | 85,448 vs 44,960 (1.9x) | 1.03 s vs 1.80 s (1.7x) | 0.69 vs 1.87 ms (2.7x) | 22,935 vs 427 (53.7x) |
| DeepSeek R1 670B | GB300 (1,400 W) | 19,641 vs 11,781 (1.7x) | 1.65 s vs 5.99 s (3.6x) | 1.43 vs 5.90 ms (4.1x) | 12,258 vs 118 (104.3x) |
| Kimi K2.5 1T | GB300 (1,400 W) | 18,195 vs 11,862 (1.5x) | 1.56 s vs 5.31 s (3.4x) | 1.44 vs 5.48 ms (3.8x) | 6,744 vs 120 (56.1x) |
Four things about that table matter more than the multipliers:
- The baselines are Blackwell, not Rubin. SemiAnalysis says so itself: "we believe that comparison to Blackwell is somewhat incomplete and unfair. Jalapeño is really competing against chips like Rubin that also use HBM4."1 NVIDIA's Rubin generation publishes 22 TB/s of HBM4 per package, which is more bandwidth than Jalapeno's 15.4 TB/s, and the executed block shows that reverses the memory-bound result.
- The denominator is package TDP, not all-in power. Host CPUs, scale-up switches, optics, power delivery, and cooling are excluded from both sides. A tokens-per-utility-megawatt figure would include them and is not derivable from this table.
- The large multipliers are iso-interactivity, not peak. The 53.7x and 104.3x columns compare Jalapeno's throughput at the baseline's best achievable time-between-tokens. They measure how far the baseline falls off its own latency cliff, not a throughput ratio at a common operating point.
- Speculative decoding is off on the Jalapeno side. SemiAnalysis notes the Jalapeno results use single-token prediction with no speculative decoding and no prefill/decode disaggregation, while the NVIDIA comparison points use multi-token prediction.1 That cuts both ways: it understates Jalapeno's headroom and it means the two stacks are not doing the same work.
When to use it (and when not)¶
For almost every reader of this page the answer is "not available." Jalapeno is first-party silicon with no announced merchant channel. OpenAI states it plans "to begin deploying Jalapeño within OpenAI's compute infrastructure by the end of the year," and Broadcom frames deployment as "gigawatt scale data centers with Microsoft and other partners beginning in 2026."24 SemiAnalysis reports production ramping gradually across 2027 with most output scheduled for the end of that year.1 Treat availability outside OpenAI and its named partners as unannounced.
Where the architecture is a good fit, on the published evidence:
- Low-concurrency, latency-sensitive decode. This is the strongest result and the one the architecture is built for. The removal of fixed per-launch and barrier overheads is what produces the 53x to 104x iso-interactivity numbers, and it is the regime where GPUs are worst.
- Mixed and shifting traffic. The homogeneous pool (below) is chosen precisely so the prefill/decode ratio can move without stranding hardware.
- MoE models at MXFP4. The matrix engine is MXFP-native, and gpt-oss-120b ships MXFP4 weights natively.10 Models whose reference weights are NVFP4 need a conversion step; the format difference is covered in NVFP4 quantization.
Where it is not the right tool, or where the evidence does not reach:
- Training. The part is inference-only by design. Nothing in any cited source claims a training capability.
- Long-context, multi-turn agentic serving. SemiAnalysis is explicit that the published runs are single-turn 8k/1k and that no results exist on its longer AgentX suite: "they're just 8k1k, a much easier workload to tune for, and there are no AgentX runs yet."1 Prefix-cache behaviour, router behaviour, and offload paths are untested in public.
- Frontier-scale open models. SemiAnalysis notes the tested models "are not on the open frontier" relative to what NVIDIA and AMD publish on.1
- Any capacity-bound deployment. HBM capacity per package is not disclosed, so the number of packages a given model requires cannot be computed from public information.
Architecture¶
Two racks form the deployment unit: a host rack of 16 CPU trays ("Katsu") and an ASIC rack of 16 accelerator trays ("Vindaloo") plus 8 scale-up switch trays ("Chana"). Each Vindaloo tray carries 8 Jalapeno ASICs, giving 128 per rack. Each Katsu tray holds two Turin-class AMD EPYC CPUs with 1.5 TB of DRAM and connects to its paired Vindaloo tray over 8 external PCIe DAC cables. SemiAnalysis reports roughly 50 kW provisioned for the host rack (31 kW in production) and 130 kW for the ASIC rack, about 160 kW for the pair.1
flowchart TB
subgraph HOST["Host rack: about 50 kW provisioned, 31 kW in production"]
K["16x Katsu tray<br/>2x AMD EPYC Turin, 1.5 TB DRAM<br/>400G (2x200G) frontend NIC"]
end
subgraph ASIC["ASIC rack: about 130 kW"]
V["16x Vindaloo tray<br/>8x Jalapeno ASIC each = 128 per rack"]
CL["6x Chana local switch tray<br/>1x Tomahawk 6 (102.4 Tb/s) each"]
CG["2x Chana global switch tray<br/>believed 2x Tomahawk 6 each"]
end
K -->|"8x external PCIe Gen5 DAC per tray pair"| V
V -->|"4.8 Tb/s per XPU, all-to-all, copper backplane"| CL
V -->|"1.6 Tb/s per XPU, copper backplane"| CG
CG -->|"1.6T transceivers to optical circuit switch"| OCS["Global domain: 8-rail<br/>16 racks = 2,048 XPU scale-up world"]The scale-up network is split in two. The local domain wires all 128 ASICs in a rack all-to-all across six Tomahawk 6 switch ASICs over a passive copper backplane, at 4.8 Tb/s unidirectional per XPU. The global domain extends the scale-up world to 2,048 XPUs across 16 racks at 1.6 Tb/s per XPU, using a rail-only 8-rail architecture and, in SemiAnalysis's reading, optical circuit switches in every rack.1 Broadcom's Tomahawk 6 is a real, shipping part: 102.4 Tb/s on a single chip in 3 nm, with 512 instances of 200G PAM4 SerDes or 1,024 of 100G, and Broadcom's own materials cite "scale-up clusters with 512 XPUs."7 General fabric background is in networking fabric and datacenter interconnect.
Inside the chip, cores and HBM are divided into slices, each core slice holding a low-latency local view of its own HBM slice, with a separate high-bandwidth collective network for cross-slice synchronisation and a general NoC for everything else.1 The design bet is that careful placement of weights and KV keeps synchronisation restricted to predictable tensor-parallel traffic that overlaps with compute.
No prefill/decode disaggregation¶
Jalapeno runs a homogeneous pool: prefill and decode share chips, and the draft model for speculation shares chips and fabric with the target. SemiAnalysis reports being surprised by this, since NVIDIA and AMD GPUs benefit significantly from disaggregation even on homogeneous hardware.1 The stated reasoning is workload drift: once devices are split into pools, the operator must continuously predict the right ratio, and "an entire chip can sit idle simply because it belongs to the wrong pool." A fungible fleet trades some local efficiency for global utilisation that survives a changing traffic mix.
This is a genuine architectural disagreement with the direction most GPU serving stacks have taken, and the counter-arguments are unchanged: see disaggregated inference and rate matching. SemiAnalysis concedes the case: disaggregation "can still win where demand is sufficiently large, stable, and predictable."1 Note also that the choice is not free of consequence for the benchmark: the published Jalapeno numbers were produced without disaggregation and without speculative decoding, so they are a floor for that stack, not a tuned peak.
Comparison arithmetic¶
The block below is self-contained and was executed. It uses vendor-published NVIDIA figures for the merchant side rather than analyst figures, checks the internal consistency of the analyst memory claims, reproduces OpenAI's own published multipliers from OpenAI's own operands, and asserts three separate ranking flips: across the roofline crossover, between decode and prefill, and on the choice of merchant baseline. The die-versus-package reconciliation in section 3b is a units identity rather than a fourth flip, and the block says so: 35 divided by 17.5 is exactly 2 by construction, so that check cannot fail and proves nothing on its own. Cost inputs are labelled illustrative and are not sourced pricing.
"""Custom-silicon vs merchant-GPU accelerator arithmetic for OpenAI Jalapeno.
Roofline ridge points, the decode/prefill ranking flip, reproduction of OpenAI's
own published perf ratios, and a TCO crossover.
Every SOURCED constant carries its origin tag. Every ILLUSTRATIVE constant is a
parameter of the model, not a vendor claim, and is labelled inline.
[S1] SemiAnalysis, "OpenAI Jalapeno: Better Than Nvidia Blackwell", 2026-08-25,
free pre-paywall body. Analyst reporting of an OpenAI Hot Chips 2026
disclosure, NOT a vendor datasheet.
[S2] OpenAI, "Jalapeno's first results show industry-leading speed and
efficiency in AI inference", openai.com/index/jalapeno-first-results/,
Appendix. Vendor-published, vendor-run.
[S3] NVIDIA GB300 NVL72 product page, nvidia.com/en-us/data-center/gb300-nvl72/
spec table: 72 Blackwell Ultra GPUs, "GPU Memory | Bandwidth 20 TB | Up to
576 TB/s", "FP4 Tensor Core 1440 | 1080 PFLOPS" (sparse | dense).
[S4] DeepSeek-R1 model card, huggingface.co/deepseek-ai/DeepSeek-R1:
671B total / 37B activated parameters.
[S5] NVIDIA Vera Rubin NVL72 product page,
nvidia.com/en-us/data-center/vera-rubin-nvl72/ spec table, "Rubin GPU"
column: NVFP4 Inference 50 PFLOPS; NVFP4 Training 35 PFLOPS (footnote 2
"Dense specification"); FP8/FP6 Training 17.5 PFLOPS (same footnote 2);
"GPU Memory | Bandwidth 288 GB HBM4 | 22 TB/s".
All rows governed by footnote 1: "Preliminary information. All values are
up to and subject to change." No per-package TDP is published.
[S6] NVIDIA developer blog, "Inside NVIDIA Rubin GPU Architecture", 2026-07-21:
"The Rubin GPU (Figure 2) is constructed from reticle limited compute dies
to achieve high density and efficiency. These two dies are unified on a
single package through a high-speed inter-die link called the NVIDIA
High-Bandwidth Interface (NV-HBI)."
Run: python3 openai-jalapeno-accelerator.py (numpy only)
"""
import numpy as np
# --------------------------------------------------------------------------
# SOURCED constants
# --------------------------------------------------------------------------
JAL_FLOPS = 13.4e15 # FLOP/s MXFP4, B0 stepping, one reticle-sized N3P die [S1]
JAL_BW = 15.4e12 # byte/s HBM4 per package [S1]
JAL_TDP = 700.0 # W package TDP [S1][S2]
JAL_MEASURED_W = 550.0 # W sustained on the tested workloads [S2]
# Blackwell Ultra (B300) per GPU, derived from the GB300 NVL72 rack spec [S3].
B300_FLOPS = 1080e15 / 72 # 15.0 PFLOP/s dense NVFP4 [S3]
B300_BW = 576e12 / 72 # 8.0 TB/s HBM3E [S3]
B300_TDP = 1400.0 # W package TDP, as used by OpenAI to normalise [S2]
# Rubin, per PACKAGE (NVIDIA's unit). A Rubin package carries two compute dies.
RUBIN_PKG_DENSE_FLOPS = 35e15 # dense NVFP4 training [S5]
RUBIN_PKG_BW = 22e12 # byte/s HBM4 per package [S5]
RUBIN_DIE_FLOPS_SA = 17.5e15 # SemiAnalysis, "per Rubin compute die" [S1]
HBM4_BITS_PER_STACK = 2048 # JEDEC HBM4 per-stack interface width
# InferenceX, nominal 8k/1k, single-token prediction, normalised by PACKAGE TDP
# (Jalapeno 700 W; GB200 1,200 W; GB300 1,400 W). [S2]
OPENAI_APPENDIX = {
# model: (peakPerkW_jal, peakPerkW_base, stated_ratio,
# e2e_lat_base_s, e2e_lat_jal_s, stated_ratio,
# minTBT_jal_ms, minTBT_base_ms, stated_ratio,
# isoTBT_jal, isoTBT_base, stated_ratio, baseline, baseline_TDP_W)
"GPT-OSS-120B": (85448, 44960, 1.9, 1.80, 1.03, 1.7,
0.69, 1.87, 2.7, 22935, 427, 53.7, "GB200", 1200.0),
"DeepSeek-R1-670B": (19641, 11781, 1.7, 5.99, 1.65, 3.6,
1.43, 5.90, 4.1, 12258, 118, 104.3, "GB300", 1400.0),
"Kimi-K2.5-1T": (18195, 11862, 1.5, 5.31, 1.56, 3.4,
1.44, 5.48, 3.8, 6744, 120, 56.1, "GB300", 1400.0),
}
def ridge_point(flops: float, bw: float) -> float:
"""Arithmetic intensity (FLOP/byte) at which a part stops being bandwidth-bound."""
assert flops > 0.0 and bw > 0.0, "ridge point undefined for non-positive peaks"
return flops / bw
def roofline(intensity: np.ndarray, flops: float, bw: float) -> np.ndarray:
"""Attainable FLOP/s at a given arithmetic intensity."""
return np.minimum(flops, intensity * bw)
def decode_ceiling(bw: float, active_params: float, bytes_per_param: float) -> float:
"""Upper bound on decode tokens/s from HBM bandwidth alone: weights stream
once per token. Ignores compute, collectives and launch overhead, so it is a
ceiling, never a prediction."""
bytes_per_token = active_params * bytes_per_param
assert bytes_per_token > 0.0, "a model with zero bytes per token is not a model"
return bw / bytes_per_token
def prefill_ceiling(flops: float, active_params: float) -> float:
"""Upper bound on prefill tokens/s from the compute roof: 2 FLOP per activated
parameter per token."""
assert active_params > 0.0
return flops / (2.0 * active_params)
# ==========================================================================
print("=" * 78)
print("1. HBM4 stack count implied by the reported bandwidth and pin rate [S1]")
print("=" * 78)
# [S1] does NOT report the pin rate independently of the bandwidth, it DERIVES one
# from the other: "The 15.4TB/s bandwidth shows its HBM4 can hit 10Gbps pin speeds".
# Solving the same relation for the remaining unknown therefore recovers only the
# stack count [S1] silently assumed. This is a consistency check on ONE claim.
jal_stacks = JAL_BW * 8.0 / (HBM4_BITS_PER_STACK * 10.0e9)
print(f" Jalapeno: {JAL_BW/1e12:.2f} TB/s at 10.0 Gbps pin "
f"-> {jal_stacks:.3f} x 2048-bit HBM4 stacks")
assert abs(jal_stacks - 6.0) < 0.05, jal_stacks
exact = 6 * HBM4_BITS_PER_STACK * 10.0e9 / 8.0
print(f" 6 stacks x 2048 bit x 10 Gbps = {exact/1e12:.2f} TB/s "
f"(reported {JAL_BW/1e12:.1f} TB/s)")
assert abs(exact / 1e12 - 15.36) < 0.01
# Round-tripping the derivation is the proof that the two figures are one claim:
# dividing the reported bandwidth over 6 stacks returns the pin rate [S1] states.
implied_pin = JAL_BW * 8.0 / (6 * HBM4_BITS_PER_STACK)
assert abs(implied_pin / 1e9 - 10.0) < 0.05, implied_pin
print(f" Round trip: {JAL_BW/1e12:.1f} TB/s over 6 stacks = {implied_pin/1e9:.2f} Gbps "
f"pin, the rate [S1]")
print(" reports. The pin rate is DERIVED from the bandwidth, so this fixes the implied")
print(" stack count at six and corroborates NEITHER number independently.")
print(f" Jalapeno HBM bandwidth advantage over B300: "
f"{JAL_BW/B300_BW:.2f}x per package, "
f"{(JAL_BW/JAL_TDP)/(B300_BW/B300_TDP):.2f}x per watt")
# ==========================================================================
print()
print("=" * 78)
print("2. Ridge points and the roofline crossover [S1][S3]")
print("=" * 78)
jal_ridge = ridge_point(JAL_FLOPS, JAL_BW)
b300_ridge = ridge_point(B300_FLOPS, B300_BW)
print(f" Jalapeno ridge = {jal_ridge:7.1f} FLOP/byte "
f"({JAL_FLOPS/1e15:.1f} PFLOP/s / {JAL_BW/1e12:.1f} TB/s)")
print(f" B300 ridge = {b300_ridge:7.1f} FLOP/byte "
f"({B300_FLOPS/1e15:.1f} PFLOP/s / {B300_BW/1e12:.1f} TB/s)")
# The lower ridge saturates its compute roof at lower intensity: it tolerates
# memory-bound work better. This is the whole design thesis of the part.
assert jal_ridge < b300_ridge, "expected Jalapeno to have the lower ridge point"
assert abs(jal_ridge - 870.1) < 0.5 and abs(b300_ridge - 1875.0) < 0.5
ai = np.array([1.0, 100.0, jal_ridge, 1000.0, 1675.0, 3000.0])
jal_att = roofline(ai, JAL_FLOPS, JAL_BW)
b300_att = roofline(ai, B300_FLOPS, B300_BW)
print()
print(" AI (FLOP/B) Jalapeno PFLOP/s B300 PFLOP/s winner")
for a, j, b in zip(ai, jal_att, b300_att):
print(f" {a:10.1f} {j/1e15:16.3f} {b/1e15:12.3f} "
f"{'Jalapeno' if j > b else ('B300' if b > j else 'tie')}")
# Below both ridges bandwidth decides; far above both, peak FLOPS decides.
assert jal_att[0] > b300_att[0], "at AI=1 the higher-bandwidth part must win"
assert jal_att[1] > b300_att[1]
assert jal_att[-1] < b300_att[-1], "at AI=3000 the higher-FLOPS part must win"
assert JAL_FLOPS < B300_FLOPS
# Exact crossover: Jalapeno is already on its compute roof while B300 is still on
# its memory diagonal, so they tie where AI * B300_BW == JAL_FLOPS.
cross = JAL_FLOPS / B300_BW
assert jal_ridge < cross < b300_ridge, (jal_ridge, cross, b300_ridge)
lo_j = roofline(np.array([cross * 0.99]), JAL_FLOPS, JAL_BW)[0]
lo_b = roofline(np.array([cross * 0.99]), B300_FLOPS, B300_BW)[0]
hi_j = roofline(np.array([cross * 1.01]), JAL_FLOPS, JAL_BW)[0]
hi_b = roofline(np.array([cross * 1.01]), B300_FLOPS, B300_BW)[0]
assert lo_j > lo_b and hi_j < hi_b, "the ranking must flip across the crossover"
print()
print(f" Per-chip ranking flips at AI = {cross:.1f} FLOP/byte "
f"(Jalapeno peak FLOPS / B300 bandwidth)")
print(f" AI={cross*0.99:7.1f}: Jalapeno {lo_j/1e15:6.3f} > B300 {lo_b/1e15:6.3f} PFLOP/s")
print(f" AI={cross*1.01:7.1f}: Jalapeno {hi_j/1e15:6.3f} < B300 {hi_b/1e15:6.3f} PFLOP/s")
print()
print(f" Peak compute per watt : Jalapeno {JAL_FLOPS/JAL_TDP/1e12:6.2f} "
f"TFLOP/s/W vs B300 {B300_FLOPS/B300_TDP/1e12:6.2f} "
f"({(JAL_FLOPS/JAL_TDP)/(B300_FLOPS/B300_TDP):.2f}x)")
print(f" Peak bandwidth per watt: Jalapeno {JAL_BW/JAL_TDP/1e9:6.2f} "
f"GB/s/W vs B300 {B300_BW/B300_TDP/1e9:6.2f} "
f"({(JAL_BW/JAL_TDP)/(B300_BW/B300_TDP):.2f}x)")
# Per watt there is NO flip: the 2x TDP gap swamps the 12% FLOPS deficit.
assert JAL_FLOPS / JAL_TDP > B300_FLOPS / B300_TDP
assert JAL_BW / JAL_TDP > B300_BW / B300_TDP
# ==========================================================================
print()
print("=" * 78)
print("3. Decode is bandwidth-bound, prefill is compute-bound: ranking flips")
print("=" * 78)
ACTIVE_PARAMS = 37e9 # DeepSeek-R1 activated parameters [S4]
TOTAL_PARAMS = 671e9 # DeepSeek-R1 total parameters [S4]
BYTES_PER_PARAM = 0.5 # MXFP4, the format [S2] used for the R1 runs
bytes_per_token = ACTIVE_PARAMS * BYTES_PER_PARAM
jal_dec = decode_ceiling(JAL_BW, ACTIVE_PARAMS, BYTES_PER_PARAM)
b300_dec = decode_ceiling(B300_BW, ACTIVE_PARAMS, BYTES_PER_PARAM)
jal_pre = prefill_ceiling(JAL_FLOPS, ACTIVE_PARAMS)
b300_pre = prefill_ceiling(B300_FLOPS, ACTIVE_PARAMS)
print(f" DeepSeek-R1: {TOTAL_PARAMS/1e9:.0f}B total / {ACTIVE_PARAMS/1e9:.0f}B "
f"active, MXFP4 -> {bytes_per_token/1e9:.1f} GB read per decode step")
print(f" Decode ceiling (bandwidth-bound) : Jalapeno {jal_dec:9.1f} tok/s "
f"B300 {b300_dec:9.1f} tok/s")
print(f" Prefill ceiling (compute-bound) : Jalapeno {jal_pre:9.0f} tok/s "
f"B300 {b300_pre:9.0f} tok/s")
assert jal_dec > b300_dec, "memory-bound decode must favour the higher-bandwidth part"
assert b300_pre > jal_pre, "compute-bound prefill must favour the higher-FLOPS part"
r_dec, r_pre = jal_dec / b300_dec, jal_pre / b300_pre
print(f" Jalapeno / B300 per chip: decode {r_dec:.3f}x vs prefill {r_pre:.3f}x")
assert r_dec > 1.0 > r_pre, "THE PER-CHIP RANKING FLIPS between decode and prefill"
# Per watt, Jalapeno leads BOTH phases: the flip is a per-chip result only, and an
# operator who is power-limited rather than chip-limited never sees it.
pw_dec = (jal_dec / JAL_TDP) / (b300_dec / B300_TDP)
pw_pre = (jal_pre / JAL_TDP) / (b300_pre / B300_TDP)
print(f" Jalapeno / B300 per watt: decode {pw_dec:.3f}x vs prefill {pw_pre:.3f}x "
f"(no flip)")
assert pw_dec > 1.0 and pw_pre > 1.0, "per watt there must be no flip"
# Capacity, not bandwidth, decides how many packages the model needs at all.
weights_gb = TOTAL_PARAMS * BYTES_PER_PARAM / 1e9
print(f" R1 weights at MXFP4 = {weights_gb:.0f} GB, so the model must be sharded;"
f" Jalapeno HBM CAPACITY is not disclosed in the free source text")
# Sanity-check the measured number against the ceiling.
MEASURED_R1 = 700.0 # tok/s/user at min TBT on DeepSeek-R1 [S2]
print(f" Measured {MEASURED_R1:.0f} tok/s/user is {MEASURED_R1/jal_dec*100:.1f}% "
f"of ONE package's bandwidth roofline")
assert MEASURED_R1 < jal_dec, "a measured rate above the single-package ceiling " \
"would falsify the sharding assumption"
frac_tp32 = MEASURED_R1 / (32 * jal_dec) # [S1] reports a TP32 rack config
print(f" At the reported TP32 config it is {frac_tp32*100:.2f}% of the aggregate "
f"roofline -> latency-bound, not bandwidth-bound")
assert frac_tp32 < 0.05, frac_tp32
# ==========================================================================
print()
print("=" * 78)
print("3b. Die vs package: the comparison unit decides the winner [S1][S5][S6]")
print("=" * 78)
# SemiAnalysis compares Jalapeno's single reticle-sized die against a single
# "Rubin compute die" at 17.5 PFLOPS dense NVFP4. NVIDIA publishes 35 PFLOPS
# dense NVFP4 per Rubin GPU PACKAGE. The evidence for 2 dies per package is
# NVIDIA's own statement [S6], NOT the division below: 35 / 17.5 is exactly 2 by
# construction, so that check cannot fail and corroborates nothing on its own.
RUBIN_DIES_PER_PKG = 2 # [S6] "These two dies are unified on a single package"
implied_dies = RUBIN_PKG_DENSE_FLOPS / RUBIN_DIE_FLOPS_SA
print(f" NVIDIA per package {RUBIN_PKG_DENSE_FLOPS/1e15:.0f} PFLOPS dense NVFP4 "
f"/ SemiAnalysis per die {RUBIN_DIE_FLOPS_SA/1e15:.1f} = "
f"{implied_dies:.1f} dies per package")
assert implied_dies == RUBIN_DIES_PER_PKG, implied_dies # consistent, not evidence
print(f" Two dies per package is NVIDIA's own statement [S6], not this division;")
print(f" {RUBIN_PKG_DENSE_FLOPS/1e15:.0f}/{RUBIN_DIE_FLOPS_SA/1e15:.1f} is exactly "
f"{RUBIN_DIES_PER_PKG} by construction and cannot fail.")
# And 17.5 PFLOPS alone does not even identify the unit: NVIDIA's own Vera Rubin
# spec table publishes 17.5 PFLOPS dense FP8/FP6 training PER RUBIN GPU PACKAGE.
# The bare number is ambiguous between "analyst per die" and "vendor per package".
RUBIN_PKG_FP8_FLOPS = 17.5e15 # [S5] "FP8/FP6 Training" row, "NVIDIA Rubin GPU" column
assert RUBIN_PKG_FP8_FLOPS == RUBIN_DIE_FLOPS_SA
print(f" Caution: {RUBIN_PKG_FP8_FLOPS/1e15:.1f} PFLOPS is ALSO NVIDIA's published "
f"dense FP8/FP6 per")
print(" Rubin GPU PACKAGE [S5], so the number alone is not diagnostic of the unit.")
# Consequence: at DIE level Jalapeno leads on bandwidth; at PACKAGE level Rubin
# does, because a Rubin package feeds two dies from 22 TB/s of HBM4.
jal_vs_b300 = JAL_BW / B300_BW
jal_vs_rubin = JAL_BW / RUBIN_PKG_BW
print(f" Jalapeno package bandwidth vs B300 package : {jal_vs_b300:.2f}x (leads)")
print(f" Jalapeno package bandwidth vs Rubin package: {jal_vs_rubin:.2f}x (trails)")
assert jal_vs_b300 > 1.0 > jal_vs_rubin, \
"THE RANKING FLIPS ON THE CHOICE OF MERCHANT BASELINE, not just the workload"
rub_dec = decode_ceiling(RUBIN_PKG_BW, ACTIVE_PARAMS, BYTES_PER_PARAM)
print(f" R1 decode ceiling per package: Jalapeno {jal_dec:.1f} tok/s, "
f"B300 {b300_dec:.1f}, Rubin {rub_dec:.1f}")
assert b300_dec < jal_dec < rub_dec
print(" So the memory-bound win reported against HBM3E Blackwell parts does NOT")
print(" carry over to HBM4 Rubin packages. NVIDIA publishes no Rubin package TDP,")
print(" so the per-watt version of this comparison cannot be computed here.")
# ==========================================================================
print()
print("=" * 78)
print("4. Reproducing OpenAI's own published ratios from its own operands [S2]")
print("=" * 78)
# The stated multipliers are printed to 1 decimal place, so the correct test is
# "does the ratio of the displayed operands round to the displayed multiplier",
# not a fixed relative tolerance.
print(" model metric computed rounded stated")
mismatches = []
for model, v in OPENAI_APPENDIX.items():
checks = [("perf/W", v[0] / v[1], v[2]),
("e2e latency", v[3] / v[4], v[5]),
("min TBT", v[7] / v[6], v[8]),
("iso-TBT", v[9] / v[10], v[11])]
for label, got, stated in checks:
ok = round(got, 1) == stated
flag = "" if ok else " <-- does not reproduce"
print(f" {model:19s} {label:11s} {got:9.3f} {round(got,1):9.1f} "
f"{stated:9.1f}{flag}")
if not ok:
mismatches.append((model, label, got, stated))
# 10 of 12 reproduce exactly under the source's own rounding. Both failures are the
# iso-INTERACTIVITY ratio (OpenAI's own label: "More throughput at previous TBT"),
# whose denominator is displayed to 3 significant figures (118, 120) while the
# headline was clearly computed on the raw value.
assert len(mismatches) == 2, mismatches
assert {m[1] for m in mismatches} == {"iso-TBT"}, mismatches
assert {m[0] for m in mismatches} == {"DeepSeek-R1-670B", "Kimi-K2.5-1T"}
assert abs(dict((m[0], m[2]) for m in mismatches)["DeepSeek-R1-670B"] - 103.88) < 0.01
print(f" ASSERT ok: 10 of 12 reproduce exactly; the 2 that do not are both iso-TBT")
for m in mismatches:
print(f" {m[0]:19s} computed {m[2]:.2f} vs stated {m[3]}")
print(" (consistent with the headline being computed on unrounded operands)")
# The denominator is PACKAGE TDP, not all-in system power. Charging Jalapeno its
# own measured sustained draw instead would RAISE every ratio.
print()
for model, v in OPENAI_APPENDIX.items():
rescaled = (v[0] * JAL_TDP / JAL_MEASURED_W) / v[1]
print(f" {model:19s} vs {v[12]}: {v[0]/v[1]:5.2f}x at 700 W TDP -> "
f"{rescaled:5.2f}x at the measured {JAL_MEASURED_W:.0f} W")
assert rescaled > v[0] / v[1]
print(" Neither figure is tokens per all-in utility watt: host CPUs, scale-up")
print(" switches, optics and cooling are excluded from BOTH sides.")
# ==========================================================================
print()
print("=" * 78)
print("5. TCO: which scarce resource binds decides the winner")
print(" COST INPUTS BELOW ARE ILLUSTRATIVE PARAMETERS, NOT VENDOR PRICING")
print("=" * 78)
# No list price for Jalapeno, B300 or Rubin has ever been published. These are
# editable parameters chosen to be plausible, and nothing on this page depends
# on their exact values: the STRUCTURE of the result is what is asserted.
COST_CUSTOM = 16_000.0 # $/accelerator, in-house ASIC at cost ILLUSTRATIVE
COST_MERCHANT = 40_000.0 # $/accelerator, merchant GPU with margin ILLUSTRATIVE
AMORT_YEARS = 4.0 # ILLUSTRATIVE
PUE = 1.25 # ILLUSTRATIVE
POWER_USD_KWH = 0.07 # ILLUSTRATIVE
HOURS_PER_YEAR = 8760.0
# Sourced per-chip decode ceilings from section 3. The custom part runs at a
# DERATE of its roofline because its kernels are three months old; the merchant
# part is credited its full roofline. The derate is the free variable.
CUSTOM_ROOFLINE = jal_dec # 832.4 tok/s [S1] bandwidth, [S4] model
MERCHANT_ROOFLINE = b300_dec # 432.4 tok/s [S3] bandwidth, [S4] model
def annual_cost(chip_cost: float, tdp_w: float, util: float) -> float:
"""Amortised capex plus energy for one accelerator over one year."""
assert 0.0 < util <= 1.0, "utilisation must be in (0, 1]"
capex = chip_cost / AMORT_YEARS
energy = tdp_w * util * PUE * HOURS_PER_YEAR / 1000.0 * POWER_USD_KWH
return capex + energy
# --- 5a. Energy is NOT the dominant term at any realistic tariff -----------
for name, cost, tdp in (("custom", COST_CUSTOM, JAL_TDP),
("merchant", COST_MERCHANT, B300_TDP)):
capex = cost / AMORT_YEARS
energy = tdp * 1.0 * PUE * HOURS_PER_YEAR / 1000.0 * POWER_USD_KWH
share = energy / (capex + energy)
print(f" {name:9s} at 100% utilisation: capex ${capex:8,.0f}/yr, "
f"energy ${energy:7,.0f}/yr -> energy is {share*100:4.1f}% of TCO")
assert share < 0.5, "energy must not dominate capex at this tariff"
# The tariff at which energy would equal amortised capex for the merchant part:
break_even_tariff = (COST_MERCHANT / AMORT_YEARS) / (
B300_TDP * PUE * HOURS_PER_YEAR / 1000.0)
print(f" Energy would only equal amortised capex at ${break_even_tariff:.2f}/kWh, "
f"far above any industrial tariff")
assert break_even_tariff > 0.30, break_even_tariff
print(" => utilisation is a WEAK lever here: capex dominates across (0, 1].")
print(" The strong levers are the chip price and the realised throughput.")
# --- 5b. Two buyers, two verdicts, one crossover in software maturity ------
# A CHIP-BUDGET-constrained buyer maximises tokens per dollar of capex.
# A POWER-constrained buyer maximises tokens per watt and ignores capex.
d_capex = (MERCHANT_ROOFLINE / COST_MERCHANT) * COST_CUSTOM / CUSTOM_ROOFLINE
d_power = (MERCHANT_ROOFLINE / B300_TDP) * JAL_TDP / CUSTOM_ROOFLINE
print()
print(f" Derate at which the custom part matches on tokens per capex dollar: "
f"{d_capex*100:5.1f}%")
print(f" Derate at which the custom part matches on tokens per watt : "
f"{d_power*100:5.1f}%")
assert d_capex < d_power, "the cheaper chip must clear the capex bar first"
print()
print(" derate capex-constrained buyer power-constrained buyer agree?")
verdicts = []
for d in (0.15, 0.21, 0.23, 0.25, 0.30, 0.60):
v_capex = (CUSTOM_ROOFLINE * d / COST_CUSTOM) > (MERCHANT_ROOFLINE / COST_MERCHANT)
v_power = (CUSTOM_ROOFLINE * d / JAL_TDP) > (MERCHANT_ROOFLINE / B300_TDP)
verdicts.append((d, v_capex, v_power))
print(f" {d*100:5.1f}% {'custom':>21s} {'custom':>22s} yes"
if (v_capex and v_power) else
f" {d*100:5.1f}% {('custom' if v_capex else 'merchant'):>21s} "
f"{('custom' if v_power else 'merchant'):>22s} "
f"{'yes' if v_capex == v_power else 'NO'}")
# Below d_capex both buyers pick the merchant part; above d_power both pick the
# custom part; in between the two verdicts DISAGREE. That disagreement band is
# the whole custom-silicon argument.
assert not verdicts[0][1] and not verdicts[0][2], "at 15% derate both pick merchant"
assert verdicts[-1][1] and verdicts[-1][2], "at 60% derate both pick custom"
disagree = [v for v in verdicts if v[1] != v[2]]
assert len(disagree) == 3, disagree
assert all(d_capex < v[0] < d_power for v in disagree), disagree
print(f" ASSERT ok: verdicts disagree exactly on the derate band "
f"({d_capex*100:.1f}%, {d_power*100:.1f}%)")
print(" a capex-constrained buyer takes the cheap chip there;")
print(" a power-constrained buyer takes the efficient one.")
print(" [S1] and [S2] both state OpenAI is power-constrained, so the right-hand")
print(" column is the one that governs the Jalapeno decision.")
# --- 5c. Sensitivity: the price lever moves only the capex bar -------------
print()
for price in (8_000.0, 16_000.0, 24_000.0):
dc = (MERCHANT_ROOFLINE / COST_MERCHANT) * price / CUSTOM_ROOFLINE
dp = (MERCHANT_ROOFLINE / B300_TDP) * JAL_TDP / CUSTOM_ROOFLINE
print(f" custom chip at ${price:,.0f}: capex bar {dc*100:5.1f}%, "
f"power bar {dp*100:5.1f}% (price does not move the power bar)")
bars = [(MERCHANT_ROOFLINE / COST_MERCHANT) * p / CUSTOM_ROOFLINE
for p in (8_000.0, 16_000.0, 24_000.0)]
assert bars[0] < bars[1] < bars[2], bars
# Halving the chip price halves the capex bar exactly: the model is linear in price.
assert abs(bars[0] - bars[1] / 2.0) < 1e-12
# --------------------------------------------------------------------------
print()
print(" Boundary and failure cases:")
for label, fn in (
("utilisation of 0 rejected",
lambda: annual_cost(COST_CUSTOM, JAL_TDP, 0.0)),
("utilisation above 1 rejected",
lambda: annual_cost(COST_CUSTOM, JAL_TDP, 1.5)),
("model with zero bytes per token rejected",
lambda: decode_ceiling(JAL_BW, 0.0, 0.5)),
("zero bandwidth rejected (infinite ridge point)",
lambda: ridge_point(JAL_FLOPS, 0.0)),
("zero peak FLOPS rejected",
lambda: ridge_point(0.0, JAL_BW)),
):
try:
fn()
raise SystemExit(f"FAIL: {label} did not assert")
except AssertionError:
print(f" ASSERT ok: {label}")
print()
print("All assertions passed.")
Executed output:
==============================================================================
1. HBM4 stack count implied by the reported bandwidth and pin rate [S1]
==============================================================================
Jalapeno: 15.40 TB/s at 10.0 Gbps pin -> 6.016 x 2048-bit HBM4 stacks
6 stacks x 2048 bit x 10 Gbps = 15.36 TB/s (reported 15.4 TB/s)
Round trip: 15.4 TB/s over 6 stacks = 10.03 Gbps pin, the rate [S1]
reports. The pin rate is DERIVED from the bandwidth, so this fixes the implied
stack count at six and corroborates NEITHER number independently.
Jalapeno HBM bandwidth advantage over B300: 1.93x per package, 3.85x per watt
==============================================================================
2. Ridge points and the roofline crossover [S1][S3]
==============================================================================
Jalapeno ridge = 870.1 FLOP/byte (13.4 PFLOP/s / 15.4 TB/s)
B300 ridge = 1875.0 FLOP/byte (15.0 PFLOP/s / 8.0 TB/s)
AI (FLOP/B) Jalapeno PFLOP/s B300 PFLOP/s winner
1.0 0.015 0.008 Jalapeno
100.0 1.540 0.800 Jalapeno
870.1 13.400 6.961 Jalapeno
1000.0 13.400 8.000 Jalapeno
1675.0 13.400 13.400 tie
3000.0 13.400 15.000 B300
Per-chip ranking flips at AI = 1675.0 FLOP/byte (Jalapeno peak FLOPS / B300 bandwidth)
AI= 1658.2: Jalapeno 13.400 > B300 13.266 PFLOP/s
AI= 1691.8: Jalapeno 13.400 < B300 13.534 PFLOP/s
Peak compute per watt : Jalapeno 19.14 TFLOP/s/W vs B300 10.71 (1.79x)
Peak bandwidth per watt: Jalapeno 22.00 GB/s/W vs B300 5.71 (3.85x)
==============================================================================
3. Decode is bandwidth-bound, prefill is compute-bound: ranking flips
==============================================================================
DeepSeek-R1: 671B total / 37B active, MXFP4 -> 18.5 GB read per decode step
Decode ceiling (bandwidth-bound) : Jalapeno 832.4 tok/s B300 432.4 tok/s
Prefill ceiling (compute-bound) : Jalapeno 181081 tok/s B300 202703 tok/s
Jalapeno / B300 per chip: decode 1.925x vs prefill 0.893x
Jalapeno / B300 per watt: decode 3.850x vs prefill 1.787x (no flip)
R1 weights at MXFP4 = 336 GB, so the model must be sharded; Jalapeno HBM CAPACITY is not disclosed in the free source text
Measured 700 tok/s/user is 84.1% of ONE package's bandwidth roofline
At the reported TP32 config it is 2.63% of the aggregate roofline -> latency-bound, not bandwidth-bound
==============================================================================
3b. Die vs package: the comparison unit decides the winner [S1][S5][S6]
==============================================================================
NVIDIA per package 35 PFLOPS dense NVFP4 / SemiAnalysis per die 17.5 = 2.0 dies per package
Two dies per package is NVIDIA's own statement [S6], not this division;
35/17.5 is exactly 2 by construction and cannot fail.
Caution: 17.5 PFLOPS is ALSO NVIDIA's published dense FP8/FP6 per
Rubin GPU PACKAGE [S5], so the number alone is not diagnostic of the unit.
Jalapeno package bandwidth vs B300 package : 1.93x (leads)
Jalapeno package bandwidth vs Rubin package: 0.70x (trails)
R1 decode ceiling per package: Jalapeno 832.4 tok/s, B300 432.4, Rubin 1189.2
So the memory-bound win reported against HBM3E Blackwell parts does NOT
carry over to HBM4 Rubin packages. NVIDIA publishes no Rubin package TDP,
so the per-watt version of this comparison cannot be computed here.
==============================================================================
4. Reproducing OpenAI's own published ratios from its own operands [S2]
==============================================================================
model metric computed rounded stated
GPT-OSS-120B perf/W 1.901 1.9 1.9
GPT-OSS-120B e2e latency 1.748 1.7 1.7
GPT-OSS-120B min TBT 2.710 2.7 2.7
GPT-OSS-120B iso-TBT 53.712 53.7 53.7
DeepSeek-R1-670B perf/W 1.667 1.7 1.7
DeepSeek-R1-670B e2e latency 3.630 3.6 3.6
DeepSeek-R1-670B min TBT 4.126 4.1 4.1
DeepSeek-R1-670B iso-TBT 103.881 103.9 104.3 <-- does not reproduce
Kimi-K2.5-1T perf/W 1.534 1.5 1.5
Kimi-K2.5-1T e2e latency 3.404 3.4 3.4
Kimi-K2.5-1T min TBT 3.806 3.8 3.8
Kimi-K2.5-1T iso-TBT 56.200 56.2 56.1 <-- does not reproduce
ASSERT ok: 10 of 12 reproduce exactly; the 2 that do not are both iso-TBT
DeepSeek-R1-670B computed 103.88 vs stated 104.3
Kimi-K2.5-1T computed 56.20 vs stated 56.1
(consistent with the headline being computed on unrounded operands)
GPT-OSS-120B vs GB200: 1.90x at 700 W TDP -> 2.42x at the measured 550 W
DeepSeek-R1-670B vs GB300: 1.67x at 700 W TDP -> 2.12x at the measured 550 W
Kimi-K2.5-1T vs GB300: 1.53x at 700 W TDP -> 1.95x at the measured 550 W
Neither figure is tokens per all-in utility watt: host CPUs, scale-up
switches, optics and cooling are excluded from BOTH sides.
==============================================================================
5. TCO: which scarce resource binds decides the winner
COST INPUTS BELOW ARE ILLUSTRATIVE PARAMETERS, NOT VENDOR PRICING
==============================================================================
custom at 100% utilisation: capex $ 4,000/yr, energy $ 537/yr -> energy is 11.8% of TCO
merchant at 100% utilisation: capex $ 10,000/yr, energy $ 1,073/yr -> energy is 9.7% of TCO
Energy would only equal amortised capex at $0.65/kWh, far above any industrial tariff
=> utilisation is a WEAK lever here: capex dominates across (0, 1].
The strong levers are the chip price and the realised throughput.
Derate at which the custom part matches on tokens per capex dollar: 20.8%
Derate at which the custom part matches on tokens per watt : 26.0%
derate capex-constrained buyer power-constrained buyer agree?
15.0% merchant merchant yes
21.0% custom merchant NO
23.0% custom merchant NO
25.0% custom merchant NO
30.0% custom custom yes
60.0% custom custom yes
ASSERT ok: verdicts disagree exactly on the derate band (20.8%, 26.0%)
a capex-constrained buyer takes the cheap chip there;
a power-constrained buyer takes the efficient one.
[S1] and [S2] both state OpenAI is power-constrained, so the right-hand
column is the one that governs the Jalapeno decision.
custom chip at $8,000: capex bar 10.4%, power bar 26.0% (price does not move the power bar)
custom chip at $16,000: capex bar 20.8%, power bar 26.0% (price does not move the power bar)
custom chip at $24,000: capex bar 31.2%, power bar 26.0% (price does not move the power bar)
Boundary and failure cases:
ASSERT ok: utilisation of 0 rejected
ASSERT ok: utilisation above 1 rejected
ASSERT ok: model with zero bytes per token rejected
ASSERT ok: zero bandwidth rejected (infinite ridge point)
ASSERT ok: zero peak FLOPS rejected
All assertions passed.
Five results are worth carrying out of that block.
The ridge point is the design thesis. Jalapeno's 870 FLOP/byte against B300's 1,875 means it reaches its compute roof at less than half the arithmetic intensity. (The B300 figures come from NVIDIA's GB300 NVL72 spec table; NVIDIA's DGX B300 page implies a different per-GPU dense FP4 number, and the discrepancy plus its effect on these results is set out in the footnote.6) Every memory-bound kernel, which is most of decode, sits further from the cliff. That, not peak FLOPS, is what the architecture bought.
The per-chip ranking flips with the workload. On a DeepSeek R1 decode step (18.5 GB of activated MXFP4 weights per token) Jalapeno's bandwidth gives it a 1.93x ceiling over B300; on compute-bound prefill its lower peak FLOPS gives it 0.89x. Per watt there is no flip, because the 2x TDP gap swamps the 12% FLOPS deficit. An operator counting chips and an operator counting megawatts read the same silicon differently.
The ranking also flips on the choice of baseline. SemiAnalysis's claim that Jalapeno "bests all the other accelerators shipping that are using HBM3E" is carefully worded, and the qualifier is doing real work: against a Rubin package at NVIDIA's published 22 TB/s of HBM4, Jalapeno's 15.4 TB/s trails by 0.70x and the memory-bound advantage disappears entirely. The block also reconciles the two sources' units, and is explicit that the reconciliation is a definition rather than a discovery: NVIDIA's own architecture blog states the Rubin GPU is built from "reticle limited compute dies" and that "These two dies are unified on a single package," which is where the two comes from; dividing NVIDIA's 35 PFLOPS dense NVFP4 per package by SemiAnalysis's "17.5 PFLOPs per Rubin compute die" only restates it. The number 17.5 is worse than uninformative on its own, because NVIDIA's Vera Rubin spec table also publishes 17.5 PFLOPS of dense FP8/FP6 per Rubin GPU package, so the bare figure does not say which unit an analyst meant. Comparing a Jalapeno package against a Rubin die is not a like-for-like comparison, and this page's arithmetic is the only place either source's unit is made explicit.
Ten of OpenAI's twelve published multipliers reproduce exactly from its own displayed operands under its own rounding. The two that do not are both iso-interactivity ratios, the row OpenAI labels "More throughput at previous TBT" (DeepSeek R1 at 103.9 computed against 104.3 stated; Kimi K2.5 at 56.2 against 56.1), consistent with the headline being computed on unrounded values that the appendix displays rounded. This is a presentation artifact, not a substantive error, but it means the displayed operands cannot be used to re-derive those two figures.
The 700 tok/s/user figure is latency-bound, not bandwidth-bound. DeepSeek R1 at MXFP4 is about 336 GB of weights, so it cannot sit on one package and must be sharded. At the TP32 configuration SemiAnalysis reports, 700 tok/s/user is 2.6% of the aggregate bandwidth roofline. The result is impressive precisely because it is not a bandwidth result: it is evidence that the fixed-overhead removal works.
How to use it¶
There is no public access path, no SDK, no cloud region, and no documented API. Nothing in this section is a procedure an external operator can run today; it is what the published record says about how the part is driven.
Serving runs through an internal engine OpenAI calls Teacup.1 Reported parallelism configurations are TP8 and TP32 for large models and EP8 for higher-concurrency mixture-of-experts points on gpt-oss; the MoE background is in MoE routing and load balancing. Models confirmed running are gpt-oss 120B, DeepSeek R1 670B, and Kimi K2.5 1T, plus internal OpenAI models including GPT-5.3-Codex-Spark.24 SemiAnalysis reports GSM8k evaluation results "on par with Nvidia chips" for all tested models, which is an accuracy sanity check rather than a quality benchmark.1
An external operator evaluating any comparable custom part should replicate the following from this record rather than the marketing numbers: fix the input/output split and state it, fix the interactivity target and compare at matched user experience, state whether speculative decoding is on for each side, and state the power denominator explicitly (package TDP, node power, or all-in utility power give materially different answers).
How to develop with it¶
The kernel language is Gluon, OpenAI's lower-level layer on top of Triton. SemiAnalysis describes it as preserving Triton's SPMD programming model while exposing low-level abstractions, with the distinguishing feature being an explicit layout abstraction built on Linear Layouts.1 Linear Layouts is not a rumour: it is a published paper by the Triton core team that models tensor layouts as linear algebra over the field with two elements, giving generic layout-to-layout conversions instead of a case-by-case table, and it is integrated into Triton.9 Each Gluon program maps to a persistent thread, which points at the persistent-kernel pattern where the programmer rather than a hardware scheduler assigns tiles.
Three reported facts about the development loop are the operationally interesting part:
- Kernels are hand-written at close to assembly level, some running to about 3,000 lines, backed by correctness checks and a custom sanitizer.1
- Model bring-up is substantially AI-driven. OpenAI states it "brought three open-weight models that were not part of Jalapeño's original production plan to high performance within two months" using Codex, and that for selected gpt-oss attention and MoE blocks "AI-generated implementations ran 1.5 to 1.8 times faster than the existing human-expert-written implementations," with the explicit caveat that "those figures apply to the selected blocks, not the full model."2 SemiAnalysis adds that OpenAI had no internal MLA kernel implementation at all until it benchmarked DeepSeek.1
- A simulator, "chilisim," is reported accurate to within 5% of measured hardware, used to validate performance before committing to silicon runs.1
The out-of-order core with a hardware-managed L1 is the developer-visible trade. It removes the explicit scratchpad choreography that TPU-class and GPU kernels need, but it makes performance depend on prefetching that is harder to reason about statically. SemiAnalysis's reading is that this is deliberate: design for the highest upper bound across all shapes, then let a code-generation model find the kernel that reaches it.1 Whether that holds outside a lab with an in-house frontier model is the open question, and it is the claim an operator should be most sceptical of.
How to maintain it¶
Reliability engineering is reported at two levels. On silicon, OpenAI has built redundancy at the tray level and yield harvesting at the core and channel level, so a failed core or a failed memory channel degrades a part rather than discarding it.1 That matters for a first-generation ASIC on a leading node, where defect density is the main yield risk.
On the fleet, SemiAnalysis reports OpenAI "is gathering reliability data with datacenter partners through January while optimizing dock-to-rack rollout time."1 That is a statement that field reliability data does not yet exist, which is the correct thing to assume about any first-generation part. No public MTBF, no error-taxonomy equivalent to NVIDIA's Xid codes, and no documented health-check tooling exist for Jalapeno.
The two-rack shape has its own maintenance consequence. Host and accelerator are in separate racks joined by 8 external PCIe DAC cables per tray pair, so a host failure, a cable failure, and an accelerator failure are three separate domains with three separate replacement procedures. That is a different operational model from the integrated superchip trays covered in DGX, HGX and MGX systems, and cable-level faults on external PCIe are a failure class that integrated designs do not have.
How to run it in production¶
Power and facility planning dominates. At roughly 160 kW for the host-plus-ASIC pair, the deployment unit draws about what a double-width GB300 rack does, which puts it firmly in liquid-cooled, high-density territory; see GPU power and thermal tuning. The gap between the 700 W package rating and the reported 550 W or lower sustained draw is real headroom for power capping, and it is also why the perf-per-watt claims are conservative on Jalapeno's side.
Capacity planning has to be done at the scale-up world size, not the chip. The 128-XPU local domain and 2,048-XPU global domain are the units that matter for placing a large model. SemiAnalysis notes scale-up networking is only about 10% of total system cost, so the large domain is cheap optionality for future 10 to 20 trillion parameter models or 2 to 4 million token contexts.1
Two supply-side risks should be priced explicitly:
- Single-source everything. One fab (TSMC N3P for compute, N3E for the I/O chiplet), one design partner (Broadcom), one integrator (Celestica), one switch vendor (Broadcom Tomahawk 6), and HBM that SemiAnalysis says is "likely provided by Samsung," a hedged attribution that no vendor has confirmed.1 There is no second source at any layer, and unlike a merchant GPU there is no alternative supplier to switch to.
- No merchant channel and no ecosystem. OpenAI states it "will continue to widely deploy accelerators from NVIDIA and other partners for both training and inference workloads."2 Jalapeno displaces some inference capacity in one operator's fleet; it does not currently offer anyone else a procurement option.
The software risk is the one that the executed TCO block quantifies. Below roughly a fifth of its roofline (on this page's illustrative prices), a custom part loses on both counts; above roughly a quarter it wins on both. The band between is where a capex-constrained buyer and a power-constrained buyer disagree. Since both OpenAI and SemiAnalysis state OpenAI is power-constrained rather than budget-constrained, the perf-per-watt column is the one that governs, and cheap silicon does not rescue an immature kernel library.
Failure modes¶
| Failure mode | Cause | Mitigation or check |
|---|---|---|
| Perf-per-watt claim does not survive contact with the facility | Published ratios use package TDP; host CPUs, switches, optics and cooling are excluded from both sides.2 | Re-derive at the utility meter. A 160 kW two-rack unit includes a host rack the chip figures ignore. |
| Comparison flatters the custom part | Baselines are GB200/GB300 on HBM3E, not the HBM4 generation the part actually competes against.1 | Compare against Rubin's published 22 TB/s per package; the executed block shows the memory-bound result reverses. |
| Die-versus-package unit confusion | SemiAnalysis quotes Rubin per compute die (17.5 PFLOPS); NVIDIA publishes per package (35 PFLOPS dense, 2 dies).15 | Normalise both sides to the same physical unit before quoting any ratio. |
| Iso-interactivity multiplier read as a throughput ratio | The 53.7x and 104.3x figures compare against the baseline's best achievable TBT, measuring its latency cliff.2 | Quote peak-versus-peak (1.5x to 1.9x) for capacity planning. |
| Benchmark does not predict production | All public runs are single-turn 8k/1k; no AgentX, no multi-turn, no long-context, no prefix-cache stress.1 | Treat routers, prefix cache and offload paths as unvalidated. See inference serving. |
| Capacity planning cannot be completed | HBM capacity per package is not disclosed anywhere in the readable sources. | Do not estimate packages-per-model from bandwidth. Bandwidth and capacity are independent. |
| Kernel performance does not generalise to a new model | Out-of-order cores depend on prefetching that is hard to reason about statically; new model families still need new kernels.12 | Budget bring-up per model family. OpenAI's own figure is two months for three open-weight models, with a frontier code model in the loop. |
| Supply shock has no fallback | Single fab, single design partner, single integrator, single switch vendor, unconfirmed HBM source.1 | Price second-source risk explicitly; there is no alternate supplier at any layer. |
| Homogeneous pool underperforms at stable high load | No prefill/decode disaggregation; the design trades peak efficiency for fungibility.1 | If traffic really is stable and predictable, disaggregation can still win. See disaggregated inference. |
| Timeline claims quoted inconsistently | "Nine months" (vendor, design to tape-out) and "~16 months" (analyst, hiring to tape-out) anchor on different events.41 | State the anchor whenever quoting either figure. |
References¶
- SemiAnalysis, "OpenAI Jalapeño: Better Than Nvidia Blackwell" (Bryan Shan, Myron Xie, Jordan Nanos and 3 others), 2026-08-25. Partially paywalled: readable through "What's Next"; the section "Direct Implications for NVIDIA, AMD and Cerebras" onward is subscriber-only. https://newsletter.semianalysis.com/p/openai-jalapeno-better-than-nvidia
- OpenAI, "Jalapeño's first results show industry-leading speed and efficiency in AI inference" (includes the numeric appendix used above): https://openai.com/index/jalapeno-first-results/
- OpenAI, "OpenAI and Broadcom unveil LLM-optimized inference chip": https://openai.com/index/openai-broadcom-jalapeno-inference-chip/
- Broadcom investor relations, "OpenAI and Broadcom Unveil LLM-Optimized Intelligence Processor," 2026-06-24: https://investors.broadcom.com/news-releases/news-release-details/openai-and-broadcom-unveil-llm-optimized-intelligence-processor
- Broadcom Tomahawk 6 / BCM78910 series product page (102.4 Tb/s, 3 nm, 512x 200G PAM4 SerDes): https://www.broadcom.com/products/ethernet-connectivity/switching/strataxgs/bcm78910-series
- Broadcom press release, Tomahawk 6 first shipment, 2025-06-03: https://www.broadcom.com/company/news/product-releases/63146
- NVIDIA GB300 NVL72 product page (spec table: 20 TB HBM3E, up to 576 TB/s, FP4 1440 | 1080 PFLOPS): https://www.nvidia.com/en-us/data-center/gb300-nvl72/
- NVIDIA DGX B300 product page (8x Blackwell Ultra, FP4 144 | 108 PFLOPS): https://www.nvidia.com/en-us/data-center/dgx-b300/
- NVIDIA Vera Rubin NVL72 product page (Rubin GPU: 50 PFLOPS NVFP4 inference, 35 PFLOPS dense NVFP4 training, 288 GB HBM4 | 22 TB/s; all values marked preliminary): https://www.nvidia.com/en-us/data-center/vera-rubin-nvl72/
- NVIDIA developer blog, "Inside NVIDIA Rubin GPU Architecture," 2026-07-21 ("up to 288 GB of HBM4 ... 12-Hi stacks, to deliver up to 22 TB/s of peak bandwidth"): https://developer.nvidia.com/blog/inside-nvidia-rubin-gpu-architecture-powering-the-era-of-agentic-ai/
- JEDEC press release, JESD270-4 HBM4 standard, 2025-04-16 ("transfer speeds up to 8 Gb/s across a 2048-bit interface"): https://www.jedec.org/news/pressreleases/jedec%C2%AE-and-industry-leaders-collaborate-release-jesd270-4-hbm4-standard-advancing
- Zhou, Lezcano, Goucher, Rakhmati, Niu, Lebar, Szczerbuk, Bell, Tillet, Raoux, Moudallal, "Linear Layouts: Robust Code Generation of Efficient Tensor Computation Using F_2": https://arxiv.org/abs/2505.23819
- CoreWeave, "NVIDIA Vera Rubin NVL72 on CoreWeave: 10x More Tokens per Megawatt than Blackwell," 2026-07-21 (DeepSeek R1, ratio only, no absolute figures published): https://www.coreweave.com/blog/nvidia-vera-rubin-nvl72-on-coreweave-10x-more-tokens-per-megawatt-than-blackwell
- SemiAnalysis InferenceX benchmark (cited by NVIDIA's own DGX B300 page as well): https://inferencex.semianalysis.com/
- Hugging Face model cards used for the parameter counts: https://huggingface.co/deepseek-ai/DeepSeek-R1 (671B total / 37B activated) and https://huggingface.co/openai/gpt-oss-120b (117B total / 5.1B active, MXFP4 MoE weights)
Related: NVIDIA Blackwell platform · NVIDIA GPU roadmap · GPU generations · DGX, HGX & MGX systems · Roofline & arithmetic intensity · NVFP4 quantization · Triton · Disaggregated inference · Inference serving · LLM inference efficiency · Networking fabric · Build vs rent GPU cost model · GPU power & thermal tuning
-
SemiAnalysis, "OpenAI Jalapeño: Better Than Nvidia Blackwell," 2026-08-25, https://newsletter.semianalysis.com/p/openai-jalapeno-better-than-nvidia (retrieved 2026-08-26, readable portion only). Source of: B0 stepping at 13.4 PFLOPS MXFP4 on a reticle-sized TSMC N3P die with roughly 25% perf-per-watt gain over A0; 700 W TDP against "Rubin's at 900-1,150W per compute die"; "17.5 PFLOPs of dense Rubin NVFP4 for a single Rubin compute die"; 15.4 TB/s HBM4 per package, "10Gbps pin speeds ... a slight edge over the 9.6Gbps Nvidia is getting out of its HBM4 in Rubin," and "The HBM is likely provided by Samsung"; "15.4TB/s of memory bandwidth per package which bests all the other accelerators shipping that are using HBM3E"; N3E I/O chiplet with 32 lanes of 800G SerDes (24 local / 8 global), PCIe Gen 5 host I/O; CoWoS tape-out November 2025 with 3 months of silicon bring-up; "Design work began in the middle of 2024, going from initial team hiring to manufacturing tape-out in ~16 months"; weight-stationary systolic array with MXFP formats, small-dimension support, 64-bit scalar and FP32/INT32 vector cores, tray-level redundancy and core/channel yield harvesting, claimed 8% SIMD and 10% matrix-engine area reduction from AI-assisted design; core/HBM slicing with a dedicated collective network plus a general NoC; out-of-order cores with L1 cache; Gluon on Triton with Linear Layouts and TensorInfo; ~3,000-line hand-tuned kernels, custom sanitizer, "Teacup" serving engine, "chilisim" simulator accurate to within 5%, "gigakernel" megakernel; no internal MLA kernel until DeepSeek was benchmarked; TP8 to TP32 in 8 days, EP8 for high-concurrency gpt-oss points; rack topology (16 Katsu host trays, 16 Vindaloo trays of 8 ASICs = 128/rack, 8 Chana switch trays = 6 local + 2 global, Tomahawk 6, copper backplane, 4.8 Tb/s local and 1.6 Tb/s global per XPU, 8-rail global domain with OCS, 2,048-XPU scale-up world); ~50 kW host rack provisioned (31 kW production) and ~130 kW ASIC rack, ~160 kW total; two EPYC Turin CPUs with 1.5 TB DRAM per Katsu tray, 8 external PCIe DAC cables per tray pair, Celestica integration; scale-up networking "only about 10% of total system cost"; production ramping over 2027; "gathering reliability data with datacenter partners through January"; the prefill/decode disaggregation argument; and the caveats that "all numbers are provided to us by OpenAI," that "they're just 8k1k ... and there are no AgentX runs yet," that the tested models "are not on the open frontier," and that "comparison to Blackwell is somewhat incomplete and unfair. Jalapeño is really competing against chips like Rubin that also use HBM4." Everything from the heading "Direct Implications for NVIDIA, AMD and Cerebras" onward is paywalled and is not used here; all charts in the readable portion are images whose plotted values could not be read. ↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩↩
-
OpenAI, "Jalapeño's first results show industry-leading speed and efficiency in AI inference," https://openai.com/index/jalapeno-first-results/ (retrieved 2026-08-26). Source of the appendix table reproduced above (InferenceX, nominal 8k/1k, STP, package TDP: Jalapeño 700 W, GB200 1,200 W, GB300 1,400 W) and of: "Jalapeño is rated at 700 watts, although its measured sustained power remained at or below 550 watts on the workloads tested"; "Although performance is sometimes reported per chip, we believe the more useful standard is performance per unit of power"; "Across all three, Jalapeño delivered 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the comparison systems"; the Codex bring-up claim ("three open-weight models that were not part of Jalapeño's original production plan to high performance within two months" and "AI-generated implementations ran 1.5 to 1.8 times faster ... Those figures apply to the selected blocks, not the full model"); "We plan to begin deploying Jalapeño within OpenAI's compute infrastructure by the end of the year"; Gen 2 "deep in development" and Gen 3 "taking shape"; and "We will continue to widely deploy accelerators from NVIDIA and other partners for both training and inference workloads." ↩↩↩↩↩↩↩↩↩↩↩
-
OpenAI, "OpenAI and Broadcom unveil LLM-optimized inference chip," https://openai.com/index/openai-broadcom-jalapeno-inference-chip/ (retrieved 2026-08-26). "Jalapeño is a blank-slate design for modern LLM inference, not a general-purpose accelerator adapted from earlier AI workloads"; "co-developed from initial design to manufacturing tape-out in just nine months"; engineering samples "running ML workloads in the lab at production target frequency and power, including GPT‑5.3‑Codex‑Spark." ↩↩
-
Broadcom investor relations, "OpenAI and Broadcom Unveil LLM-Optimized Intelligence Processor," GLOBE NEWSWIRE, 2026-06-24, https://investors.broadcom.com/news-releases/news-release-details/openai-and-broadcom-unveil-llm-optimized-intelligence-processor (retrieved 2026-08-26). Establishes the announcement date and the corporate structure of the program: OpenAI design, Broadcom silicon implementation and Tomahawk networking, Celestica board/rack/system. Hock Tan: "we are enabling the deployment of gigawatt scale data centers with Microsoft and other partners beginning in 2026." Richard Ho leads OpenAI's hardware program. ↩↩↩↩↩
-
NVIDIA Vera Rubin NVL72 product page, https://www.nvidia.com/en-us/data-center/vera-rubin-nvl72/ (retrieved 2026-08-26), "Rubin GPU" column: NVFP4 Inference 50 PFLOPS; NVFP4 Training 35 PFLOPS; "GPU Memory | Bandwidth 288 GB HBM4 | 22 TB/s"; NVLink 6 at 3.6 TB/s. Two caveats the page itself imposes: footnote 1 states "Preliminary information. All values are up to and subject to change," and footnote 2 ("Dense specification") is attached to the NVFP4 training, FP8/FP6, INT8, FP16/BF16 and TF32 rows but not to the NVFP4 inference row, which NVIDIA therefore leaves unqualified as dense or sparse. NVIDIA publishes no per-package TDP for Rubin and no HBM4 stack count ("12-Hi" is stack height, not count), so the per-watt and per-stack comparisons against Rubin could not be computed. There is no non-CPX, non-Ultra "Vera Rubin NVL144" product page; that URL returns 404. ↩
-
NVIDIA GB300 NVL72 product page, https://www.nvidia.com/en-us/data-center/gb300-nvl72/ (retrieved 2026-08-26): "Configuration 72 NVIDIA Blackwell Ultra GPUs, 36 NVIDIA Grace CPUs", "GPU Memory | Bandwidth 20 TB | Up to 576 TB/s", "FP4 Tensor Core 1440 | 1080 PFLOPS" (sparse | dense). Per GPU that is 8.0 TB/s and 15.0 PFLOPS dense, the figures used in the executed block. Inconsistency worth flagging: the NVIDIA DGX B300 page (https://www.nvidia.com/en-us/data-center/dgx-b300/) states "FP4 Tensor Core: 144 PFLOPS | 108 PFLOPS" for 8 Blackwell Ultra GPUs, which is 13.5 PFLOPS dense per GPU, not 15.0. NVIDIA does not reconcile the two on either page; the likeliest explanation is a clock or power difference between the liquid-cooled NVL72 part and the air-cooled DGX form factor, but that is an inference, not a vendor statement. The executed block uses the NVL72 figure and the conclusions are unchanged at 13.5 (the ridge point becomes 1,687 FLOP/byte and the prefill ranking narrows but does not reverse). ↩
-
Broadcom BCM78910 series (Tomahawk 6) product page, https://www.broadcom.com/products/ethernet-connectivity/switching/strataxgs/bcm78910-series (retrieved 2026-08-26): "102.4 Tb/s StrataXGS® Tomahawk® 6 Ethernet Switch Series"; "128 integrated Peregrine 106.25G PAM4 SerDes cores (BCM78910) or 64 Condor 212.5G PAM4 SerDes cores (BCM78914)"; sample I/O configurations "64 x 1.6TbE, 128 x 800GbE, 256 x 400GbE, 512 x 200GbE"; "implemented in 3nm technology"; "Support for scale-up clusters with 512 XPUs and scale-out clusters with 1M+ XPUs"; Ultra Ethernet compliant. Volume production was announced 2026-03-12 (https://www.broadcom.com/company/news/product-releases/64031). Note the Broadcom URL path
/strataxgs/tomahawk6returns 404; the BCM78910 path above is the live one. ↩ -
JEDEC, "JEDEC® and Industry Leaders Collaborate to Release JESD270-4 HBM4 Standard," 2025-04-16, https://www.jedec.org/news/pressreleases/jedec%C2%AE-and-industry-leaders-collaborate-release-jesd270-4-hbm4-standard-advancing (retrieved 2026-08-26): "With transfer speeds up to 8 Gb/s across a 2048-bit interface, HBM4 boosts total bandwidth up to 2 TB/s"; 32 channels per stack with 2 pseudo-channels each; 4/8/12/16-high stacks at 24 Gb or 32 Gb die density. JESD270-4A was published December 2025 with no press release restating a higher pin rate, and no JEDEC statement of an HBM4 rate above 8 Gb/s was found. Both the 10 Gbps (Jalapeño) and 9.6 Gbps (Rubin) pin rates reported by SemiAnalysis are therefore above the JEDEC baseline, which is normal for vendor-binned HBM but is not JEDEC-confirmed. ↩
-
Zhou, Lezcano, Goucher, Rakhmati, Niu, Lebar, Szczerbuk, Bell, Tillet, Raoux, Moudallal, "Linear Layouts: Robust Code Generation of Efficient Tensor Computation Using $\mathbb{F}_2$," https://arxiv.org/abs/2505.23819 (abstract retrieved 2026-08-26). Models tensor layouts as binary matrices acting on the bits of the hardware representation, "enabl[ing] a generic layout definition ... and allow[ing] for generic layout-to-layout conversions, eliminating the quadratic explosion that plagues existing solutions," integrated with Triton. The author list is the Triton core team, which corroborates SemiAnalysis's account of Gluon's provenance; the paper itself makes no claim about Jalapeño. ↩
-
gpt-oss-120b model card, https://huggingface.co/openai/gpt-oss-120b (retrieved 2026-08-26): "117B parameters with 5.1B active parameters"; "The models were post-trained with MXFP4 quantization of the MoE weights, making
gpt-oss-120brun on a single 80GB GPU." The native MXFP4 weights match Jalapeño's MXFP-native matrix engine without a conversion step. ↩