GPU & AI servers · Comparison
H100 vs H200 vs B200, matched to the workload.
The H100, H200, and B200 are NVIDIA's data-centre GPUs for AI, and the honest headline is that newest is not automatically best. The H100 is the cost-efficient workhorse for models that fit in 80 GB; the H200 adds memory for inference on larger models; the B200 is a generational leap for the biggest models and FP4 work, at a higher price and power cost. We host all three on single-tenant bare metal and help you pick by workload, not by spec-sheet number.
In short
- H100 — the workhorse. 80 GB, broadly available, lowest cost, deepest software stack. Right when your model fits in 80 GB.
- H200 — same compute, more memory. 141 GB HBM3e at 4.8 TB/s on the same Hopper silicon, for memory-bound inference and larger models on one GPU.
- B200 — the generational leap. Around 180–192 GB, FP4 support, and far higher throughput — for the largest models, at 1,000 W and the highest cost.
- Cost per result, not per hour. The cheapest GPU that fits, or the fastest that finishes, often beats the one with the bigger number.
- We host all three. Single-tenant bare metal, EU or US, and an honest steer toward the one your workload actually needs.
How should you think about the choice?
The temptation with GPUs is to read down the spec sheet and pick the biggest numbers, and it is an expensive habit. These three cards are not a simple good-better-best ladder; they are tools shaped for different jobs. The H100 and H200 share the same Hopper compute silicon, so the H200 is not a faster chip — it is the same chip with much more memory. The B200 is a genuine generational jump on a new architecture, but that jump comes with a thousand-watt power draw and a price that only some workloads can justify. The right question is never "which is newest" but "which matches my model and my budget".
The single most useful lens is your bottleneck. Large-model inference is usually memory-bound: the Tensor Cores can process tokens faster than the memory can feed them, so capacity and bandwidth decide throughput, and that is the H200's whole reason to exist. Training and workloads that fit in memory are more often compute-bound, where the H100 and H200 perform alike and the H100's lower cost wins. The B200 changes both equations at once for the largest models, but only there. Identify whether memory or compute is your limit and most of the decision makes itself.
The second lens is cost per result rather than cost per hour. A B200 can cost roughly three times an H100 per hour, which looks unfavourable until you account for finishing a large training run in half the time or serving many more tokens per second. Equally, paying for a B200 to run a model that would sit happily on an H100 is money lit on fire. We work this through on your actual workload, because the honest answer depends on numbers specific to you, not on a general ranking of the cards.
Side by side
The three cards, spec by spec
| H100 | H200 | B200 | |
|---|---|---|---|
| Architecture | Hopper (2022) | Hopper (2024) | Blackwell (2025) |
| Memory | 80 GB HBM3 | 141 GB HBM3e | ~180–192 GB HBM3e |
| Memory bandwidth | 3.35 TB/s | 4.8 TB/s | ~8 TB/s |
| Compute silicon | GH100 die | Same GH100 die | Dual-die Blackwell |
| Low precision | FP8 | FP8 | FP8 + FP4 / FP6 |
| NVLink | Gen 4, 900 GB/s | Gen 4, 900 GB/s | Gen 5, 1.8 TB/s |
| Power (TDP) | 700 W | 700 W | 1,000 W |
| Headline gain vs H100 | Baseline | ~1.9× inference (Llama 70B) | Up to ~15× inference, ~4× training (system) |
| Availability + cost | Broad, lowest cost | Broad, premium | Newer, highest cost + power |
Vendor figures, indicative and configuration-dependent; B200 memory and system-level gains vary by platform. Check current specs before committing.
Which one fits your workload?
The clean way to choose is by the job in front of you. Here is the honest short version of when each card is the right call.
Choose the H100 when
- Your model and KV cache fit in 80 GB at your precision.
- The workload is compute-bound training of sub-100B models.
- Cost and broad availability matter and you do not need the extra memory.
- You want the most mature, widely tested software path.
Choose the H200 when
- Inference is memory-bound and you keep hitting the 80 GB wall.
- You want to serve a 70–100B model on a single GPU, not split it.
- Long-context or large-batch serving makes the KV cache the bottleneck.
- You want the memory upgrade with no change to your Hopper code.
Choose the B200 when
- You are training or serving the largest models, at hundreds of billions of parameters and up.
- FP4 or FP6 precision is on your roadmap and you can validate it.
- Throughput per result justifies the cost, power, and cooling.
- You are building rack-scale and can support 1,000 W per GPU.
The spectrum
Where each card sits
From the cost-efficient workhorse to the generational leap, the three cards trade rising memory, bandwidth, and throughput against rising price and power.
The arrow only runs one way on price and power. On value, it does not — the best card is the leftmost one that still fits your workload, which for a great many teams is the H100 or H200, not the B200.
Why is the H200 a memory upgrade, not a faster chip?
This catches people out, so it is worth being explicit. The H200 is built on the same Hopper compute die as the H100. Its Tensor Cores, its FP8 throughput, its raw compute — all identical. What changed is the memory subsystem: 141 GB of HBM3e at 4.8 TB/s in place of 80 GB of HBM3 at 3.35 TB/s, a 76 percent jump in capacity and 43 percent in bandwidth. So on any workload that is limited by compute and already fits in 80 GB, the two cards perform alike, and the H100 is the cheaper way to get there.
Where the H200 pulls ahead is the large and growing set of workloads that are limited by memory. A 70B model in 16-bit precision needs roughly 140 GB for weights alone, which spills past an H100 and forces you to split across two cards with the inter-GPU traffic that brings. The H200 holds it on a single GPU, which is why NVIDIA's own figures show it serving such models close to twice as fast — a gain that comes entirely from feeding the same compute with more, faster memory. If your constraint is capacity, that is the whole game.
What makes the B200 a generational step?
The B200 is the first of these three on NVIDIA's Blackwell architecture, and it is a different kind of change. Rather than a memory refresh, it connects two Blackwell dies in one package, roughly doubling compute density, and pairs them with around 180 to 192 GB of HBM3e at close to 8 TB/s. It adds fifth-generation NVLink at 1.8 TB/s per GPU and, crucially, native FP4 and FP6 precision, which lets the largest models run at far higher throughput where lower precision is acceptable. At the system level NVIDIA quotes up to fifteen times the inference and four times the training of an H100 generation system.
The catch is everything around the chip. The B200 runs at 1,000 watts against 700 for the Hopper cards, which pushes most deployments toward liquid cooling and demands real power and rack planning. It is newer, so it costs the most and is less abundant. For the largest training runs and rack-scale inference it earns that premium handily; for a mid-sized model that an H200 would serve comfortably, it is capability you pay for and do not use. This is exactly why we treat the host, the cooling, and the interconnect as part of the decision rather than an afterthought to the GPU.
How we build GPU servers →Deciding
How to choose, step by step
A short process that starts from your model and your bottleneck, not from the newest launch.
- 01
Start from model size
Work out how much memory your model and its KV cache actually need. If the weights fit comfortably in 80 GB at your precision, the H100 is likely the cost-efficient answer.
- 02
Find your bottleneck
Decide whether you are memory-bound or compute-bound. Inference on large models is usually memory-bound, which is where the H200's larger, faster memory pays off even though its compute matches the H100.
- 03
Weigh the newest against its cost
The B200 is a generational leap for the largest models and FP4 work, but it draws more power, often wants liquid cooling, and costs the most. Reach for it when the workload genuinely needs it.
- 04
Think in cost per result
Compare on cost per token or per training run, not cost per hour. A pricier GPU that finishes in half the time can be cheaper for the job, and a cheaper one that fits can win on steady inference.
- 05
Match the host to the choice
Whichever you pick, single-tenant hardware with the right power and cooling matters as much as the chip. We size the host, the interconnect, and the cooling to the GPU rather than the other way round.
Questions
The GPU choice, answered plainly
The things teams email us about most.
Is the B200 always better than the H100 and H200?
No, and treating newest as best is the most expensive mistake in this category. The B200 is a generational leap for the largest models, FP4 work, and rack-scale inference, but it costs the most, draws a thousand watts, and often wants liquid cooling. For a workload that fits in an H100's 80 GB, or an inference job that is memory-bound rather than compute-bound, an H100 or H200 frequently delivers a better cost per result. The right GPU is the one matched to your model and budget, not the one with the highest number.
What is the real difference between the H100 and H200?
Memory, not compute. The H200 uses the same Hopper compute silicon as the H100 — identical Tensor Cores and FP8 performance — but pairs it with 141 GB of faster HBM3e at 4.8 TB/s instead of the H100's 80 GB at 3.35 TB/s. That makes the H200 markedly better for memory-bound inference and for serving large models on a single GPU, while offering no advantage on compute-bound work that already fits in 80 GB. If you are not hitting the memory wall, the H100 is the more cost-effective choice.
Which GPU should I use for LLM inference?
It depends on the model size and your latency target. For models that fit in 80 GB at your chosen precision, the H100 is cost-efficient and widely available. For 70B-class and larger models, or long-context serving where the KV cache dominates, the H200's 141 GB lets you serve on a single GPU and avoid splitting across cards. For the very largest models and the highest throughput, the B200's FP4 acceleration is in a different league — at a price and power cost to match.
Do I need to change my code to move between them?
Very little. All three share NVIDIA's CUDA toolchain, so moving from H100 to H200 needs no changes — it is the same Hopper architecture — and code that runs on an H100 runs on a B200 without modification, since the Blackwell software stack is compatible. Standard frameworks like PyTorch and vLLM work across all three. The migration effort is in infrastructure and tuning, not in rewriting your model code.
Why does the B200 need so much more power?
Because it does much more in one package. The B200 connects two Blackwell dies, roughly doubling compute density, and runs at a 1,000-watt TDP against the 700 watts of the H100 and H200. That extra power has real consequences for the host: denser cooling, often liquid rather than air, and power and rack planning that the Hopper cards do not demand. It is part of why the right comparison includes the infrastructure around the GPU, not just the chip.
Can you host all three?
Yes. We provision the H100, H200, and B200 on single-tenant AMD EPYC bare metal, in the EU or the US, sized with the right interconnect, power, and cooling for the GPU you choose. Because we are not trying to move a particular card off a shelf, we will tell you honestly which one fits your workload and budget — including when the H100 you already know is still the right answer.
Tell us what you are training or serving.
We will match the GPU to your model and budget — including when the H100 you already know is the right answer — and build the host, the cooling, and the interconnect around it.