Compute · GPU & AI

GPU servers, sized to the model — not the price list.

A GPU server runs its work on data-centre accelerators rather than the CPU, which is what AI training and inference need. VV Internet Hosting builds single-tenant GPU nodes on NVIDIA Blackwell and Hopper — B200 and H200 — atop high-frequency AMD EPYC hosts, with NVLink, fast fabric, and liquid cooling. We size to your model and workload, and give honest advice on renting versus owning, on when an H200 beats waiting for Blackwell, and on where AMD Instinct fits.

See plans & pricing Talk to an engineer →

Reviewed by Iria Moreau, Lead Systems Engineer, Compute · Accelerator landscape last reviewed 2026-06 · Updated 2026-06-17

In short

VRAM decides what fits. A 70B model at FP16 needs ~140 GB, so it sits on one H200 (141 GB) or B200 (192 GB); larger models want 288 GB or NVLink sharding.
Train versus infer changes the answer. Training rewards interconnect and the CUDA ecosystem; inference rewards memory, bandwidth, and FP4.
Newest is not automatically best. The H200 handles most workloads now at lower power; Blackwell earns its place when memory or FP4 is the real constraint.
Rent or own is an honest calculation. Bursty work rents; sustained, round-the-clock load and data-residency needs favour owning a single-tenant node.
NVIDIA leads, AMD is competitive. CUDA still wins large-scale training; AMD Instinct is strong on memory-bound inference, with a tighter rental market.

What is a GPU server, and what is it for?

It is a server whose real work happens on graphics processors rather than the central one. A modern AI accelerator is a different kind of chip from a CPU: thousands of parallel units, tens to hundreds of gigabytes of high-bandwidth memory beside the silicon, and Tensor Cores built to multiply matrices at the precisions machine learning uses. For training a model or serving one, that parallelism is the whole game, and the CPU's job shrinks to feeding the GPUs and moving data in and out.

Three numbers describe most of what matters. Memory capacity decides how large a model fits without being split across cards. Memory bandwidth decides how fast it runs once it is loaded, because generating each token means reading the model's weights again. And the interconnect — NVLink inside a node, a fast network between nodes — decides how well many GPUs work as one when a job is too big for a single card. The rest of this page is mostly about matching those three numbers to what you are trying to do.

It helps to picture where the time goes. During training the GPUs grind through the dataset over and over, exchanging gradients constantly, so the links between them are under steady pressure. During inference the model already exists and the work is reading its weights to produce each next token, which leans on raw memory bandwidth. The same card does both, but a node tuned for one is rarely the cheapest way to do the other — which is why the first question we ask is what you are actually running.

The accelerators

Which GPU do you actually need?

The answer is a workload, not a product. Here is the current field and what each part is genuinely for, reviewed 2026-06.

NVIDIA B200 (Blackwell)

NVIDIA

Blackwell · dual-die · TSMC 4NP

VRAM192 GB HBM3e (~180 GB usable) Bandwidth~8 TB/s TDP1000 W (up to 1200 W) — liquid cooling for most builds Interconnect5th-gen NVLink 1.8 TB/s; PCIe Gen5

Best forNew 8–64 GPU training clusters and high-throughput LLM inference; 70B at FP16 or up to ~120B at FP8 on a single GPU.

Second-generation Transformer Engine with native FP4 gives roughly 4× the inference of an H100 and ~2–2.5× training. Supply is the constraint — large order backlogs through mid-2026 mean cloud rental is the fast path, and a US export-control variant exists.

NVIDIA B300 (Blackwell Ultra)

NVIDIA

Blackwell Ultra

VRAM288 GB HBM3e Bandwidth~8 TB/s TDP~1400 W — liquid cooling Interconnect5th-gen NVLink; ConnectX-8 (doubled inter-node)

Best forWhen a single GPU must hold a 70B+ model without quantisation, or FP4 throughput is the hard constraint.

A higher-binned B200 with ~50% more VRAM and ~67% more FP4 compute; worth it only when the extra memory or networking is the actual bottleneck.

NVIDIA H200 (Hopper)

NVIDIA

Hopper (memory upgrade on H100)

VRAM141 GB HBM3e Bandwidth~4.8 TB/s TDP700 W — air-coolable Interconnect4th-gen NVLink; PCIe Gen5

Best forThe pragmatic default: ample memory, mature software, lower power, available now.

Handles the large majority of training and inference today; the right pick when 141 GB is enough and you do not need Blackwell's FP4 throughput.

NVIDIA H100 (Hopper)

NVIDIA

Hopper

VRAM80 GB HBM3 Bandwidth~3.35 TB/s TDP700 W — air-coolable Interconnect4th-gen NVLink; PCIe Gen5

Best forCost-effective workhorse for fine-tuning and inference at sensible model sizes.

Still capable and abundant; prices keep falling as Blackwell supply grows, which makes it strong value when 80 GB fits the job.

AMD Instinct MI355X (CDNA 4)

AMD

CDNA 4 · chiplet · TSMC 3nm

VRAM288 GB HBM3e Bandwidth~8 TB/s TDP~1400 W — liquid cooling (MI350X air-cooled, lower TBP) InterconnectInfinity Fabric mesh (~1.075 TB/s peer); ROCm 7 software

Best forMemory-bound inference and models that benefit from fitting on one GPU (500B+ parameters in 288 GB).

Competitive with B200 on single-node FP4 inference and near-parity on Llama-class fine-tuning; the trade-offs are the smaller ROCm ecosystem and a tighter rental market than NVIDIA's.

AMD Instinct MI300X (CDNA 3)

AMD

CDNA 3 · chiplet

VRAM192 GB HBM3 Bandwidth~5.3 TB/s TDP750 W InterconnectInfinity Fabric; ROCm

Best forMemory-bound models that do not fit a single H200, where ROCm support is in place.

More memory than an H200 and strong value for the right inference workloads; multi-node scaling and ecosystem maturity trail NVIDIA.

NVIDIA holds roughly 80% of AI accelerator share against AMD's ~5–7%. The gap is less about raw silicon than about software: CUDA, NCCL, and years of framework tuning still give NVIDIA the edge for large-scale multi-node training, while AMD has become genuinely competitive on single-node inference and memory-bound models.

Training or inference — what changes?

They stress different parts of the machine, and confusing them is how money gets wasted. Training rewards interconnect and ecosystem — NVLink within a node, InfiniBand across nodes, and CUDA/NCCL scaling — which is where NVIDIA is strongest. Inference rewards memory capacity and bandwidth to hold a model and stream tokens, where large-VRAM parts (B200, H200, MI355X, MI300X) and FP4 matter most and AMD is most competitive.

In practice that means a training cluster is bought around its fabric — how many GPUs you can make behave as one — while an inference fleet is bought around memory and bandwidth per dollar, because the job is to hold a model and stream tokens cheaply. The precision you run at sits underneath both: FP4 halves the bytes each parameter needs compared with FP8, which doubles the effective bandwidth and lets a single card serve a model that used to need two. We design the node for the side of that line you actually live on, rather than buying a training rig to do inference or starving a training run of interconnect.

Sizing follows from that split in a concrete way. For training we weight the node toward interconnect and the GPU count that scales cleanly on your framework; for inference we weight it toward memory per dollar and bandwidth, and lean on FP4 where the model tolerates it. Batch size, sequence length, and how many users you serve at once then set how much headroom to leave. None of this is exotic, but getting it roughly right is the difference between paying for accelerators that work and paying for ones that wait.

The node

What the GPU sits in: the host and the fabric

A GPU is only as useful as the machine feeding it. The host CPU, the links between GPUs, and the network to other nodes decide whether the accelerators are busy or waiting.

An 8-GPU node, end to end

The host matters more than people expect. A high-frequency AMD EPYC CPU — the 'F' parts built for clock speed — keeps eight hungry GPUs fed with data instead of letting them idle between batches, which is why GPU nodes pair accelerators with fast cores rather than the highest core count. PCIe Gen5 moves data to the cards, NVLink links the GPUs to each other so they share memory at terabytes per second, and a fast NIC on InfiniBand or RoCE joins nodes into a cluster. Get any one of those wrong and the expensive silicon waits.

an 8× B200 node — nvidia-smi

$ nvidia-smi --query-gpu=name,memory.total,power.limit --format=csv
name, memory.total [MiB], power.limit [W]
NVIDIA B200, 183360 MiB, 1000.00 W
NVIDIA B200, 183360 MiB, 1000.00 W
... (8 GPUs total on the HGX baseboard)

$ nvidia-smi topo -m   # interconnect matrix (excerpt)
       GPU0   GPU1   GPU2   ...   CPU Affinity   NUMA
GPU0    X     NV18   NV18          0-63          0
GPU1   NV18    X     NV18          0-63          0
# NV18 = 18 NVLink connections — full all-to-all over the NVLink switch

Eight GPUs on one baseboard, fully connected over NVLink; the host and fabric are sized so they stay busy.

From a single card to a rack

Most of what we have described assembles into a few standard shapes, and which one you need follows from scale. A single accelerator suits development and modest inference; an eight-GPU node is the workhorse for serious training and high-throughput serving; and a full rack ties dozens of GPUs into one fabric for the largest jobs. The systems below are the units we work with.

HGX B200 — eight Blackwell GPUs on one NVLink fabric (~144 PFLOPS FP4); the practical unit for 8–64 GPU training.
GB200 NVL72 — a liquid-cooled rack pairing 36 Grace CPUs with 72 Blackwell GPUs as one 72-GPU NVLink domain; rack-scale, for the largest training and trillion-parameter inference.
8-GPU MI355X / MI350X OAM platforms — air- or liquid-cooled, an open-standard alternative on ROCm.

The jump between these is a step in cost, power, and operational complexity each time, so we size to the smallest shape that does the job. A team that needs forty GPUs of training for a few weeks is usually better served by an eight-GPU node run hard, or by rented capacity, than by committing to a rack it will not keep full.

Choosing

The six things that decide the build

Once the workload is clear, the choice comes down to these, roughly in the order they bite.

Model size vs VRAM VRAM decides what fits without sharding: ~70B at FP16 needs ~140 GB, so 141–192 GB fits a 70B on one card; 288 GB reaches far larger models.

Memory bandwidth Token generation is bandwidth-bound; more TB/s is faster decode for inference.

Precision (FP4/FP8) FP4 halves bytes-per-parameter versus FP8, doubling effective bandwidth and throughput where accuracy allows.

Interconnect NVLink or Infinity Fabric for in-node multi-GPU; InfiniBand or RoCE for multi-node — the bottleneck for large training.

Software ecosystem CUDA/NCCL maturity versus ROCm; this, more than FLOPS, often decides real training throughput.

Power and cooling 1000 W+ accelerators need liquid cooling and reduce GPUs per rack; it is an infrastructure decision, not just a spec.

Should you rent or own GPUs?

This is the question we are asked most, and the honest answer is that it depends on how steadily you will use them. Renting wins when the work is spiky or exploratory. Bursty or experimental work, short projects, or any need for GPUs this month — cloud rental is the fast path while new-silicon backlogs persist. Owning wins when the work is steady. Sustained, near-24/7 utilisation, data-residency or isolation requirements, or multi-year horizons where a dedicated or colocated node beats per-hour rental.

Two cautions keep the decision honest. The first is supply: The newest parts carry long order backlogs and US export controls; we are honest about lead times rather than promising silicon we cannot source on your timeline. The second is the temptation to wait for the next thing. Waiting for the next generation often delays a project more than it saves — H100 and H200 handle the large majority of workloads today, and their prices keep falling. Put together, the sober path for most teams is to start on available, proven hardware — often Hopper — prove the workload, and commit to owning capacity only once utilisation justifies it.

Rent when

Bursty or experimental work, short projects, or any need for GPUs this month — cloud rental is the fast path while new-silicon backlogs persist.

Own when

Sustained, near-24/7 utilisation, data-residency or isolation requirements, or multi-year horizons where a dedicated or colocated node beats per-hour rental.

NVIDIA or AMD?

The market answers this loudly — NVIDIA holds the large majority of share — but the loud answer is not the whole one. NVIDIA's lead in large-scale training rests on software as much as silicon: CUDA, NCCL, and years of framework tuning make many GPUs scale predictably, and that is hard to match. If you are training a foundation model across dozens or hundreds of GPUs, that ecosystem is a real reason to choose it.

AMD has earned a genuine place all the same. With up to 288 GB of memory on a single Instinct card, it can hold models that would need sharding on smaller GPUs, and on ROCm it is competitive — sometimes ahead — on single-node inference. The honest catch is the rental market: far fewer providers offer AMD on short terms, so for sub-six-month contracts NVIDIA usually wins on price simply through supply. We will recommend AMD where its memory and inference value genuinely fit, and say plainly when CUDA's ecosystem or the deeper rental market makes NVIDIA the safer call.

There is a quieter factor too: people. A team already fluent in CUDA moves faster on NVIDIA, while one comfortable with ROCm can capture AMD's memory and price advantages without friction. The best hardware for a workload is partly the hardware your engineers can get the most out of, and we factor that into the recommendation rather than treating the chips as interchangeable boxes.

Power, cooling, and why these are single-tenant

Blackwell-class accelerators draw a thousand watts or more each, and eight of them in a chassis put out more heat than air can carry away, so most builds are liquid cooled. That is not a luxury; it is what lets the GPUs hold their clocks under sustained load instead of throttling. It also changes the data centre around them — fewer GPUs per rack, more attention to power delivery — which is part of why a 700-watt H200 is sometimes the calmer, air-cooled choice when its memory is enough.

Every GPU node we build is single-tenant, for the same reasons bare metal is. A training run or an inference fleet wants the whole machine — all the GPUs, all the NVLink bandwidth, the full host and its NVMe — with no other workload competing and no hypervisor between the model and the hardware. Single-tenancy also gives the physical isolation and data-residency control that AI work under real compliance pressure increasingly needs. It is the same principle as our dedicated servers, applied to accelerators. See bare metal →

Density is the other consequence worth planning for. Because each card draws so much and needs so much cooling, the number of GPUs you can put in a rack falls as the silicon gets hotter, and the supporting power and cooling become a real part of the cost. We account for that up front, so the node you size is one the facility can actually feed and cool at full tilt rather than one that throttles the first time it is genuinely busy.

What we run, and what we will not promise

We build single-tenant GPU nodes on high-frequency AMD EPYC hosts, mainly NVIDIA B200 and H200, with AMD Instinct where it fits, in the region you need. You get root, your choice of CUDA or ROCm stack, the NVLink fabric and fast networking the workload calls for, and the same operated-not-just- racked support that covers our other machines — we keep the hardware, network, and cooling healthy, and the model and data stay yours.

What we will not do is promise silicon we cannot source on your timeline. The newest parts carry long backlogs and sit under export controls, and pretending otherwise would only set up a missed date. We would rather tell you when Hopper is the faster path to running, when renting beats owning for your usage, or when a managed inference service would serve you better than any server we could sell. The point is to get your model running well and affordably, measured against your own numbers, not to move the most expensive box.

Bare metal → Cloud compute → Talk through a build →

Questions

GPU servers, answered plainly

The questions teams ask before they commit budget.

What is a GPU server?

A server whose work is done mainly by graphics processors rather than the CPU. For AI it means one or more data-centre accelerators — NVIDIA B200 or H200, or AMD Instinct — each carrying tens to hundreds of gigabytes of high-bandwidth memory, linked to each other by NVLink or Infinity Fabric and to other nodes by a fast network. The CPU's job becomes feeding the GPUs and handling data, not the maths.

Which GPU do I actually need?

It depends on the model size and whether you train or infer. VRAM decides what fits without splitting a model across cards: a 70B model at FP16 needs about 140 GB, so an H200 at 141 GB or a B200 at 192 GB holds it on one GPU. Bandwidth decides how fast tokens generate, and FP4 support roughly doubles inference throughput where accuracy allows. We size to the workload rather than to the top of the price list.

B200 or H200 — is the newest always better?

No. The B200 gives roughly four times the inference and around twice the training of an H100, but it draws 1000 watts, usually needs liquid cooling, and carries long supply backlogs. An H200 has ample memory, mature software, lower power, and is available now, and it handles the large majority of workloads. We reach for Blackwell when the memory or FP4 throughput is the real constraint, not by default.

Should I rent or own GPUs?

Rent for bursty or experimental work, short projects, or anything you need this month while new-silicon backlogs persist. Own — on a dedicated or colocated node — when utilisation is sustained and near round-the-clock, when data residency or isolation matters, or over a multi-year horizon where per-hour rental adds up. We can do either and will tell you which the numbers favour.

NVIDIA or AMD?

NVIDIA holds around 80 percent of the market and still leads large-scale multi-node training, mostly because of the CUDA and NCCL software ecosystem rather than raw silicon. AMD Instinct, with up to 288 GB of memory, has become genuinely competitive on single-node inference and memory-bound models on ROCm. The honest catch is supply: the NVIDIA rental market is far deeper, so for short contracts it usually wins on price too.

How much memory do I need to fit my model?

As a rough guide, parameters times two bytes for FP16, or times one for FP8. A 70B model is roughly 140 GB at FP16 or 70 GB at FP8, so it fits one H200 or B200 at full precision, or smaller cards when quantised. Above that, you either move to a 288 GB part or shard across GPUs over NVLink, which is where interconnect bandwidth starts to matter.

Why do these servers need liquid cooling?

Because the newest accelerators draw 1000 watts or more each, and eight of them in a chassis produce more heat than air can reasonably remove. Liquid cooling handles that thermal load and lets the GPUs hold their clocks under sustained work. Hopper-generation cards at 700 watts can still run air-cooled, which is one more reason an H200 is sometimes the simpler choice.

Can I get a single-tenant GPU server?

Yes — that is what we build. The whole node is yours, on a high-frequency AMD EPYC host, with the GPUs, NVLink fabric, and local NVMe dedicated to your workload and no other tenant sharing the machine. You get root, your choice of CUDA or ROCm stack, and the isolation that data-residency and compliance requirements tend to ask for.

Tell us about your model.

Its size, whether you train or infer, and how steadily you will run it — and we will size a single-tenant GPU node, or tell you when renting is the smarter spend.

Talk to an engineer See pricing