Use case · AI & GPU compute
AI & ML training
AI and ML training needs GPUs with enough memory for your models, fast interconnect to scale across them, and storage that can feed them without bottlenecking. Dedicated GPU servers suit sustained training better than on-demand cloud, and EU-hosted hardware keeps training data under European jurisdiction.
Key points
- Training is bound by GPU memory, interconnect, and data throughput — not just raw compute.
- GPU memory has to fit your model and batch; it often decides which GPU you need.
- Sustained training utilises hardware for weeks, which favours dedicated GPU servers over hourly cloud.
- Feeding GPUs needs fast local storage, or the GPUs sit idle waiting on data.
- EU-hosted training keeps sensitive datasets under European jurisdiction, outside US CLOUD Act reach.
What infrastructure does AI/ML training actually need?
Training a machine-learning model is one of the most demanding things you can ask of computing infrastructure, and it stresses several parts of a system at once. The obvious requirement is GPU compute — training runs the enormous parallel mathematics of deep learning, which GPUs perform far faster than CPUs. But raw GPU compute is only one of the constraints. Training is equally bound by how much memory each GPU has, how quickly GPUs can communicate when a job spans several of them, and how fast data can be fed to them. A system strong in one of these and weak in another will bottleneck on the weak part, so training infrastructure has to be balanced across all of them.
In practice, this means training infrastructure is defined by four things: GPUs with sufficient memory for the model and batch size, fast interconnect between GPUs so multi-GPU jobs are not starved by communication, storage and data pipelines that keep the GPUs fed rather than idle, and the power, cooling, and density to run high-wattage GPUs continuously. Get all four right and the GPUs stay busy doing useful work; get any one wrong and expensive GPUs sit waiting. The rest of this page walks through each of these, and where a dedicated GPU server is the right home for training versus where something else fits better.
Choosing GPUs for training: memory first, then performance
The first question in specifying training hardware is usually not how fast a GPU is but how much memory it has. A GPU can only work on data held in its own memory, so the model, its gradients and optimiser state, and the batch of data being processed must all fit within the GPU's memory. If they do not, the model cannot be trained on that GPU as-is — you must reduce the batch size, use techniques to split the model, or move to a GPU with more memory. This makes GPU memory capacity the constraint that most often decides which GPU a training workload needs, ahead of the GPU's speed.
Among the GPUs used for training, there is a clear progression in memory and performance. An NVIDIA H100 provides substantial high-bandwidth memory suited to large models; the H200 offers more memory again, which matters as models grow; and the Blackwell-generation B200 pushes both memory and performance further, aimed at the largest and most demanding training. The honest guidance is to size the GPU to the workload rather than reaching for the largest by default: a model and batch that fit comfortably on an H100 gain nothing from a more expensive GPU except headroom, while a very large model that will not fit on smaller memory genuinely needs the larger-memory parts. Matching the GPU's memory and performance to what you are actually training is where the money is well or badly spent.
Scaling across GPUs, and why interconnect matters
Many training jobs do not fit on, or would take too long on, a single GPU, so they are spread across several GPUs, or several servers each with several GPUs. When a job spans multiple GPUs, those GPUs must constantly exchange data — gradients and parameters — to keep the distributed training coherent. The speed of this communication becomes a limiting factor: if the GPUs cannot exchange data fast enough, they spend time waiting on one another instead of computing, and the benefit of adding more GPUs diminishes. So for multi-GPU training, the interconnect between GPUs matters as much as the GPUs themselves.
This is why training servers use high-speed interconnect between their GPUs, and why multi-server training uses fast networking between nodes. Within a server, technologies that link GPUs at high bandwidth let them exchange data quickly, so a multi-GPU job scales efficiently rather than stalling on communication. Across servers, high-throughput, low-latency networking serves the same purpose at a larger scale. The practical implication is that scaling training is not simply a matter of adding GPUs — the GPUs have to be connected well enough that adding them actually speeds up the job. A well-designed training server keeps its GPUs communicating fast enough that they scale together, which is a large part of what distinguishes purpose-built training hardware from a machine that merely has several GPUs in it.
Keeping the GPUs fed: storage and data throughput
Training GPUs are only useful when they have data to work on, and a large training dataset must be read and delivered to the GPUs continuously throughout a training run. If the storage and data pipeline cannot supply data as fast as the GPUs consume it, the GPUs stall — waiting on data rather than computing — which wastes the most expensive part of the system. This makes data throughput a real and often underappreciated constraint on training: expensive GPUs held back by slow data delivery deliver far less than they could.
Avoiding this means providing fast local storage and a data pipeline sized to the GPUs' appetite. Fast NVMe storage close to the GPUs can serve data at the rates training demands, and staging datasets onto such storage keeps the GPUs fed. The balance to strike is between the GPUs' consumption rate and the storage's delivery rate, so that the pipeline is not the bottleneck. In specifying training infrastructure, it is a mistake to focus entirely on GPUs and treat storage as an afterthought; the storage and data path have to be fast enough to keep those GPUs busy, or the GPU investment is partly wasted. A balanced training server pairs its GPUs with storage and I/O that can actually feed them.
Bare metal versus cloud GPUs for training
For training specifically, the choice between dedicated bare-metal GPU servers and on-demand cloud GPUs comes down mostly to how sustained the workload is. Training runs tend to utilise GPUs heavily and for extended periods — hours, days, or weeks of near-continuous use — and under sustained utilisation, dedicated hardware at a fixed cost is usually far more economical than paying an hourly cloud rate for the same continuous use. On-demand cloud GPUs are priced for flexibility, which is valuable for short or bursty needs but expensive when the hardware is busy most of the time, as training makes it.
The honest framing is therefore about your utilisation pattern. If you train intermittently, in short bursts, or need to spin up briefly and stop, on-demand cloud GPUs let you pay only for the short time you use them, which suits that pattern well. If you train continuously or near-continuously — a team with ongoing training work keeping GPUs busy most of the time — dedicated GPU servers avoid paying a premium hourly rate for hardware you are using anyway, and the fixed cost is recovered quickly at high utilisation. We host dedicated GPU servers for exactly the sustained case, and we will say plainly that for genuinely bursty or occasional training, on-demand cloud may serve you better. The right answer follows from how much of the time your GPUs would actually be working.
Data sovereignty for training data
Training data is often sensitive: it may contain personal data, proprietary information, or material subject to regulation, and where that data is stored and processed carries legal weight. Under European data-protection law, and for organisations concerned with data sovereignty, it matters that training data is held under a jurisdiction whose protections and reach are understood and acceptable. Data placed on infrastructure subject to foreign legal access can be exposed to that access regardless of where the data sits, which is a consideration for training on sensitive datasets.
This is a wedge where EU-hosted, EU-operated infrastructure genuinely differs. VV Internet Hosting is incorporated in the Netherlands, within the EU, which places our infrastructure under European jurisdiction and outside the direct reach of the US CLOUD Act that can compel US-linked providers to produce data. For training on personal or sensitive data where European data protection and sovereignty matter, hosting the training — data and compute together — in the EU keeps it under European law rather than exposing it to foreign legal access. This is not relevant to every training workload, but where your training data is sensitive or regulated, the jurisdiction of the infrastructure is a real part of the decision, and it is one we are positioned to answer clearly.
Power, cooling, and density for training GPUs
The GPUs used for training draw a great deal of power and generate correspondingly large amounts of heat — a single high-end training GPU can draw hundreds of watts, and a server packed with several draws well into the kilowatts. Running such hardware continuously, as training does, demands a datacenter able to deliver that power to each rack and remove the heat it produces. This is a real constraint: high-density GPU servers cannot simply be placed anywhere, because the power and cooling must be there to support them, and a facility built for conventional servers may not be able to power or cool a rack of training GPUs.
This is why where training hardware is hosted matters as much as the hardware itself. A datacenter designed for high power density and effective cooling can run training GPUs at full utilisation reliably; one that is not may throttle, fail, or simply be unable to host the hardware. When we place GPU servers for training, the facility's power density and cooling are part of the specification, not an afterthought, because sustained training pushes both hard. For anyone planning training infrastructure, it is worth confirming that the hosting can actually power and cool the GPUs continuously — the most capable GPUs are of no use if the facility cannot keep them running.
Reliability and checkpointing for long training runs
Training runs can last days or weeks, and the longer a run, the more it matters that the hardware stays up throughout. An interruption partway through a long run — a hardware fault, a power event, an unplanned reboot — can cost enormous amounts of compute if progress is lost, since the run may have to resume from much earlier or restart entirely. This makes the reliability of the underlying hardware and hosting a genuine concern for training, not an abstraction: the cost of downtime during a long run is measured in the compute wasted and the time lost, both of which can be substantial.
The practical defences are reliable hardware and disciplined checkpointing. Checkpointing — periodically saving the training state so a run can resume from the last checkpoint rather than the beginning — limits how much is lost to any interruption, and is essential practice for long runs; it depends on storage that can hold checkpoints and I/O fast enough that checkpointing does not itself become a drag. On the hosting side, dedicated hardware in a datacenter with reliable power and cooling reduces the frequency of interruptions in the first place, so that long runs are less likely to be disrupted. Together, reliable hosting and sound checkpointing keep long training runs from losing expensive progress. When we host training hardware, this reliability is part of what we provide, because for multi-week runs it is as important as the GPUs themselves.
Where VV Internet Hosting fits — and where it does not
We host dedicated GPU servers for AI and ML training: current-generation NVIDIA GPUs with the memory and interconnect training needs, paired with fast NVMe storage to keep them fed, in EU datacenters built for the power and cooling that sustained training demands, under European jurisdiction. This suits teams running sustained or ongoing training, workloads where dedicated hardware at a fixed cost beats hourly cloud rates, and cases where training data is sensitive enough that European sovereignty matters. For those situations, we are a strong fit, and we will help size the GPUs, storage, and interconnect to what you are actually training rather than selling you more than you need.
We are equally clear about where we are not the answer. If you need to spin GPUs up for minutes at a time and stop, occasional or bursty training is better served by on-demand cloud, and we will tell you so. If you need a hyperscaler's managed training platforms, vast multi-thousand-GPU clusters on demand, or a specific proprietary orchestration ecosystem, that is a different kind of provider than we are. We are for dedicated, sustained, sovereignty-aware training infrastructure operated on hardware you control — not for every AI workload. If that matches what you are building, we can help; if it does not, we would rather point you to what fits than take on work we are not the right home for.
Related use cases
Questions
AI/ML training, answered plainly
Common questions about hosting for AI/ML training.
What matters most when choosing a GPU for training?
Usually memory capacity, ahead of raw speed. The model, its gradients and optimiser state, and the batch of data must all fit in the GPU's memory, so memory often decides which GPU a workload needs. Among training GPUs, the H100, H200, and B200 offer progressively more memory and performance — size the GPU to what you're training rather than reaching for the largest by default.
Is it cheaper to train on dedicated GPUs or cloud GPUs?
It depends on utilisation. Training tends to keep GPUs busy for extended periods, and under sustained utilisation, dedicated GPU servers at a fixed cost are usually far more economical than hourly cloud rates. On-demand cloud suits short or bursty training, where you pay only for brief use. Continuous training favours dedicated hardware; occasional training favours cloud.
Why does where I train matter for data protection?
Training data is often sensitive or regulated, and the jurisdiction of the infrastructure determines what legal access applies. EU-incorporated, EU-hosted infrastructure keeps training data under European law and outside the direct reach of the US CLOUD Act. For sensitive or regulated training data, hosting data and compute together in the EU keeps it under European jurisdiction.
Planning AI/ML training infrastructure?
We host dedicated, EU-sovereign infrastructure sized to your workload — and we will tell you plainly when something else fits better. Tell us what you're building.