GPU servers

A graphics card fitted in a KVM node can be given to one server whole, through PCI passthrough. The server sees the real card, and the customer installs its driver and uses it as on their own machine: CUDA, rendering, video encoding, a remote desktop. Cards of any make work (NVIDIA, AMD or Intel, consumer or data center), and you sell GPU servers like any other server, from a package.

Two ways to sell a GPU

A node’s cards go either to servers or to containers. A node is one or the other: a KVM node passes cards through to servers, and a GPU container host rents them out in containers.

	GPU servers (this page)	GPU containers
What the customer gets	A whole virtual machine with the card in it	A container with whole cards, over SSH or JupyterLab
Cards	Any make, passed through with `vfio-pci`	NVIDIA, on the NVIDIA driver
Operating system	Any you offer, Windows included	The container image they pick
Ready in	As long as the server takes to install and boot	As long as the image takes to pull
Isolation	A virtual machine of its own	A container sharing the host’s kernel, in user namespaces
Sold as	A package with GPUs, like any server	A GPU offer, often by the hour
Node needs	VT-x or AMD-V, and IOMMU	No virtualization or IOMMU

Both take one license slot per node.

1. Get the node ready

The node needs IOMMU on, and each card alone in its IOMMU group. In short:

In the node’s BIOS, turn on VT-d (Intel) or AMD-Vi (AMD).
Add intel_iommu=on iommu=pt (Intel) or amd_iommu=on iommu=pt (AMD) to GRUB_CMDLINE_LINUX in /etc/default/grub, run update-grub and reboot.
If the node loaded a driver for the card (nvidia, amdgpu or nouveau) that won’t let go of it, blacklist that driver on the node.

What each of these is for, and what to do when a card shares its group with something else, is under What the node needs.

2. Switch the cards on

Open the node’s Assets tab. The agent lists every graphics card it found, with its PCI address, IOMMU group and driver; Look again asks it again after you fit one. New cards start switched off: choose Let servers use it for each card you want to sell.

3. Sell GPU servers from a package

Under Packages, open a package (or make one, such as GPU-4090) and, under GPUs, tick Servers on this package get GPUs. Set Per server to how many cards each server gets, and Model to the kind of card, or Any card. The GPUs section shows once a node has a card switched on for servers.

Each server made from the package gets its cards as it’s created, on a node with enough free cards of that model. Where none are free, the package shows as unavailable there, with the reason, so your billing panel’s orders are refused rather than left waiting. See Packages: GPUs.

The package is sold like any other:

WHMCS: pick it as the product’s Package on its Module Settings tab (WHMCS module).
Blesta: pick it as the Blesta package’s Panel package under Module Options (Blesta module).
Your own billing: create the server with the package’s ID, as for any server (Connecting a billing panel).

Giving a card to one server

To give a card to a server that already exists, or one that isn’t on a GPU package, open the server’s Hardware tab, pick a free card on its node under GPUs, and choose Give it this card. The server picks it up the next time it’s stopped and started; Take off works the same way.

What customers see

The server’s Overview shows the card’s name, not where it is. Inside, it’s an ordinary PCI device: the customer installs the card’s driver (NVIDIA’s, AMD’s or Intel’s) in their operating system. For NVIDIA cards, the server is hidden from the driver as a virtual machine, so older consumer drivers that refused to run in one (error 43) work too.

Good to know

Moves: a server with a card can’t move to another node, since the card is in that node. Take the card off, move the server, then give it a card on the new node. Emptying a node lists such servers as blocked.
Changing package: Change package only offers packages with the same GPU settings (count and model) as the server’s. To give a server more or fewer cards, use its Hardware tab.
Split cards: cards that can be split between servers (NVIDIA vGPU or MIG, AMD MxGPU, Intel SR-IOV) are marked on the Assets tab, and go to one server whole.
AMD cards that don’t reset: if a card stops working after its server restarts (the “reset bug”), install vendor-reset on the node.
A card that moved: a card the node no longer reports, because it was removed or moved to another slot, isn’t passed through. Choose Look again, then take the old card off the server and give it the card again.

The full detail is under Nodes: Assets and GPUs.