Moving servers between nodes

Staff move a server with Move to another node… in the server’s ⋯ menu. Let placement choose the target, or pick one. Only nodes in the same region are offered, so the server keeps its IP addresses.

A server keeps its addresses when the target node is in each address’s IP pool (the pool’s Nodes, under IP pools), which is where the subnet is routed. Put every node of a location in its pools and moves never touch addresses. Let placement choose and emptying a node only pick such nodes.

Moving with new addresses

Pick a node outside the server’s pools and the dialog says needs new IPs, and lists what changes: each address gets one from the target’s pools, and a floating IP is taken off the server and stays in the customer’s account. Tick Give it new IP addresses to move it; Email the customer their new addresses is ticked too. Live moves still run live:

1. The server moves as usual and keeps running.
2. The new addresses come from the target’s pools, the node lets them through, and its guest agent changes them inside the server: the saved network settings (netplan, ifupdown, ifcfg, NetworkManager or systemd-networkd on Linux; the adapter’s settings on Windows) and the running interface. Open connections drop once.
3. The customer gets an in-panel notice, and the Server’s IP addresses changed email if you chose it. Billing gets server.addresses_changed.

The transfer’s page shows old and new addresses, whether the server uses them yet, and an Email customer button to send (or resend) the email. A powered-off server, or one whose guest agent doesn’t answer, switches over once it’s running and the agent answers. If the change inside it fails, the page says why; set the addresses from the console. IPv4 addresses keep their reverse DNS names; IPv6 records for the old prefix are removed. Over the API, the same move needs changeAddresses: true (and emailCustomer); without it, a move to such a node is refused with addresses_not_routed.

Moves run virsh migrate over SSH between the nodes’ management addresses. The disk and memory go through an SSH tunnel too, so they’re encrypted, and the target only listens for them on its own loopback (TCP 49152–49215): only TCP 22 needs to be open between nodes (see Adding a node). The nodes exchange keys over their agent connections and pin each other’s host keys with StrictHostKeyChecking=yes. A node’s key on another is only let in from its management address, and can only forward to those loopback ports.

Both nodes need an agent from the release that brought the tunnel; a move to a node with an older agent stops straight away and says to update that node first (it updates itself within minutes).

CPUs and live moves

A server keeps the CPU it booted with until it’s next stopped and started, so a live move only works if the new node’s CPU can do everything the old one could. With host-model and qemu64, libvirt checks that itself. With host-passthrough, the guest sees the node’s real CPU, so the panel compares the two nodes’ CPUs (each node’s agent reports its model and features, shown on the node’s Overview):

• Same model, or a newer one with every feature of the old one (say EPYC Milan to Genoa): moves live, like any other server.
• The new CPU lacks some features (Genoa back to Milan, which has no AVX-512): offline by default, and the move dialog lists the missing features. Staff can still choose Move live anyway. Whether that works depends on whether the guest uses those features: if it does, programs or the whole server can crash after the switch. An offline move restarts it on the new CPU, which is always safe.
• Intel to AMD or back: offline only. KVM on one can’t take over the other’s CPU state, so a live move copies everything and then fails at the switch-over (QEMU’s log on the target says Failed to put registers after init: Invalid argument). Servers that need to move live between Intel and AMD nodes should use the qemu64 CPU model instead.
• A node that hasn’t reported its CPU yet (an older agent): treated like missing features until its agent updates itself.

Only instruction-set features count. Power management, the host’s own virtualization features and speculation mitigations differ between identical machines with other firmware, and don’t matter to the guest.

When emptying a node, servers that can’t move live safely move offline.

A server waiting for its setup has nothing on its node yet, so moving it just changes where it’s held: the node, its storage and, if needed, new addresses, all at once with no copy and no job. That works even when its node is down. Emptying a node moves these servers the same way.

Choosing the storage

Placement picks the target’s storage pool the way it does for new servers: the node’s default pool if it has the server’s disk tier and room, otherwise the matching pool with the most free space. Once you pick a node, Storage lists its pools with how much is free and how full each would be afterwards; pools of another tier, switched off, or without room are greyed out with the reason. Additional disks go to the same pool. Over the API, send storagePoolId (from pools in the move options) with a targetNodeId.

Moving a disk to other storage on the same node

On a server’s Hardware tab, Disks lists its system disk and any additional disks, each with the storage it’s on. Move… puts one on another pool of the same node: off a pool that’s filling up, from HDD to NVMe, or from one kind of storage to another (LVM, LVM thin, ZFS, qcow2 files).

• Running server: it keeps running. The node copies the disk while it’s in use with libvirt’s block copy (virsh blockcopy), mirrors new writes, then switches the server over to the copy. There’s no restart.
• Stopped or suspended server: the disk is copied directly with qemu-img.
• Either way the old copy is removed only once the new one is in use, so a failed move leaves the server as it was. It runs as a job on the server and the Jobs page; power actions wait until it’s done.
• Snapshots are stored next to the system disk (LVM and ZFS snapshots, qcow2 copies), so they can’t come along: delete them first, or take a backup instead. Additional disks have no snapshots.
• Another tier: the system disk can go to a pool of another tier than its package’s only when you tick Change its disk type. The server becomes a custom size, and later moves keep it on that tier. Additional disks can go on any tier.

It needs the servers:move permission. Over the API: POST /admin/servers/{id}/actions/move-disk with { storagePoolId, diskId?, changeTier? }, where diskId is an additional disk’s ID and is left out for the system disk. To move a disk to another node, move the server.

Emptying a node

On a draining or in-maintenance node, Move servers off plans where every server goes, using the group’s placement rule and taking earlier moves into account. It moves them two at a time, with a progress banner and a way to stop after the current moves. Servers that don’t fit anywhere stay put, and the dialog says why. Once a node has no servers left, it can be removed from its Settings tab.

Transfers

Every move is listed under Operations → Transfers (/admin/transfers). The sidebar shows a count while anything is moving. The list shows:

• the server and its owner
• the source and target node, and the storage pool the disk goes to
• live or offline
• progress, or how it ended: downtime, how long it took, or the error
• who started it, including “Emptying node” for evacuations

Filter by in progress, done, or failed and cancelled; by node; or search. A transfer’s page shows each step as it happens, the copy rate and time left during the disk copy, and a timestamped log you can copy.

• Cancel a queued or running move (not during the sub-second switch-over). The server stays on its source.
• Try again reruns a failed or cancelled move to the same node.

Every other kind of job (starts, stops, installs, backups, snapshots) has its own live log under Operations → Jobs; see Following a job.

Moves send transfer.started, transfer.finished and transfer.failed webhooks to staff webhooks only.