VorticPanel

Encrypted, off-node backups for a KVM VPS fleet

Backing up KVM VPS servers to S3 or SFTP: incremental chains, encryption on the node, what a broken-into node can reach, and restores that check every piece.

4 min read

Snapshots in VorticPanel are kept next to the server's disk on its node, so a failed disk or a lost node takes them along with the server. Backups are the copies that leave the node, and once they do, they sit on storage someone else runs. This post covers where VorticPanel sends them, how they're encrypted before they go, and the one part of the setup that's on you.

Where backups go

Server backups go to an S3-compatible bucket or an SFTP server. Each location backs up to one destination, set under Fleet → Backup storage, and Test uploads, reads back and deletes a test file.

Nodes don't hold the storage's credentials. With S3, the controller hands a node a signed link for each object a backup or restore needs, valid for an hour. With SFTP, nodes read and write through the controller, which signs in to the SFTP server itself.

S3 is the stronger choice, and with versioning or object lock on the bucket, a deleted backup can come back. For SFTP, give the panel an account that can reach the backup directory and nothing else, with ChrootDirectory and ForceCommand internal-sftp in the server's sshd_config. The controller records the SFTP server's host key at the first test and refuses any other.

Plans

A backup plan sets how often backups run (every 1 to 12 hours, daily, weekly or monthly), whether they're incremental or full, how often an incremental chain starts with a new full backup, and how many restore points to keep. Each package has a default plan, and one server can have its own.

An incremental backup sends only the chunks that changed since the last backup in its chain, so daily backups of a mostly idle server cost little. Retention removes a whole chain at a time, so everything kept can still be restored. At most two scheduled backups run at once on a node.

Encrypted on the node

Every chunk is compressed with zstd and sealed with AES-256-GCM on the node, before upload, and named by an HMAC rather than a plain hash of its content.

Each backup storage has a root key that stays on the controller. Each server's backups have a key derived from it, and a node gets only the key of the server it's backing up or restoring, for that task. If someone breaks into a node, they get the keys of the servers that node has worked with: they can't read other servers' backups, or change a backup so that it still restores.

A restore checks the seal on every piece, and checks the disk it wrote against the SHA-256 the controller recorded when the backup was made. Downloads are checked against the same SHA-256 and cut short if they don't match. What's checked when a backup is restored or downloaded has the details.

Customers restore their own backups from their server's Backups tab, in place or onto another of their servers with at least as much disk, and can download one through a link that lasts an hour.

Back up the controller too

The backup keys live in the controller's database, encrypted with its secret.key. Lose both and no backup can be decrypted. The installer copies the database and secret.key to /var/backups/panel every hour, but that's on the same server.

Turn on Settings → Backups in the panel to send encrypted copies of the controller to a backup storage every hour or every day, under a passphrase you choose. Pick a storage that isn't with the controller's own provider, write the passphrase down away from the server, and keep a copy of secret.key apart from the database copies. See Backing up the controller.

Selling backups

A package's default plan is included with every server on it. To sell backups as an add-on instead, add a Backups configurable option to the product in WHMCS or Blesta, with a choice for each plan you sell and one for none: the module puts each server on the plan the client picked, when it's ordered and when the client changes it. Other billing panels set a server's plan through the API (PUT /servers/{id}/backups/plan), and resellers can put only the plans you allow them on their customers' servers.

Fleet → Backups lists every server's backups across the fleet, with totals and failures at the top. Its filters are kept in the page's address, so a view such as failed backups on one node can be bookmarked.

Related