Snapshots, replication and the scheduler

Snapshots that only run when someone remembers are not backups. This page is about making them happen on their own — and then about getting a copy of them somewhere else, which is the part that actually survives a dead machine.

Start here: a freshly installed node has no scheduled snapshots. The installer sets up storage, networking and services, but it does not create scheduler jobs. Until you run the command in the next section, the only snapshots that exist are the ones you took by hand.

The three layers

They stack, and each one protects against something the one below it does not.

A snapshot on the same disk as the thing it protects is not a backup of that disk. It is an undo button, and an excellent one. Replication is what turns it into a copy.

Give the node a snapshot schedule

One command writes a sensible default schedule:

hoster scheduler generate-snapshot-jobs

Look before you leap — --dry-run prints the file it would write instead of writing it:

hoster scheduler generate-snapshot-jobs --dry-run

That produces /opt/hoster/conf/snapshot_schedule.json with six jobs:

Read that as a retention ladder rather than six unrelated jobs. Fifteen frequent snapshots is the last few hours at fifteen-minute resolution; twenty-four hourly is yesterday; seven daily is last week; and so on out to two years. Each type prunes only its own kind, so the hourly job cannot delete your monthly snapshots.

Every default job is online-only — it snapshots guests that are running, and skips the ones that are not.

yearly is in the schedule but not in the CLI’s help. The generated schedule uses seven types; hoster vm snapshot create --type lists six, omitting yearly. The scheduler and the CLI are describing the same set of retention buckets and disagreeing about it.

Retention is per type, and the CLI shares it

This is the same --type and --keep pair from Your first VM. A snapshot you take by hand with --type daily lands in the same retention bucket as the scheduled daily job and will be pruned by it. If you want one that does not get swept up, give it a type nothing else is using — custom exists for exactly that.

Changing the schedule

The file is JSON and you can edit it. The thing worth knowing is what happens when you regenerate afterwards:

Regenerating does not destroy your own jobs. generate-snapshot-jobs rewrites the six generated entries and re-appends every entry marked IsCustom. So a job you added survives; a default you edited in place does not. If you want a modified schedule to stick, add it as a custom entry rather than changing a generated one.

Seeing what the scheduler is doing

hoster scheduler schedule     # what is configured
hoster scheduler jobs         # what has run, and how it went
hoster scheduler stats        # totals

All three take --output table|unix|json, with --json as a shortcut — unix is the one to reach for from a shell script. Each has per-category forms too, so hoster scheduler jobs snapshot narrows it to snapshots, and the same for replication and templates.

Do not wait an hour to find out you got it wrong

The scheduler can run a job immediately, using the local executor:

hoster scheduler debug run <job-selector>

That is the difference between testing a schedule and hoping about one. Change the entry, run it now, look at hoster scheduler jobs, then leave it to the cron expression.

Replication: getting a copy off the machine

Replication sends a guest’s data to another host as an incremental ZFS send, so after the first run it moves only what changed.

In a cluster, the schedule is generated from cluster state — Hoster already knows the hosts and what is on them:

hoster scheduler generate-replication-jobs --hourly

Standalone, without a cluster, you name the target yourself. This is the one people assume does not exist — it does, and it is how a two-machine setup gets off-site copies without becoming a cluster:

hoster scheduler generate-standalone-replication-jobs \
  --vm my-first-vm --vm another-vm \
  --remote-host 192.0.2.30 \
  --target-name backup-box \
  --hourly

--vm is repeatable. The remote side is reached over the agent’s gRPC API on port 8443 by default with mTLS, falling back to the HTTP API on 8080 — --remote-port, --remote-user, --remote-protocol, --remote-grpc-port and --remote-dataset-base are all there when the defaults do not fit.

The written plan lands in /opt/hoster/conf/replication_schedule.json.

Replication is not a backup either, quite. It is a copy of your data as it is, including a mistake you made ten minutes ago, propagated on the next run. Snapshots on the far side are what let you go back past the mistake — which is why --enable-replication-snapshot exists.

Disaster recovery

DR backups are a separate schedule, written to /opt/hoster/conf/dr_schedule.json, and they work per resource: each VM and microVM carries its own DR target, configured on the guest rather than centrally.

hoster vm dr-config my-first-vm --enable --url … --username … --password … --schedule daily
hoster vm dr-backup my-first-vm      # run one now

The equivalents exist for microVMs — hoster microvm dr-config, dr-backup.

Enabling DR is enough to schedule it. The scheduler synthesises a job from each guest’s own DR configuration, so a guest you enable today is backed up tonight without a generate step. hoster scheduler generate-dr-jobs still exists and writes an explicit plan file; entries in that file win on a name clash, which makes it an override for a guest that needs a different cadence — not a step in the normal path.

→ Disaster recovery for what DR Manager is, why repositories stay closed until a host knocks, and how to restore from one.

Because the target is per guest, a fleet does not have to agree on one destination: the database can go somewhere with retention you pay for, and the scratch build runner can go nowhere at all.

Templates keep themselves in step

The fourth job type is not about your data — it distributes image templates so a node is never missing the template you are about to deploy from:

hoster scheduler generate-template-jobs

That writes hourly jobs to /opt/hoster/conf/template_schedule.json. In a cluster it is what keeps every node’s templates current; see MicroVMs for the manual equivalents.

A sensible starting point

For a single node you actually care about:

hoster scheduler generate-snapshot-jobs
hoster scheduler generate-standalone-replication-jobs \
  --vm important-thing --remote-host <second-machine> --hourly
hoster scheduler schedule

Snapshots for undo, an hourly copy on another machine for the disk dying, and a look at what you just configured. Add DR when there is something whose loss would cost more than the second machine did.

Next

• Why Hoster — where this sits in the wider design, including what clustering adds.
• Your first VM — the by-hand snapshot and clone commands these jobs automate.
• The scheduler — inspecting what these jobs actually did, and running one now instead of waiting for its cron.
• Restoring a guest — using the restore points this page creates, which is the half most people only read during an incident.
• Disaster recovery — DR Manager, and restoring when the hosts and the replicas are both gone.