LeilFS

Everything else in Hoster is shared-nothing: each host owns its pool, its guests live on it, and copies move between hosts by replication. That is the right default and it is why a node keeps working when the others do not.

LeilFS is the exception, for the cases where it is the wrong default: a workload that needs one filesystem visible from several hosts at once, or a dataset larger than any single machine’s disks.

What it is, and what Hoster does about it

LeilFS is a distributed filesystem — a separate piece of software with its own daemons, its own configuration and its own operational model. Hoster does not implement it and does not install it for you.

What Hoster does is make a node ready to participate and then tell you how it is doing:

Hoster does	Hoster does not
Provision a tuned dataset for chunk storage at install time	Install the LeilFS packages
Know the six service units and report their status	Start or configure the cluster for you
Read and write the nine configuration files	Validate what you put in them
Poll the cluster and present a health overview	Mount it into guests automatically

That boundary is the thing to understand before you plan around it. Treat LeilFS as a system you run alongside Hoster, which Hoster is aware of — not as a Hoster feature you switch on.

The name is mid-migration. The filesystem is LeilFS. On a node today the systemd units are still saunafs-*, the configuration lives in /etc/saunafs/ as sfs*.cfg, and the installer’s dataset is named saunafs — the older naming, which the rename has not reached yet. This page uses the concrete names where you would actually type them, and LeilFS everywhere else.

The dataset the installer creates

During installation, HosterOS provisions a dedicated dataset for chunk storage — this is the part that is genuinely automatic.

It is quota’d at one fifth of the root pool, and tuned for what a chunkserver does rather than for what guests do:

Property	Why
recordsize=1M	chunks are large files, up to 64 MiB
compression=lz4	cheap, and chunk data is often compressible
atime=off	nothing reads access times off a chunk store
logbias=throughput	streaming writes, not latency-sensitive ones
redundant_metadata=most	the filesystem above already replicates
exec=off, setuid=off, devices=off	it holds data, never anything runnable
com.sun:auto-snapshot=false	snapshotting a replicated chunk store duplicates work

The last one is worth pausing on: the chunk dataset is deliberately excluded from snapshots. LeilFS keeps its own redundancy across nodes, and snapshotting the chunks underneath it would store the same data twice under two schemes. Your data protection for anything on LeilFS is LeilFS’s replication goals, not Hoster’s snapshots.

The one-fifth quota is a default, not a recommendation for every workload. A node intended mainly as a storage node wants more; a node that will never chunk-serve wants none.

The services

Six units, covering the roles a LeilFS cluster is built from:

Unit	Role
saunafs-master	metadata server — the authority on where everything is
saunafs-ha-master	the HA variant of the above
saunafs-uraft	consensus for master failover
saunafs-metalogger	keeps a replica of the metadata, for recovering a master
saunafs-chunkserver	stores the actual data, on the dataset above
saunafs-cgiserv	the filesystem’s own web interface

Hoster reports their status as part of its component view; starting, stopping and enabling them is systemctl — see hoster service only knows two services.

A typical layout puts the master role on three nodes with uraft between them, a metalogger somewhere that is not a master, and the chunkserver on every node with disks to contribute. Which nodes take which role is a LeilFS design decision, and its own documentation is the authority.

Configuration

Nine files in /etc/saunafs/, readable and writable through Control Center or the agent API:

File	For
sfsmaster.cfg	the metadata server
sfsexports.cfg	which clients may mount what, and with which permissions
sfsgoals.cfg	replication goals — how many copies of a chunk, and where
sfstopology.cfg	rack and location awareness, so copies are spread
sfsmount.cfg	client mount defaults
sfschunkserver.cfg	the chunk server
sfshdd.cfg	which paths the chunkserver stores chunks in
sfsmetalogger.cfg	the metadata replica
saunafs-uraft.cfg	master failover consensus

sfsgoals.cfg is the one that decides your redundancy, and sfshdd.cfg is the one that must point at the dataset the installer created. Those two are where a misconfiguration costs you data rather than uptime.

Writing them is deliberate but unguarded

A write is done safely at the file level: the previous contents are copied to <name>.bak, the new contents go to a temporary file, and that is renamed into place — so an interrupted write cannot leave a half-written config. Existing file permissions are preserved. Only the nine names above are accepted.

Nothing validates the contents. The allowlist checks the filename; the body is written as given. A syntactically invalid or semantically wrong config is accepted and only fails when the service restarts and refuses to start.

And writing a config does not reload anything. The file changes; the running daemon does not. You have to restart the relevant unit yourself, which is also the moment a bad config surfaces. Check <name>.bak if you need to get back.

Monitoring it

Hoster polls the cluster and presents a single overview, surfaced in Control Center and available from the agent API at /api/v1/host/saunafs/overview. It reports:

• Availability, and when it is not available, the reason and detail — so “no data” is distinguishable from “cluster is down”.
• Summary: version, total, available, trash and reserved space, node and directory counts, memory use of the master.
• Metadata servers and metadata loggers, with their status.
• Chunkservers, and the disks behind each one.
• Chunk health against the configured goals — the number that tells you whether your redundancy is actually satisfied.
• Mount count, and any warnings.

Chunk health is the field to watch. Everything else tells you the cluster is running; chunk health tells you whether the copies you asked for in sfsgoals.cfg actually exist. A cluster that is up with under-replicated chunks is one disk away from a bad day, and it looks healthy in every other field.

There is no CLI

LeilFS is not in the hoster command at all. No hoster leilfs, no hoster fs, no subcommand elsewhere. The overview and the configuration files are reachable only through Control Center or the agent API — the library implements the client, and nothing on the CLI calls it.

On a node with only a terminal, LeilFS is operated with the filesystem’s own tools and systemctl, and Hoster’s view of it is not available to you. This is the widest instance of a pattern that shows up elsewhere — here it is the whole subsystem rather than one command.

When to use it, and when not to

Reach for LeilFS when:

• Several guests, on different hosts, must see the same files at the same time.
• A dataset is larger than any single node’s disks.
• You want storage capacity to scale by adding nodes rather than by growing disks.

Stay with plain ZFS and replication when:

• Guests own their own disks and share nothing — the overwhelmingly common case.
• You want the recovery story on Restoring a guest, which is built on ZFS snapshots and does not apply to data living on LeilFS.
• You have fewer than three nodes. Master failover needs consensus, and the same quorum argument that applies to clustering applies here.

The honest summary: LeilFS solves a real problem, and it is a second distributed system to operate. Adopt it because a workload needs shared storage, not because it sounds like the more capable option.

Next

• Clustering — the cluster LeilFS is deployed across, and the quorum requirement it shares.
• Snapshots, replication and the scheduler — the shared-nothing data protection LeilFS deliberately sits outside.
• Troubleshooting a node — hoster host zfs and hoster datasets, for looking at the pool underneath all of this.