Storage Backend Comparison¶

Which storage backend to choose for a given VM or workload.

At-a-glance¶

Backend	Replication	Live Migration	Snapshot Speed	Best For
LINSTOR / DRBD	2 or 3-way sync	✅ (sub-sec)	Fast (LVM thin)	Production HA — default for new VMs.
Ceph RBD	3-way (size=3)	✅	Instant	Large pools, multi-tenant, > 4 nodes.
ZFS local	Async, pool-level (optional)	❌ (local only)	Instant	Single-host workloads, scratch, dev/test.
NFS	NAS-managed	✅ (slow)	Depends on NAS	Reusing an existing on-prem NAS.

Detailed comparison¶

Read/write latency (4K random)¶

Backend	Local read	Local write	Notes
ZFS	~0.1 ms	~0.2 ms	Limited by local NVMe.
LINSTOR	~0.3 ms	~0.5 ms	DRBD adds ~0.2 ms write penalty.
Ceph RBD	~0.8 ms	~1.5 ms	Network + RBD client overhead.
NFS	~0.5 ms	~1.5 ms	Highly dependent on NAS hardware.

Failure tolerance¶

Backend	1 node down	2 nodes down
ZFS local	VMs on that node are down unless replicated to another host and manually restored.	n/a
LINSTOR `--auto-place 2`	Survives — read-only on degraded node.	Resource offline.
LINSTOR `--auto-place 3`	Survives, no degraded state.	Survives with one healthy replica.
Ceph `size=3`	Survives — no degraded reads, slower writes.	Survives if PGs still have 1 replica.
NFS	Depends on NAS HA design (DRBD, RAID, etc.).

Snapshot model¶

ZFS — native CoW. Snapshot in O(1), no space cost until divergence. Best snapshot UX.
LINSTOR / DRBD — LVM thin snapshots underneath. Fast but thin pool fragments over time; periodic merges recommended.
Ceph RBD — native object-level snapshots. Free until divergence, then COW per 4-MB object.
NFS — depends entirely on the NAS. Don't rely on it for backups.

Capacity planning rough cut¶

LINSTOR 2-way → usable = raw / 2.
LINSTOR 3-way → usable = raw / 3.
Ceph size=3 → usable = raw / 3.
Ceph EC 4+2 → usable = raw × 4/6 ≈ 67 % efficient (vs 33 % for size=3). CPU cost on writes is much higher; pick for cold-ish data only.
ZFS — usable = raw minus parity (RAIDZ1 -1 disk; mirror /2; etc.) minus reservation (~10 %).
NFS — whatever the NAS says.

Golden image templates¶

Templates are converted in place — a template stays on whatever backend the source VM used, named <vm>_tmplt.qcow2:

Source backend	Template lands at	In the Deploy dialog
ZFS	`/<pool>/<vm>_tmplt.qcow2`	only on the converting node
NFS	`<mount>/<vm>_tmplt.qcow2`	only on the converting node¹
Local	`/var/lib/libvirt/images/<vm>_tmplt.qcow2`	only on the converting node
Ceph / LINSTOR	— not supported (no local file to convert)	—

A template is therefore node-local in practice: deploy from it on the node it was created on, or scp the _tmplt.qcow2 to another node first. Deploying from a template is independent of where the new VM lands — a ZFS golden image can clone onto Ceph, NFS, or local just fine.

Decision tree¶

Do I need the VM to migrate between hosts?
├─ NO → ZFS (fastest, simplest, single host).
└─ YES, and:
   ├─ I have < 4 hosts and care about latency → LINSTOR (default).
   ├─ I have 4+ hosts and want a shared pool → Ceph.
   └─ I already have a corporate NAS → NFS.

For most lab and small-cluster deployments LINSTOR is the right default and what MFCloud picks if the operator doesn't choose otherwise.