Skip to content

Add a Ceph Cluster

MFCloud bootstraps and manages its own hyperconverged Ceph cluster across your compute nodes (via cephadm) — it does not onboard a pre-existing external Ceph cluster. VM disks live on Ceph RBD, so any node in the cluster can read/write the same volume for live migration and HA.

Prerequisites

  • At least one Active compute node with a spare raw disk to dedicate as an OSD.
  • Root SSH access to that node (password auth) — the bootstrap SSHes in directly.
  • Network reachability between compute nodes on the Ceph MON/OSD port ranges (see Default Ports).

Bootstrap the cluster

  1. Storage → 🟢 Ceph (or wherever "Hyperconverged Storage" is offered) → Initialize Cluster.
  2. Fill in:
  3. Cluster Name — friendly label (defaults to ceph-cluster-01).
  4. Primary Monitor IP — the compute node cephadm bootstraps on; becomes the first MON.
  5. Root Password — that node's root SSH password, used once to run the bootstrap.
  6. Initialize Cluster. This runs in the background — cephadm bootstrap on the primary node, creates the fleet-wide default RBD pool (vms), and creates a client.libvirt cephx user scoped to it (mon 'profile rbd', osd 'profile rbd pool=vms').

Wire it up for VM disks

Once the cluster is up:

  1. 🔗 Link KVM to Ceph — generates the client.libvirt libvirt secret (virsh secret-define/secret-set-value) on every Active compute node.
  2. Sync Secret — re-pushes that same libvirt secret; run this again any time you enroll a new compute node, since it won't have the secret until this (or Link KVM) runs on it.
  3. Init HA Storage — finishes wiring the default vms pool for cross-node HA use.

Once done, Ceph appears in the Storage picker on the Create VM modal.

Adding OSDs (disks)

Storage → Ceph OSDs → Ceph tab → Add Ceph Disk (OSD):

  1. Pick a Compute Node — MFCloud scans it for raw, unpartitioned disks.
  2. Click a disk in Available Raw Disks to select it (adding it to Ceph permanently wipes existing data on the drive).
  3. Format & Add to Ceph Cluster.

Additional pools (per-cluster isolation)

Every VM authenticates to Ceph as client.libvirt regardless of which pool its disk lives in, so a new pool works out of the box for any VM once created — no separate cephx setup needed. In the same Ceph OSDs → Ceph tab:

  1. New Pool Name, Replication (default 3), PGs (default 128).
  2. Create Pool.
  3. Assign it to a cluster from that cluster's Edit modal (Dedicated Ceph Pool field, see Organize Hosts into Datacenters & Clusters) so VMs placed in that cluster deploy into it instead of the shared vms pool.

Pool sizing reference

Pool size OSDs needed Use case
size=2 2+ Lab / cost-sensitive (data loss possible on dual failure).
size=3 3+ Production default — survives one OSD failure with no degraded reads.
EC 4+2 6+ Cold-ish data, ~33 % storage overhead vs size=3's 200 %.

See also