Skip to content

Update a Console Without Internet Access

A console with no internet access can't pull a new image from Docker Hub. Instead, bring the release in as one signed file, a release bundle (.mfbundle), and apply it from the console's own release store. Nothing is downloaded at any step.

A bundle holds the console image, its software bill of materials, the code the console pushes to compute nodes, and the release notes. Its manifest lists every file with its size and SHA-256, and is signed by the vendor with ECDSA P-384.

A bundle can also carry packages for the compute nodes. Once the console runs that release, install them from Settings → Node Updates: see Update Compute Nodes from a Release.

1. Import the bundle

Settings → Release Updates, as a Super Admin. Either:

  • Upload bundle… and pick the .mfbundle file, or
  • copy the file into the release store's incoming folder on the console host, then choose Check incoming folder:

    docker compose cp mfconsole-1.3.0-rc2.mfbundle app-1:/app/release_store/incoming/
    

The console checks the signature first, then every file against the manifest. If anything doesn't match, it keeps nothing and says why; a rejected file in incoming is renamed .rejected. A bundle signed with a key the console doesn't trust is refused.

Three-host control plane

Each console host keeps its own copy of the store. After an import, the other hosts fetch the release from the one that took it, check every file against the signed manifest, and the Copies column counts the hosts that hold a verified copy. It takes about a minute per host.

Or download it from the online release channel

A console that can reach the internet (or just the release channel's host) can fetch the same bundles instead of carrying them in. Under Online release channel on the same card:

  • the console reads the channel's release index every 6 hours, and on Check now; the index is signed with the same key as the bundles;
  • releases newer than the running console are listed with a Download button, and a release.available notification is sent once per new version;
  • Download new releases automatically (off by default) fetches the newest one this console can update to on each check.

A downloaded bundle goes into the release store through the same checks as an uploaded one and is copied to every console host. Nothing is applied until you run the update below.

The vendor's channel, and the default, is https://dl.mfcloud.io/releases/. With sovereign mode on, add the channel's host (dl.mfcloud.io) to the allowed outside hosts to keep the console offline for everything except updates. The channel URL can point at your own mirror: copy the vendor's folder (the bundles plus index.json and index.json.sig) to any web server the console can reach. It must serve the files directly, because the console doesn't follow redirects.

2. Apply it

In the list, Details on a release that is newer than the running console shows the command to run. On the console host, in the install folder (next to docker-compose.yml):

./update-from-store.sh 1.3.0-rc2

The script:

  1. has the console check the release again: signature, every file, and that this console may update to it;
  2. loads the console image from the store into Docker or Podman;
  3. points MFCONSOLE_IMAGE in .env at that image, saving the old .env as .env.before-<version>;
  4. restarts app, worker and beat on the new image.

The console comes back in about 30 seconds. After that, it pushes the new compute-node code to every active node on its own, as an online update does.

On a three-host control plane, run it on each host and name that host's app service: MFCONSOLE_APP_SERVICE=app-2 ./update-from-store.sh 1.3.0-rc2.

A release older than the one running, or one that needs an intermediate version first, is listed but refused by the script.

Going back

Restore the saved .env and recreate the containers. The previous image is still in the container engine:

cp .env.before-1.3.0-rc2 .env
docker compose -f docker-compose.yml -f compose.prod.yml --profile single up -d --force-recreate --no-deps app-1 worker beat

Deleting a release

Delete removes the release from every console host's store. It doesn't affect the version that is running.

Kubernetes and image content (1.3.0-rc5)

The store also takes content bundles: signed .mfbundle files that carry content instead of a console release. Import them the same way (upload, or the incoming folder). Release Updates lists each one with what it carries, copies it to every console host and deletes it the same way. A content bundle is never applied with update-from-store.sh.

  • Kubernetes content holds k3s and RKE2 with their air-gap image archives, Helm, the Ceph CSI and MetalLB charts, the images those and kube-vip need, and a few RPMs minimal guests lack. While the store has it, every Kubernetes cluster whose engine it carries installs from it: the console copies the files to each node over SSH, and no node fetches anything from the internet. This covers new clusters, added nodes, and Ceph storage or MetalLB added later. With sovereign mode on, the console no longer refuses those requests. LINSTOR and ZFS storage on Kubernetes and GPU workers still need the internet, because they install kernel modules inside each guest.
  • Image content holds cloud images, ISOs (including virtio-win.iso) and LXD images. They appear in Storage → Content Library → From the release store. Copy to nodes… has the console copy a template or ISO into each node you pick, or import an LXD image into each LXD node you pick.

When the store holds several content bundles of one type, the newest is used. Delete one, and new installs fall back to the next newest, or to the internet when none is left. Clusters and images already installed from it are not affected.

The vendor builds content with make_k8s_content.sh or make_image_content.py and signs it with make_content_bundle.py (mfconsole-ship).

API

Call Notes
GET /api/system/releases releases and content bundles in the store, copies per host, whether each release can be applied
GET /api/library/store-images images the newest image content offers, plus the Active nodes
POST /api/library/store-images/{id}/copy?targets=all\|ip,ip copy one image to nodes from the control plane
POST /api/system/releases/upload body is the .mfbundle (application/octet-stream); audited
POST /api/system/releases/scan import everything in incoming; audited
DELETE /api/system/releases/{version} audited
GET /api/system/releases/channel channel settings, last check, listed releases, download progress
PUT /api/system/releases/channel {"url", "enabled", "auto_download"}; audited
POST /api/system/releases/channel/check read the index now; audited
POST /api/system/releases/channel/download {"version"}; starts the download and returns at once; audited

All are Super Admin only.