Updating Fuzzball

Updating Fuzzball is a straightforward process that consists of updating the Substrate installation running on compute nodes, the Orchestrate cluster, and ensuring that client CLIs are up-to-date.

Update Substrate on the compute nodes

Assuming that you installed Fuzzball substrate from the Depot yum/dnf repository, updating substrate is simply a matter of running dnf update or similar.

Substrate is backward compatible with Orchestrate, so the newest version of Substrate will always run with your Orchestrate installation. But don’t forget to update Substrate when you update Orchestrate since an older Substrate installation may be incompatible with a newer version of Orchestrate.

Updating the Orchestrate cluster

It’s easy to update your Fuzzball Orchestration cluster using the same K8s Custom Resource Definition (CRD) that installs Fuzzball. Simply repeat the steps that you used to install the Fuzzball Operator with a new version.

You can safely update from one patch release to another non-sequentially. (e.g Updating from v2.1.8 to v2.1.10 is supported.) But you should avoid skipping major or minor releases when updating.

First, make sure that you are logged into CIQ Depot.

# DEPOT_USER="" # populate with your username for CIQ Depot

# ACCESS_KEY="" # populate with the Depot key obtained from the CIQ sales/support team

# helm registry login depot.ciq.com --username "${DEPOT_USER}" --password "${ACCESS_KEY}"

Now simply update the version number and run the same commands that you originally ran to install the Fuzzball Operator the first time.

# VERSION="" # insert the updated version number

# CHART="oci://depot.ciq.com/fuzzball/fuzzball-images/helm/fuzzball-operator"

# IMAGE="depot.ciq.com/fuzzball/fuzzball-images/fuzzball-operator"

# helm upgrade --install fuzzball-operator "${CHART}" \
  --namespace fuzzball-system --create-namespace \
  --version "${VERSION}" \
  --set "image.repository=${IMAGE}" \
  --set "image.tag=${VERSION}" \
  --set "imagePullSecrets.name=repository-ciq-com" \
  --set "imagePullSecrets.inline.registry=depot.ciq.com" \
  --set "imagePullSecrets.inline.username=${DEPOT_USER}" \
  --set "imagePullSecrets.inline.password=${ACCESS_KEY}" \
  --set "storageClassName=local-path"

Running this command will automatically re-deploy the Orchestrate cluster with updated images. You can monitor the progress of the update with the same command that you ran when first installing the cluster.

# kubectl logs -l app.kubernetes.io/name=fuzzball-operator -n fuzzball-system -f --tail=-1

Under unusual circumstances, the update may not start automatically. For instance, if you incorrectly specify your Depot username or password the deployment will not proceed. Correcting this mistake will not cause the deployment to restart automatically. In cases like this, you have the option to manually start the redeployment process. Here is an example.

# kubectl get deployments -A
NAMESPACE            NAME                                   READY   UP-TO-DATE   AVAILABLE   AGE
cert-manager         cert-manager                           3/3     3            3           26h
cert-manager         cert-manager-cainjector                1/1     1            1           26h
cert-manager         cert-manager-webhook                   1/1     1            1           26h
cert-manager         trust-manager                          1/1     1            1           26h
fuzzball-system      fuzzball-operator-controller-manager   1/1     1            1           26h
[...snip]

# kubectl rollout restart deployment/fuzzball-operator-controller-manager -n fuzzball-system
deployment.apps/fuzzball-operator-controller-manager restarted

# kubectl logs -l app.kubernetes.io/name=fuzzball-operator -n fuzzball-system -f --tail=-1
[...snip]
Resources:
    + 7 created
    265 unchanged

Duration: 10s

2025-05-29T21:05:00Z	DEBUG	events	Resources have been deployed successfully	{"type": "Normal", "object": {"kind":"FuzzballOrchestrate","name":"fuzzball-orchestrate","uid":"22ca168d-a7a7-46f8-a144-33b42937f50f","apiVersion":"deployment.ciq.com/v1alpha1","resourceVersion":"11472"}, "reason": "DeploymentSucceeded"}
2025-05-29T21:05:00Z	INFO	Updated Fuzzball status to ReconciliationComplete - Reconciliation completed successfully	{"controller": "fuzzballorchestrate", "controllerGroup": "deployment.ciq.com", "controllerKind": "FuzzballOrchestrate", "FuzzballOrchestrate": {"name":"fuzzball-orchestrate"}, "namespace": "", "name": "fuzzball-orchestrate", "reconcileID": "162d9224-5d75-49a7-829b-cc6ad8381ebe"}

In the example above, we first determine the name of the deployment that we want to restart (fuzzball-operator-controller-manager). We then use the proper kubectl command to restart the deployment process. Finally, we use the appropriate command to monitor the logs as the deployment proceeds to successful completion.

Update the Fuzzball CLI on clients

The exact method that you use to update the Fuzzball CLI will depend on the way in which you originally installed it. See the CLI installation documentation for more information.

Older versions of the Fuzzball CLI are not guaranteed to work with newer versions of Orchestrate, so it’s a good practice to keep it up to date. This may require announcing updates to your users so that they can update the CLI on their personal machines too!