Install Fuzzball on an NVIDIA DGX Spark

This guide walks through bringing up a complete Fuzzball cluster on a single NVIDIA DGX Spark using the fuzzball cluster docker-compose deploy flow with GPU passthrough enabled. Once finished, you’ll have an orchestrator, web UI, auth (Keycloak), persistent state (PostgreSQL), and a substrate node that schedules GPU workflows onto the DGX Spark’s Blackwell GPU — all running on the single host.

The DGX Spark is a Grace Blackwell GB10 personal supercomputer: linux/arm64, 128 GB of unified LPDDR5x, an integrated Blackwell GPU, and DGX OS (Ubuntu-derived). Because every Fuzzball image is published as a multi-arch manifest, no custom builds are required — the operator just runs the standard CLI.

The compose stack brings up five services on the host:

The substrate runs workflow jobs as rootless OCI containers directly on the DGX Spark — no Kubernetes, no cloud, no external infrastructure. The GPU passes through to workflow containers via the NVIDIA Container Toolkit on the host.

• DGX OS or an Ubuntu-derived distribution shipping the NVIDIA Blackwell driver. Confirm with:

  $ nvidia-smi

  Expected output: at least one GPU listed with a non-empty driver version.

The Fuzzball images (orchestrate + substrate GPU) are pulled from depot.ciq.com. If your Docker daemon isn’t already authenticated to it, log in first with the Depot credentials from the CIQ sales/support team:

$ docker login depot.ciq.com

Install the fuzzball command-line tool for linux/arm64. Follow the CLI installation guide — on DGX OS (Ubuntu-derived), use the arm64 .deb package. Once installed, confirm the binary runs:

$ fuzzball --version

The single command that wires everything up:

$ fuzzball cluster docker-compose deploy \
    --gpu \
    --update-etc-hosts \
    --up

What each flag does:

• --gpu — swap the substrate image to the GPU-enabled variant (fuzzball-substrate-orchestrate-gpu), which bakes in the NVIDIA device plugins.
• --update-etc-hosts — append the ui, api, keycloak, endpoints, and nats hostnames to /etc/hosts pointing at 127.0.0.1. Requires sudo (you’ll be prompted). Skip this flag if you’d rather inspect the snippet first; the CLI prints the lines it would add to stdout.
• --up — run docker compose up -d immediately after rendering the deployment files.

/etc/hosts doesn’t support wildcards, so the per-endpoint hostnames endpoint-<sha1>.endpoints.fb.local — which serve workflow service endpoints (Jupyter, RStudio, novnc) — aren’t covered by --update-etc-hosts (it only adds the bare endpoints.fb.local). Two ways around this:

Option A — add per-endpoint entries by hand each time you submit a workflow that exposes a service. List the live endpoints and their URLs with fuzzball workflow endpoints list -o json, then add the corresponding hostname to /etc/hosts.

Option B (recommended) — run a lightweight DNS resolver. A single dnsmasq wildcard covers every subdomain (ui, api, keycloak, nats, and each endpoint-<sha1>.endpoints), so it also replaces --update-etc-hosts:

$ sudo apt-get install -y dnsmasq
$ sudo tee /etc/dnsmasq.d/fb-local.conf <<EOF
address=/fb.local/127.0.0.1
EOF
$ sudo systemctl restart dnsmasq

Then point your DGX Spark’s resolver at dnsmasq (typically by editing /etc/resolv.conf or systemd-resolved’s upstream list).

Before the CLI can reach the cluster you must trust the deployment’s self-signed certificate (or connect with --insecure). fuzzball cluster docker-compose info prints the service URLs, the exact command to trust the certificate, and the ready-to-paste context commands:

$ fuzzball cluster docker-compose info
Deployment:
  Name:      default
  Directory: /home/dgx/.config/fuzzball/docker-compose/default

Service URLs:
  UI:        https://ui.fb.local
  API:       https://api.fb.local
  Keycloak:  https://keycloak.fb.local
  Endpoints: https://endpoints.fb.local
  JetStream: nats://nats.fb.local:4222

Configuration:
  ... (deployment .env values; passwords hidden unless you pass --secrets)

TLS:
  Certs: /home/dgx/.config/fuzzball/docker-compose/default/certs
  To trust the deployment's wildcard certificate:
    export SSL_CERT_DIR="/home/dgx/.config/fuzzball/docker-compose/default/certs"

Connect:
  fuzzball context create default https://api.fb.local https://keycloak.fb.local/realms/e700253c-8ce8-4e36-a420-1e4cab4cb7ef fuzzball-cli
  fuzzball context login

UI:
  Open https://ui.fb.local in your browser

Run the export SSL_CERT_DIR=... line and the two fuzzball context commands exactly as printed. context login launches the device-code flow in your browser against Keycloak; sign in as the admin user admin@fb.local plus an auto-generated password (reveal it again with fuzzball cluster docker-compose info --secrets).

$ fuzzball node list --available-resources
NODE ID         | HOSTNAME              | CPU TYPE  | AVAILABLE CORES | AVAILABLE MEMORY (GB) | AVAILABLE DEVICES | RUNNING JOBS | CLUSTER
172.21.0.5/7333 | substrate-localnode-1 | cpu/arm64 | 20              | 127.9                 | nvidia.com/gpu:1  | 0            | local-dev

If AVAILABLE DEVICES shows nvidia.com/gpu:N, the substrate-side GPU plumbing is working.

pull access denied for depot.ciq.com/...

Docker isn’t authenticated to the registry. Run docker login depot.ciq.com with your Depot credentials, then re-run the deploy.

High memory pressure or OOM during workflow runs

The DGX Spark’s 128 GB is unified between CPU and GPU — a large GPU-resident model competes with the rest of the stack. The base Fuzzball stack rests at ~2-3 GB. Leave at least 4-8 GB headroom for the control plane when sizing GPU workflow memory budgets.

Browser doesn’t trust the cert

deploy generates a self-signed root CA (rootCA.pem) and a wildcard leaf cert under the deployment’s certs/ directory (~/.config/fuzzball/docker-compose/default/certs/). Either accept the browser warning (fine for local development) or import rootCA.pem into your OS/browser trust store. To make the fuzzball CLI trust it, point SSL_CERT_DIR at that certs/ directory.

To bump the stack to a newer Fuzzball image tag:

$ fuzzball cluster docker-compose update --upgrade

--upgrade moves the deployment to the version your current fuzzball CLI binary was built against; use --version <tag> to pick a specific one instead. The same update command rotates credentials (--reset-owner-password, --reset-keycloak-password, --reset-database-password) and renews the wildcard TLS leaf certificate (--tls-renew) – see fuzzball cluster docker-compose update --help for the full set.

$ fuzzball cluster docker-compose down

This stops the stack but preserves the persistent volumes (postgres-data, jetstream-data, substrate-shared-fs, etc.) — useful when you want to come back to your workflows later. To completely remove the deployment including persistent state:

$ fuzzball cluster docker-compose delete

A multi-host topology — DGX Spark + remote substrate nodes — is supported via fuzzball cluster docker-compose generate-substrate-config, which exports the orchestrate’s CA, NATS NKey, and connection config for deployment to a separate machine. See --help on that command and Federation for the cross-cluster story.

Hardening for production (real TLS via cert-manager / Caddy, external managed Postgres + Keycloak, backup automation, log shipping) is covered in Cloud Deployment — most of the recommendations there apply equally to a single-host DGX Spark deployment when you want to move beyond local development.