Driver Types

Fuzzball V4 includes seven built-in storage drivers. Each driver handles a different type of storage backend. All drivers are built into the Fuzzball platform — no external container-based drivers need to be installed. The on-premises drivers are NFS and Hostpath; the cloud drivers are AWS EFS, Google Cloud Filestore, Azure Files, Oracle Cloud File Storage (FSS), and Oracle Cloud Managed Lustre.

You can list the drivers available on your cluster, along with their required and optional configuration fields, with:

$ fuzzball volume provisioner list-drivers

The NFS driver manages volumes as subdirectories on an NFS export. It is the primary driver for on-premises multi-node clusters where shared storage is provided by an NFS server.

shared-nfs:
  description: "Shared NFS storage"
  driver:
    type: nfs
    target: "nfs-server.internal:/exports/fuzzball"
    version: 4
    options: "hard,intr"
  access: all
  ephemeral: all

When a volume is created, the driver:

1. Mounts the NFS export to a temporary location
2. Validates available capacity on the export
3. Creates a subdirectory named after the volume
4. Sets POSIX ownership (UID/GID) on the directory
5. Unmounts the temporary mount

When a volume is published (mounted into a workflow container), the driver mounts the specific volume subdirectory to the target path:

NFS Export Root (/exports/fuzzball)
├── my-volume/        ← volume directory, mounted to container path
├── project-data/
└── scratch-wf-123/

NFSv3 and NFSv4 have different mount option syntax and behavior. If you specify mount options in the options field, ensure they are compatible with the NFS version you select.

NFS exports configured with root_squash will remap root (UID 0) operations to the nobody user. This affects volume creation and ownership assignment. If your NFS server uses root_squash, configure the export with no_root_squash for the network range where Fuzzball substrate nodes reside, or use the volume’s UID/GID settings to work within the squash rules.

The hostpath driver manages volumes as directories under a base path on the local filesystem. It supports two modes: shared (default) and node-local.

In shared mode (local: false or omitted), the base path is expected to be available on all nodes in the cluster — typically via a shared filesystem like Lustre, GPFS, or a network mount. Volumes can be accessed from any node.

lustre-storage:
  description: "Lustre shared filesystem"
  driver:
    type: hostpath
    path: /mnt/lustre/fuzzball
  access: all
  ephemeral: all

In node-local mode (local: true), volumes exist only on the node where they were created. Fuzzball’s scheduler ensures that jobs using node-local volumes are placed on the correct node.

local-nvme:
  description: "Local NVMe scratch"
  driver:
    type: hostpath
    path: /mnt/nvme0n1
    local: true
  ephemeral:
    - compute-jobs

Node-local volumes are ideal for ephemeral scratch storage on fast local disks (NVMe, SSD). They are not suitable for persistent data that needs to survive node failures or be accessed from multiple nodes. When local: true is set, Fuzzball’s scheduler constrains job placement to the node where the volume was created — this applies to both ephemeral and persistent volumes using this provisioner.

When a volume is created, the driver:

1. Validates available capacity at the base path
2. Creates a subdirectory named after the volume
3. Sets POSIX ownership (UID/GID) on the directory

When a volume is published, the driver creates a bind mount from the volume directory to the target path:

Base Path (/mnt/fuzzball)
├── volume-a/         ← bind-mounted to container path
├── volume-b/
└── scratch-wf-456/

The EFS driver manages volumes as access points on an Amazon Elastic File System. It is designed for AWS cloud deployments where shared, elastic storage is needed.

Provide filesystemId or subnetIds, not both. Use filesystemId to attach an existing EFS filesystem (BYO mode). Use subnetIds to have Fuzzball create the EFS filesystem and mount targets automatically (self-provisioned mode).

BYO mode — use an existing EFS filesystem:

aws-efs:
  description: "AWS EFS for shared storage"
  driver:
    type: efs
    filesystemId: fs-12345678
    region: us-west-2
  access: all
  ephemeral: all

Self-provisioned mode — Fuzzball creates the EFS infrastructure:

aws-efs-auto:
  description: "Auto-provisioned AWS EFS"
  driver:
    type: efs
    region: us-west-2
    subnetIds:
      - subnet-abc123
      - subnet-def456
  access: all
  ephemeral: all

The EFS driver is unique among Fuzzball’s drivers — it is a control-plane driver, meaning volume creation and deletion happen via AWS API calls from the Fuzzball control plane, not from a substrate node. This provides faster operations and eliminates the need for a running substrate node during volume management.

When a volume is created, the driver:

1. Creates an AWS EFS access point on the filesystem
2. Sets the access point’s root directory to /<volume-name>
3. Configures POSIX identity via the access point’s PosixUser and CreationInfo (UID, GID, and directory permissions)

When a volume is published (mounted into a workflow container), the driver:

1. Mounts the access point via NFS 4.1 to the target path
2. Mount uses the access point’s root directory for isolation

EFS Filesystem (fs-12345678)
├── /volume-a/    ← access point root, mounted to container
├── /volume-b/    ← separate access point
└── /scratch-123/ ← ephemeral access point

Before creating an EFS provisioner, ensure:

• The EFS filesystem exists and is in the available state
• Mount targets are available in the subnets where Fuzzball nodes run
• Security groups allow NFS traffic (TCP port 2049) from the node subnet
• IAM permissions allow elasticfilesystem:CreateAccessPoint, elasticfilesystem:DeleteAccessPoint, and elasticfilesystem:DescribeAccessPoints

EFS storage is elastic — there are no per-access-point capacity quotas. The size field in volume definitions is accepted but not enforced at the EFS level.

The Azure Files driver manages volumes as subdirectories of an Azure Files NFS share. Each volume is a subdirectory in the share, mounted on substrate nodes over NFSv3. It is designed for Azure cloud deployments.

The Azure Storage account must have the NFS protocol enabled on the file share. SMB shares are not supported by this driver.

azure-files:
  description: "Azure Files NFS storage"
  driver:
    type: azure_files
    accountName: fuzzballstorage
    shareName: workflowio
  access: all
  ephemeral: all

Azure deployments bootstrapped before this release created two built-in provisioners named persistent and ephemeral instead of a single default. Both are backed by the same Azure file share, and neither can be selected by name in a workflow, because persistent and ephemeral are reserved values of a volume’s use field.

Upgrading creates the correct default provisioner but leaves the old pair in place, so no existing volume becomes unreachable. To finish the consolidation:

1. List the volumes still recorded on the old provisioners:

   $ fuzzball volume list --provisioner persistent
   $ fuzzball volume list --provisioner ephemeral

2. Point workflows at default (or omit use entirely to auto-select it).

3. Once no workflows reference them and any remaining ephemeral volumes have drained, remove the old provisioners:

   $ fuzzball volume provisioner remove persistent
   $ fuzzball volume provisioner remove ephemeral

Do not run fuzzball volume provisioner scan default while the old persistent provisioner still holds volume records. All three provisioners share one share-wide directory namespace, so scanning imports duplicate records under default and subsequent lookups by volume name fail with “exists on multiple provisioners”. Delete the old provisioners first.

The OCI FSS driver manages volumes on an Oracle Cloud File Storage Service file system, with parity to the AWS EFS driver. Volumes mount over NFSv3, and per-volume POSIX ownership is enforced server-side by an OCI FSS IdentitySquash policy. It supports both bring-your-own (BYO) mode and self-provisioned mode.

For BYO mode, provide both filesystemId and mountTargetId. For self-provisioned mode, omit them and provide subnetIds; the driver creates the file system and mount target from the compartment, availability domain, and subnets.

BYO mode — use an existing FSS file system and mount target:

oci-fss:
  description: "OCI File Storage"
  driver:
    type: oci_fss
    region: us-ashburn-1
    compartmentId: ocid1.compartment.oc1..aaaaaaaa
    availabilityDomain: "Uocm:US-ASHBURN-AD-1"
    filesystemId: ocid1.filesystem.oc1.iad.aaaaaaaa
    mountTargetId: ocid1.mounttarget.oc1.iad.aaaaaaaa
  access: all
  ephemeral: all

Self-provisioned mode — Fuzzball creates the FSS infrastructure:

oci-fss-auto:
  description: "Auto-provisioned OCI File Storage"
  driver:
    type: oci_fss
    region: us-ashburn-1
    compartmentId: ocid1.compartment.oc1..aaaaaaaa
    availabilityDomain: "Uocm:US-ASHBURN-AD-1"
    subnetIds:
      - ocid1.subnet.oc1.iad.aaaaaaaa
  access: all
  ephemeral: all

Each volume is an export on the FSS file system. Exports are created and deleted through OCI API calls, and volume creation also briefly mounts the new export from a substrate node to set its POSIX owner.

When a volume is created, the driver:

1. Creates an OCI FSS export on the mount target with the export path /<volume-name>
2. Mounts the export from a substrate node and sets the volume’s owner UID/GID on the export root
3. Enables POSIX identity enforcement via the export’s IdentitySquash settings (maps all file operations to the volume’s owner)

When a volume is published (mounted into a workflow container), the driver mounts the export via NFSv3 at the target path, so each volume sees only its own export directory.

When a volume is deleted, the driver removes the export via the OCI API. Data on the underlying file system is retained — the same behavior as EFS access-point deletion.

OCI FSS FileSystem (ocid1.filesystem...)
├── /volume-a/    ← export path, mounted to container
├── /volume-b/    ← separate export
└── /scratch-123/ ← ephemeral export

Before creating an OCI FSS provisioner, ensure:

For BYO mode:

• The file system exists and is in the ACTIVE state
• A mount target exists in a subnet accessible from Fuzzball substrate nodes

For self-provisioned mode:

• The compartment and availability domain are correct and have capacity
• Subnet IDs are provided for automatic mount-target creation

For both modes:

• Security lists allow NFS traffic (TCP/UDP ports 111 and 2048-2050) from the node subnet
• IAM policies allow managing file systems, exports, and mount targets (for example, via a manage file-family policy)

OCI FSS storage is elastic — there are no per-export capacity quotas. The size field in volume definitions is accepted but not enforced at the FSS level. POSIX ownership is enforced via IdentitySquash on the export, which maps all file operations to the volume’s configured UID/GID.

The OCI Managed Lustre driver backs a provisioner with a single Oracle Cloud File Storage with Lustre file system — a high-performance parallel filesystem for HPC and AI/ML workloads. Each Fuzzball volume is a subdirectory of the shared file system, owned by the volume’s POSIX uid/gid (the same model the GCP Filestore driver uses).

The driver operates in one of two modes, selected by which fields the definition sets:

• Static (bring-your-own) — filesystemId is set. The file system already exists (created outside Fuzzball, for example through the OCI console), and Fuzzball only creates and removes per-volume subdirectories inside it. The provisioner never creates, resizes, or deletes the file system itself; its only use of the OCI API is read-only, to resolve the mount coordinates (MGS address and file-system name). Operationally this is the same model as the hostpath driver pointed at a pre-mounted shared filesystem.
• Dynamic (self-provisioned) — filesystemId is omitted and subnetIds is set. Fuzzball creates the file system itself in the configured compartment, availability domain, and subnet, sized by lustreCapacityGbs / lustrePerformanceTier.

If a definition sets both filesystemId and subnetIds, filesystemId takes precedence and the provisioner operates statically — no file system is created.

In both modes the capacity is fixed once the file system exists: Fuzzball does not grow or shrink a Lustre file system after creation, whether it created the file system or not. To add capacity, expand the file system directly in OCI (console, CLI, or API — subject to OCI’s capacity step sizes). Fuzzball uses the new capacity transparently, since volumes are subdirectories of the file system and lustreCapacityGbs is only applied at creation.

Volumes mount with the native Lustre client (mount -t lustre <mgs>@tcp:/<fsname>/<volume>), so every substrate node that runs workloads using this provisioner must have the Lustre client kernel modules installed.

Substrate node prerequisite: Lustre client. Unlike NFS, the Lustre client is not part of a stock Linux install. Substrate node images must include the client packages matching the node kernel (kmod-lustre-client and lustre-client, e.g. Lustre 2.15.x from Whamcloud) and an LNet configuration that is not tied to a NIC name — cloud instances get different interface names per shape (ens3, enp0s5, …), so bind LNet by subnet instead, e.g. options lnet ip2nets="tcp0 10.0.0.*" in /etc/modprobe.d/lustre.conf.

Volume operations that fail with unknown filesystem type 'lustre' or mount.lustre: ... No such device indicate the client packages or kernel modules are missing on the node; LNetError: ... Error -100 starting up LNI tcp indicates the LNet configuration references an interface that does not exist on that node.

In dynamic (self-provisioned) mode, size and throughput come from lustreCapacityGbs / lustrePerformanceTier, defaulting to OCI’s minimum (31.2 TB at the 125 MBps/TB tier). The capacity is set once at creation — Fuzzball does not expand the file system as it fills (expand it directly in OCI if more space is needed). Self-provisioning is asynchronous: provisioner add returns immediately with the provisioner in Provisioning status, and file-system creation takes ~10–15 minutes before it transitions to Ready (or Error on failure) — check with fuzzball volume provisioner info <name>.

OCI Managed Lustre has a 31.2 TB minimum capacity across all performance tiers (~28.4 TiB usable as reported by df) — there is no smaller option. A self-provisioned file system is therefore a large, billable resource; use BYO mode to share one filesystem across provisioners where possible.

Static (BYO) mode — use an existing Managed Lustre file system; Fuzzball only manages subdirectories within it:

oci-lustre:
  description: "OCI Managed Lustre"
  driver:
    type: oci_lustre
    region: us-ashburn-1
    compartmentId: ocid1.compartment.oc1..aaaaaaaa
    availabilityDomain: "Uocm:US-ASHBURN-AD-1"
    filesystemId: ocid1.lustrefilesystem.oc1.iad.aaaaaaaa
  access: all
  ephemeral: all

Dynamic (self-provisioned) mode — Fuzzball creates the Lustre file system:

oci-lustre-auto:
  description: "Auto-provisioned OCI Managed Lustre"
  driver:
    type: oci_lustre
    region: us-ashburn-1
    compartmentId: ocid1.compartment.oc1..aaaaaaaa
    availabilityDomain: "Uocm:US-ASHBURN-AD-1"
    subnetIds:
      - ocid1.subnet.oc1.iad.aaaaaaaa
  access: all
  ephemeral: all

The Filestore driver manages volumes as subdirectories on a Google Cloud Filestore instance. It is the driver for GCP deployments, where Filestore provides the shared NFS storage that an NFS server provides on-premises.

When target names only the endpoint, the share defaults to workflowio. Write host:/shareName to select a different share.

The Filestore driver does not support node-local mode. A Filestore instance is reachable from every node in the cluster, so local: true is rejected.

gcp-filestore:
  description: "Google Cloud Filestore for shared storage"
  driver:
    type: filestore
    target: "10.224.0.2:/workflowio"
  access: all
  ephemeral: all

Using the default share name, and a DNS endpoint rather than an IP:

gcp-filestore:
  description: "Google Cloud Filestore, default share"
  driver:
    type: filestore
    target: "filestore.internal"
  access: all
  ephemeral: all

Each volume is a subdirectory of the share. Fuzzball creates the directory when the volume is created, applies the requested ownership and mode, and mounts the share over NFS on the node that runs the job. Removing a volume removes its directory.

The Filestore instance itself is created outside Fuzzball — by your deployment tooling or by hand — and the provisioner points at the endpoint it exposes. Fuzzball does not create or delete Filestore instances.

The AWS EFS and Azure Files drivers accept an optional nfsSyncMode field that controls whether NFS volumes are mounted with synchronous or asynchronous writes:

For most clusters the auto default is correct. Override it only when you have a specific durability or performance requirement.

Use this guide to select the right driver for your deployment: