Tenant Isolation

In a deployment that serves more than one organization, storage has to keep those organizations apart. This page describes how Fuzzball does that, and what an operator is responsible for.

A single-organization installation does not need any of this. The behaviour described here activates only when a deployment is configured as multi-tenant, and the defaults are unchanged otherwise.

Volume names are unique within a provisioner, and provisioners belong to an organization. Two organizations can therefore each own a volume called data, and nothing about the name distinguishes them. Isolation has to come from where the volume is placed and how it is mounted, not from what it is called.

Fuzzball scopes an organization’s storage by whichever means its driver supports.

EFS can create filesystems on demand, so each organization gets one of its own. This is the strongest form: the organizations do not share a filesystem at all. It costs nothing extra because EFS is elastic and bills only for what is stored.

Hostpath and Filestore address volumes by path on a store the operator provides, so each organization is confined to a subdirectory of it — the provisioner’s subPath. Volumes cannot be created or listed outside it.

A driver that can do neither is refused rather than shared silently, so a misconfiguration surfaces as an error at organization creation instead of as several organizations quietly writing to the same directory.

subPath is immutable. Every existing volume lives beneath the current value, so changing it would not move them — it would point the provisioner at an empty directory and orphan the lot.

Placing volumes apart on the backing store is only half of it. They also have to stay apart when mounted onto a compute node, which may be running jobs for several organizations at once.

Each published volume is mounted under a path scoped to the allocation that requested it, so two jobs never resolve to the same mount point. That holds across organizations, and also between two jobs of the same workflow that mount the same volume — under a shared path, the first job to finish would unmount it out from under the second.

Before serving or removing a mount, Fuzzball verifies that what is mounted there is the volume it expects. A mismatch is refused rather than served.

Restricting privileged drivers. The hostpath and nfs drivers name a location on the compute node or an arbitrary remote server, so in a multi-tenant deployment they are restricted to cluster administrators, and may be confined further to an allowlist of paths and servers. See Driver Types.

Bring-your-own storage is unavailable. Attaching an existing cloud filesystem by ID is a single-tenant feature: ownership of a filesystem the platform can merely reach cannot be verified. Multi-tenant deployments create storage instead of attaching it.

Compute nodes. Storage isolation assumes the compute node itself is trustworthy. Anyone able to run arbitrary code as root on a node — through node provisioning configuration, for example — is outside the boundary this page describes.

Network reachability. On AWS, the security group for a filesystem’s mount targets admits NFS traffic from the VPC CIDR. Separation between organizations rests on each one only being told its own filesystem ID, not on network segmentation — so treat the VPC as inside the boundary.

Per-organization filesystems are close to free on the bill: EFS has no per-filesystem charge, and you pay for stored data either way. The limit you will reach first is a quota, not a cost.

Each filesystem places one mount target per availability zone, and each mount target consumes an address in its subnet. A /24 per zone across three zones therefore supports roughly 250 organizations — well before the default limit of 1,000 EFS filesystems per region.

Both limits are worth raising before onboarding at scale, and subnets are worth sizing generously up front, because neither can be widened once addresses are exhausted without renumbering. A provisioner that fails for either reason reports it: see the DETAIL column in Managing Provisioners.

Splitting storage per organization also removes a ceiling that used to be global. A single shared filesystem allows 1,000 access points, which capped the total number of volumes across every organization on the cluster; that budget is now per organization.

Fuzzball creates these filesystems encrypted at rest, in Elastic throughput mode. Elastic is deliberate: the alternative scales a filesystem’s baseline throughput with how much it stores, which works for one large shared filesystem but leaves a small per-organization one starved.

Removing a provisioner whose filesystem Fuzzball created also deletes that filesystem, its mount targets, and its security group. This happens in the background and does not delay the removal.

A filesystem the operator supplied is never deleted, and neither is one created before this behaviour existed. Fuzzball deletes storage only when it has two independent confirmations that it created it: a record on the provisioner, and matching provenance on the cloud resource itself.

Deleting a provisioner deletes its data. Volumes on it are removed with it, and where Fuzzball owns the backing filesystem, that filesystem goes too.