Volume Operations
Suppose you have raw data in an AWS S3 bucket. You create a workflow with several jobs that process and analyze the data in multiple steps. You need to perform the following:
- create (or locate) a volume for use within the job
- copy the data from your S3 bucket to the volume
- mount the volume within the first job that
- performs preprocessing, and then mount the same volume in the second job to perform analysis on the preprocessed data
- copy the analysis results back to your S3 bucket at workflow completion
These operations are referred to as volume creation, ingress, mounting, and egress respectively. We discuss these operations in a little more detail in the next sections.