Last modified February 6, 2020
Using Persistent Volumes on AWS with EFS
If your cluster is running in the cloud on Amazon Web Services (AWS) the most common way to store data is using EBS volumes with the dynamic provisioner. Sometimes EBS is not the optimal solution.
The advantages of using EFS over EBS are:
- EFS data can be accessed from all Availability Zones in the same region while EBS is tied to a single Availability Zone.
- EFS has the capability to mount the same Persistent Volume to multiple pods at the same time using the ReadWriteMany access mode.
- EFS will not hit the AWS Instance Volume Limit as it is a software mount and will avoid the Impaired EBS issue.
- EFS mount times are better than EBS.
If you need to use EFS to provision volumes, be advised:
- All Kubernetes Persistent Volumes will be stored in the same EFS instance. You can deploy multiple provisioners per cluster, each having its own storage-class and EFS instance.
- EFS throughput need to be set up accordingly in order to not have performance issues. We only recommend Provisioned Throughput, and if you need high performance you will need EBS.
- EFS backups are done with AWS Backup and it does not have the snapshot feature of EBS.
- You cannot limit the amount of data stored in an EFS volume. The requested value in Kubernetes is ignored.
- EFS mount targets are limited to 1 subnet per Availability Zone. Each NodePool will create a different subnet per AZ, plan accordingly.
Provision an EFS instance on AWS
Before installing the provisioner in Kubernetes we will need to create the EFS instance in the same AWS account:
- Open the AWS management console in the account your cluster is located.
- Select EFS from the services list.
- Create a new EFS mount and select the VPC where your cluster is located.
- Select the Availability Zone with the subnets your instances are located on and the security-groups of the workers.
- Choose the throughput and performance mode, no file system policy or access points are needed.
- Create the instance and note the EFS instance id.
EFS provisioner configuration file
In order to configure the EFS provisioner, you will need to create a file on your local machine named
efs-provisioner.yaml with the following content:
global: deployEnv: production efsProvisioner: efsFileSystemId: fs-90f935c8 awsRegion: eu-central-1 path: / provisionerName: giantswarm.io/aws-efs storageClass: name: efs isDefault: false gidAllocate: enabled: true gidMin: 40000 gidMax: 50000 reclaimPolicy: Delete mountOptions:  rbac: create: true podSecurityPolicy: enabled: true
You will need to populate the
awsRegion parameters to match the configured values of the EFS instance.
For additional configuration parameters see the upstream documentation.
Installing EFS Provisioner
To install the provisioner you will need to follow these steps:
- Access Giant Swarm web UI and select the cluster on which you want to install the provisioner.
- Open the Helm Stable catalog.
efs-provisionerin the search bar.
- Select the efs-provisioner application and then click the Configure & Install button.
- Upload the
efs-provisioner.yamlfile created in the previous step.
- Finally, you can click the Install App button and it will be installed into your cluster.
Using EFS Volumes
Your Kubernetes cluster will have a new Storage Class
efs deployed, which you will have to reference when creating persistent volumes.
In order to check that the storage class exits, you should see something similar to:
$ kubectl get storageclass NAME PROVISIONER AGE efs giantswarm.io/aws-efs 23m gp2 (default) kubernetes.io/aws-ebs 5h31m
In the following example you can see the annotation
volume.beta.kubernetes.io/storage-class matches the
efs Storage Class:
kind: PersistentVolumeClaim apiVersion: v1 metadata: name: test annotations: volume.beta.kubernetes.io/storage-class: "efs" spec: accessModes: - ReadWriteMany resources: requests: storage: 100Mi
Testing the new storage class
You can create a file with the example above in a file called
pvc_claim.yaml and instantiate with the following command:
kubectl apply -f pvc_claim.yaml
In order to check the status of the volume we can check the status of the PVC:
$ kubectl get pvc test NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE test Bound pvc-0d7b988e-4eed-4cf7-918b-30964774fa13 100Mi RWX efs 5s
If the status is
Bound, everything worked as expected.
If there is an error, check the logs of the provisioner pod in the namespace where the application was deployed.