Last modified December 13, 2022
Using persistent volumes on AWS with EFS
If your cluster is running in the cloud on Amazon Web Services (AWS) the most common way to store data is using EBS volumes with the dynamic provisioner. Sometimes EBS is not the optimal solution.
The advantages of using EFS over EBS are:
- EFS data can be accessed from all availability zones in the same region while EBS is tied to a single availability zone.
- EFS has the capability to mount the same persistent volume to multiple pods at the same time using the ReadWriteMany access mode.
- EFS will not hit the AWS Instance Volume Limit as it is a software mount and will avoid the Impaired EBS issue.
- EFS mount times are better than EBS.
- EFS provides encryption in transit support using TLS and it’s enabled by default.
If you need to use EFS to provision volumes, be advised:
- All Kubernetes persistent volumes will be stored in the same EFS instance. You can deploy multiple provisioners per cluster, each having its own storage-class and EFS instance.
- EFS throughput need to be set up accordingly in order to not have performance issues. We only recommend Provisioned Throughput, and if you need high performance you will need EBS.
- EFS backups are done with AWS Backup and it does not have the snapshot feature of EBS.
- You cannot limit the amount of data stored in an EFS volume. The requested value in Kubernetes is ignored.
- EFS mount targets are limited to 1 subnet per availability zone. Each NodePool will create a different subnet per AZ, plan accordingly.
Provision an EFS instance on AWS
Note: Currently only static provisioning is supported. This means an AWS EFS file system needs to be created manually on AWS first. After that it can be mounted inside a container as a volume using the driver.
Before installing the provisioner in Kubernetes we will need to create the EFS instance in the same AWS account:
- Open the AWS management console for the AWS account holding your cluster resources.
- From the
- Create a new file system and select the VPC where your cluster is located. The VPC can be identified by your cluster ID.
- Select the availability zone with the subnets your instances are located on and the security groups of your node pools. Subnets and security groups can be identified by you cluster ID.
- Choose the throughput and performance mode. Setting a file system policy or access points is optional.
- Create the instance and note the EFS instance ID.
- In case of dynamic provisioning, you need to add the roles to the instance profile or use service account annotation in controller to enable access to AWS EFS.
Installing the EFS CSI driver
To install the EFS CSI driver in the workload cluster, you will need to follow these steps:
- Access the web interface and select the cluster on which you want to install the EFS CSI driver.
- Open the Apps tab.
- Click the Install App button
- Select the Giant Swarm Playground catalog.
- Select the App named
- Click the Configure & Install button. Make sure that the correct cluster is selected.
- Create and submit a configuration values file in case you want to deploy the storage class and controller (by default disable to allow a smooth transitation to CSI driver)
storageClasses: - name: efs-sc annotations: storageclass.kubernetes.io/is-default-class: "true" mountOptions: - tls parameters: provisioningMode: efs-ap fileSystemId: fs-92107666 directoryPerms: "700" gidRangeStart: "1000" gidRangeEnd: "2000" controller: create: true
- Click the Install App button.
Deploy a sample application
In order to verify that the EFS CSI driver works as expected, we suggest to deploy a test workload based on the following manifests.
First we apply a persistent volume claim with a capacity of 5 GB storage.
apiVersion: v1 kind: PersistentVolumeClaim metadata: name: efs-claim spec: accessModes: - ReadWriteMany storageClassName: efs-sc resources: requests: storage: 5Gi
And finally, we create a pod which repeatedly writes the current date into a file located in our EFS storage mount:
apiVersion: v1 kind: Pod metadata: name: efs-app spec: containers: - name: linux image: amazonlinux:2 command: ["/bin/sh"] args: ["-c", "while true; do echo $(date -u) >> /efs-data/out.txt; sleep 5; done"] volumeMounts: - name: efs-storage mountPath: /efs-data volumes: - name: efs-storage persistentVolumeClaim: claimName: efs-claim
Warning: By default, new EFS file systems are owned by root:root. You might need to change file system permissions if your container is not running as root. To learn about exposing separate data stores with independent ownership and permissions, check the AWS guide on working with Amazon EFS Access Points.
- AWS Elastic File System
- AWS Elastic File System Access Points Example
- Persistent volumes
- Persistent volume claims
- Claim persistent volumes in pods
- Kubernetes CSI developer documentation
- Container Storage Interface (CSI) for Kubernetes GA
Need help, got feedback?
We listen to your Slack support channel. You can also reach us at email@example.com. And of course, we welcome your pull requests!