Install AWS EBS CSI Driver
This page explains steps by step instruction to install the AWS EBS CSI driver on AWS EKS cluster.
A CSI driver with snapshotting capability is a requirement for T4K to function properly. It has been discovered that in the Fall of 2022, the snapshot controller has been removed from the AWS EBS CSI driver install process. In addition, CSI is not installed by default with the creation of an EKS cluster so a customer will have to add this CSI support manually after the creation of an EKS cluster.
The instructions are validated for AWS EKS v 1.21, 1.22, and 1.23.
- 2.Once you have an existing EKS cluster user with
eks-admin
privileges then fetch thekubeconfig
file of the clusteraws eks update-kubeconfig --region us-east-1 --name sa-eks-cluster-1
- 2.Install the AWS EBS CSI Driver through Amazon EKS Add-on. You can follow the below step-by-step instruction with commands:
- 3.Get OIDC endpoint:aws eks describe-cluster \--name sa-eks-cluster-1 \--query "cluster.identity.oidc.issuer" \--output text
- 4.The output will show the OIDC endpoint:sa-eks-cluster-1https://oidc.eks.us-east-1.amazonaws.com/id/A357D6680CACC3FE811997EAE0BEDDCD
- 1.
- 2.In the left pane, select Clusters, and then select the name of your cluster on the Clusters page.
- 3.In the Details section on the Overview tab, note the value of the OpenID Connect provider URL.
- 4.
- 5.In the left navigation pane, choose Identity Providers under Access management. If a Provider is listed that matches the URL for your cluster, then you already have a provider for your cluster. If a provider isn't listed that matches the URL for your cluster, then you must create one.
- 6.To create a provider, choose Add provider.
- 7.For Provider type, select OpenID Connect.
- 8.For Provider URL, enter the OIDC provider URL for your cluster, and then choose Get thumbprint.
- 9.
- 1.Get OIDC provider for cluster:[email protected]:~# aws eks describe-cluster \--name sa-eks-cluster-1 \--query "cluster.identity.oidc.issuer" \--output text
- 2.The output will show the OIDC endpoint with region name:sa-eks-cluster-1https://oidc.eks.us-east-1.amazonaws.com/id/A357D6680CACC3FE811997EAE0BEDDCD
- 3.Create role JSON file to be applied with AWS CLI
- 4.Change the AWS account number, OIDC provider URL and region code as per the EKS cluster deployment.% cat aws-ebs-csi-driver-trust-policy-eks1.json{"Version": "2012-10-17","Statement": [{"Effect": "Allow","Principal": {"Federated": "arn:aws:iam::195495575045:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/A357D6680CACC3FE811997EAE0BEDDCD"},"Action": "sts:AssumeRoleWithWebIdentity","Condition": {"StringEquals": {"oidc.eks.us-east-1.amazonaws.com/id/A357D6680CACC3FE811997EAE0BEDDCD:aud": "sts.amazonaws.com","oidc.eks.us-east-1.amazonaws.com/id/A357D6680CACC3FE811997EAE0BEDDCD:sub": "system:serviceaccount:kube-system:ebs-csi-controller-sa"}}}]}
- 5.Create Role by applying role ISON file with AWS CLI:aws iam create-role \--role-name AmazonEKS_EBS_CSI_DriverRole-eks1 \--assume-role-policy-document file://"aws-ebs-csi-driver-trust-policy-eks1.json"
- 6.Attach above created role to Amazon EBS CSI policy% aws iam attach-role-policy \--policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy \--role-name AmazonEKS_EBS_CSI_DriverRole-eks1
- 7.Adding the Amazon EBS CSI add-on:% aws eks create-addon \--cluster-name sa-eks-cluster-1 \--addon-name aws-ebs-csi-driver \--service-account-role-arn arn:aws:iam::195495575045:role/AmazonEKS_EBS_CSI_DriverRole-eks1
- 8.Add
ebs-sc
storage class, example pod and PVC using a CSI volume:% kubectl get scNAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGEgp2 (default) kubernetes.io/aws-ebs Delete WaitForFirstConsumer false 63d% git clone https://github.com/kubernetes-sigs/aws-ebs-csi-driver.gitCloning into 'aws-ebs-csi-driver'...remote: Enumerating objects: 23572, done.remote: Counting objects: 100% (90/90), done.remote: Compressing objects: 100% (62/62), done.remote: Total 23572 (delta 25), reused 70 (delta 19), pack-reused 23482Receiving objects: 100% (23572/23572), 24.90 MiB | 36.01 MiB/s, done.Resolving deltas: 100% (12207/12207), done.% cd aws-ebs-csi-driver/examples/kubernetes/dynamic-provisioning/% lsREADME.md manifests% kubectl apply -f manifests/persistentvolumeclaim/ebs-claim createdpod/app createdstorageclass.storage.k8s.io/ebs-sc created% kubectl get csidriversNAME ATTACHREQUIRED PODINFOONMOUNT STORAGECAPACITY TOKENREQUESTS REQUIRESREPUBLISH MODES AGEebs.csi.aws.com true false false <unset> false Persistent 5m14sefs.csi.aws.com false false false <unset> false Persistent 17m% kubectl get scNAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGEebs-sc ebs.csi.aws.com Delete WaitForFirstConsumer false 17sgp2 (default) kubernetes.io/aws-ebs Delete WaitForFirstConsumer false 17m% kubectl get pvcNAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGEebs-claim Bound pvc-ed4dbc13-10f6-4176-96e1-3e0d2146f345 4Gi RWO ebs-sc 22s% kubectl get podNAME READY STATUS RESTARTS AGEapp 1/1 Running 0 26s - 9.Change default
StorageClass
toebs-sc
:% kubectl get csidriversNAME ATTACHREQUIRED PODINFOONMOUNT STORAGECAPACITY TOKENREQUESTS REQUIRESREPUBLISH MODES AGEebs.csi.aws.com true false false <unset> false Persistent 5m14sefs.csi.aws.com false false false <unset> false Persistent 17m% kubectl get scNAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGEebs-sc ebs.csi.aws.com Delete WaitForFirstConsumer false 17sgp2 (default) kubernetes.io/aws-ebs Delete WaitForFirstConsumer false 17mkubectl patch storageclass gp2 -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'kubectl patch storageclass ebs-sc -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'% kubectl get scNAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGEebs-sc (default) ebs.csi.aws.com Delete WaitForFirstConsumer false 4m33sgp2 kubernetes.io/aws-ebs Delete WaitForFirstConsumer false 22m - 10.Add VolumeSnapshot CRDs and Snapshot Controller:# Change to the latest supported snapshotter version# https://kubernetes-csi.github.io/docs/external-snapshotter.html$ SNAPSHOTTER_VERSION=v5.0.1# Apply VolumeSnapshot CRDs$ kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/${SNAPSHOTTER_VERSION}/client/config/crd/snapshot.storage.k8s.io_volumesnapshotclasses.yaml$ kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/${SNAPSHOTTER_VERSION}/client/config/crd/snapshot.storage.k8s.io_volumesnapshotcontents.yaml$ kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/${SNAPSHOTTER_VERSION}/client/config/crd/snapshot.storage.k8s.io_volumesnapshots.yaml# Create snapshot controller$ kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/${SNAPSHOTTER_VERSION}/deploy/kubernetes/snapshot-controller/rbac-snapshot-controller.yaml$ kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/${SNAPSHOTTER_VERSION}/deploy/kubernetes/snapshot-controller/setup-snapshot-controller.yaml
- 11.With all above steps, EBS CSI driver with VolumeSnapshot capability is installed.
Install and run the T4K preflight plugin:
% kubectl tvk-preflight run --storage-class ebs-sc
INFO[0000] Created log file with name - preflight-2022-11-4T16-44-27.log
INFO[0000] Setting log level as info
INFO[0001] ====PREFLIGHT RUN OPTIONS====
INFO[0001] LOG-LEVEL="INFO"
INFO[0001] KUBECONFIG-PATH=""
INFO[0001] NAMESPACE="default"
INFO[0001] INCLUSTER="false"
INFO[0001] STORAGE-CLASS="ebs-sc"
...
INFO[0134] ✔ restored pod from volume snapshot of unmounted pv has expected data
INFO[0134] ✔ Preflight check for volume snapshot and restore is successful
INFO[0134] All preflight checks succeeded!
INFO[0138] All preflight resources cleaned
Last modified 20h ago