3.1.X
Search
⌃K

Install AWS EBS CSI Driver

This page explains steps by step instruction to install the AWS EBS CSI driver on AWS EKS cluster.
A CSI driver with snapshotting capability is a requirement for T4K to function properly. It has been discovered that in the Fall of 2022, the snapshot controller has been removed from the AWS EBS CSI driver install process. In addition, CSI is not installed by default with the creation of an EKS cluster so a customer will have to add this CSI support manually after the creation of an EKS cluster.
The instructions are validated for AWS EKS v 1.21, 1.22, and 1.23.

Getting Started with AWS CLI and EKS Cluster deployment:

  1. 1.
    ​Getting started with AWS EKS​
  2. 2.
    Once you have an existing EKS cluster user with eks-admin privileges then fetch the kubeconfig file of the cluster
    aws eks update-kubeconfig --region us-east-1 --name sa-eks-cluster-1

Add CSI driver:

  1. 1.
    ​Install AWS EBS CSI Driver​
  2. 2.
    Install the AWS EBS CSI Driver through Amazon EKS Add-on. You can follow the below step-by-step instruction with commands:
  3. 3.
    Get OIDC endpoint:
    aws eks describe-cluster \
    --name sa-eks-cluster-1 \
    --query "cluster.identity.oidc.issuer" \
    --output text
  4. 4.
    The output will show the OIDC endpoint:
    sa-eks-cluster-1
    https://oidc.eks.us-east-1.amazonaws.com/id/A357D6680CACC3FE811997EAE0BEDDCD

To create an IAM OIDC identity provider for your cluster with the AWS Management Console

  1. 1.
    Open the Amazon EKS console at https://console.aws.amazon.com/eks/home#/clusters.
  2. 2.
    In the left pane, select Clusters, and then select the name of your cluster on the Clusters page.
  3. 3.
    In the Details section on the Overview tab, note the value of the OpenID Connect provider URL.
  4. 4.
    Open the IAM console at https://console.aws.amazon.com/iam/​
  5. 5.
    In the left navigation pane, choose Identity Providers under Access management. If a Provider is listed that matches the URL for your cluster, then you already have a provider for your cluster. If a provider isn't listed that matches the URL for your cluster, then you must create one.
  6. 6.
    To create a provider, choose Add provider.
  7. 7.
    For Provider type, select OpenID Connect.
  8. 8.
    For Provider URL, enter the OIDC provider URL for your cluster, and then choose Get thumbprint.
  9. 9.
    For Audience, enter sts.amazonaws.com and choose Add provider.

Create role with policy:

  1. 1.
    Get OIDC provider for cluster:
    [email protected]:~# aws eks describe-cluster \
    --name sa-eks-cluster-1 \
    --query "cluster.identity.oidc.issuer" \
    --output text
  2. 2.
    The output will show the OIDC endpoint with region name:
    sa-eks-cluster-1
    https://oidc.eks.us-east-1.amazonaws.com/id/A357D6680CACC3FE811997EAE0BEDDCD
  3. 3.
    Create role JSON file to be applied with AWS CLI
  4. 4.
    Change the AWS account number, OIDC provider URL and region code as per the EKS cluster deployment.
    % cat aws-ebs-csi-driver-trust-policy-eks1.json
    {
    "Version": "2012-10-17",
    "Statement": [
    {
    "Effect": "Allow",
    "Principal": {
    "Federated": "arn:aws:iam::195495575045:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/A357D6680CACC3FE811997EAE0BEDDCD"
    },
    "Action": "sts:AssumeRoleWithWebIdentity",
    "Condition": {
    "StringEquals": {
    "oidc.eks.us-east-1.amazonaws.com/id/A357D6680CACC3FE811997EAE0BEDDCD:aud": "sts.amazonaws.com",
    "oidc.eks.us-east-1.amazonaws.com/id/A357D6680CACC3FE811997EAE0BEDDCD:sub": "system:serviceaccount:kube-system:ebs-csi-controller-sa"
    }
    }
    }
    ]
    }
  5. 5.
    Create Role by applying role ISON file with AWS CLI:
    aws iam create-role \
    --role-name AmazonEKS_EBS_CSI_DriverRole-eks1 \
    --assume-role-policy-document file://"aws-ebs-csi-driver-trust-policy-eks1.json"
  6. 6.
    Attach above created role to Amazon EBS CSI policy
    % aws iam attach-role-policy \
    --policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy \
    --role-name AmazonEKS_EBS_CSI_DriverRole-eks1
  7. 7.
    Adding the Amazon EBS CSI add-on:
    % aws eks create-addon \
    --cluster-name sa-eks-cluster-1 \
    --addon-name aws-ebs-csi-driver \
    --service-account-role-arn arn:aws:iam::195495575045:role/AmazonEKS_EBS_CSI_DriverRole-eks1
  8. 8.
    Add ebs-sc storage class, example pod and PVC using a CSI volume:
    % kubectl get sc
    NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
    gp2 (default) kubernetes.io/aws-ebs Delete WaitForFirstConsumer false 63d
    % git clone https://github.com/kubernetes-sigs/aws-ebs-csi-driver.git
    Cloning into 'aws-ebs-csi-driver'...
    remote: Enumerating objects: 23572, done.
    remote: Counting objects: 100% (90/90), done.
    remote: Compressing objects: 100% (62/62), done.
    remote: Total 23572 (delta 25), reused 70 (delta 19), pack-reused 23482
    Receiving objects: 100% (23572/23572), 24.90 MiB | 36.01 MiB/s, done.
    Resolving deltas: 100% (12207/12207), done.
    % cd aws-ebs-csi-driver/examples/kubernetes/dynamic-provisioning/
    % ls
    README.md manifests
    % kubectl apply -f manifests/
    persistentvolumeclaim/ebs-claim created
    pod/app created
    storageclass.storage.k8s.io/ebs-sc created
    % kubectl get csidrivers
    NAME ATTACHREQUIRED PODINFOONMOUNT STORAGECAPACITY TOKENREQUESTS REQUIRESREPUBLISH MODES AGE
    ebs.csi.aws.com true false false <unset> false Persistent 5m14s
    efs.csi.aws.com false false false <unset> false Persistent 17m
    % kubectl get sc
    NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
    ebs-sc ebs.csi.aws.com Delete WaitForFirstConsumer false 17s
    gp2 (default) kubernetes.io/aws-ebs Delete WaitForFirstConsumer false 17m
    % kubectl get pvc
    NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
    ebs-claim Bound pvc-ed4dbc13-10f6-4176-96e1-3e0d2146f345 4Gi RWO ebs-sc 22s
    % kubectl get pod
    NAME READY STATUS RESTARTS AGE
    app 1/1 Running 0 26s
  9. 9.
    Change default StorageClass to ebs-sc:
    % kubectl get csidrivers
    NAME ATTACHREQUIRED PODINFOONMOUNT STORAGECAPACITY TOKENREQUESTS REQUIRESREPUBLISH MODES AGE
    ebs.csi.aws.com true false false <unset> false Persistent 5m14s
    efs.csi.aws.com false false false <unset> false Persistent 17m
    % kubectl get sc
    NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
    ebs-sc ebs.csi.aws.com Delete WaitForFirstConsumer false 17s
    gp2 (default) kubernetes.io/aws-ebs Delete WaitForFirstConsumer false 17m
    ​
    kubectl patch storageclass gp2 -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'
    kubectl patch storageclass ebs-sc -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
    ​
    % kubectl get sc
    NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
    ebs-sc (default) ebs.csi.aws.com Delete WaitForFirstConsumer false 4m33s
    gp2 kubernetes.io/aws-ebs Delete WaitForFirstConsumer false 22m
  10. 10.
    Add VolumeSnapshot CRDs and Snapshot Controller:
    # Change to the latest supported snapshotter version
    # https://kubernetes-csi.github.io/docs/external-snapshotter.html
    $ SNAPSHOTTER_VERSION=v5.0.1
    ​
    # Apply VolumeSnapshot CRDs
    $ kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/${SNAPSHOTTER_VERSION}/client/config/crd/snapshot.storage.k8s.io_volumesnapshotclasses.yaml
    $ kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/${SNAPSHOTTER_VERSION}/client/config/crd/snapshot.storage.k8s.io_volumesnapshotcontents.yaml
    $ kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/${SNAPSHOTTER_VERSION}/client/config/crd/snapshot.storage.k8s.io_volumesnapshots.yaml
    ​
    # Create snapshot controller
    $ kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/${SNAPSHOTTER_VERSION}/deploy/kubernetes/snapshot-controller/rbac-snapshot-controller.yaml
    $ kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/${SNAPSHOTTER_VERSION}/deploy/kubernetes/snapshot-controller/setup-snapshot-controller.yaml
  11. 11.
    With all above steps, EBS CSI driver with VolumeSnapshot capability is installed.

Validate the EBS driver installation

If T4K Preflight plugin is not already installed, follow the instructions here​
Install and run the T4K preflight plugin:
% kubectl tvk-preflight run --storage-class ebs-sc
INFO[0000] Created log file with name - preflight-2022-11-4T16-44-27.log
INFO[0000] Setting log level as info
INFO[0001] ====PREFLIGHT RUN OPTIONS====
INFO[0001] LOG-LEVEL="INFO"
INFO[0001] KUBECONFIG-PATH=""
INFO[0001] NAMESPACE="default"
INFO[0001] INCLUSTER="false"
INFO[0001] STORAGE-CLASS="ebs-sc"
...
INFO[0134] ✔ restored pod from volume snapshot of unmounted pv has expected data
INFO[0134] ✔ Preflight check for volume snapshot and restore is successful
INFO[0134] All preflight checks succeeded!
INFO[0138] All preflight resources cleaned
Previous
Installing VolumeSnapshot CRDs
Next - Appendix
T4K Product FAQ
Last modified 20h ago