# Getting Started with Trilio on VMware Tanzu Kubernetes Grid (TKG)
## Table of Contents
1. [What is Trilio for Kubernetes?](#what-is-trilio-for-kubernetes)
2. [Prerequisites](#prerequisites)
1. [Verify Prerequisites](#verify-prerequisites-with-the-trilio-preflight-check)
3. [Installation Options](#users-have-two-options-for-installation-methods)
1. [Helm Method 1](#helm-installation-method-1)
2. [Upstream Method 2](#upstream-kubernetes-installation-method-2)
3. [Potential Issues and Workarounds](#potential-issues-and-workarounds)
4. [Access the Trilio Web UI](#access-trilio-ui)
5. [Testing Backup and Restore](#testing-backup-and-restore-operation)
1. [Create a Target](#create-a-backup-target)
2. [Create a Backup](#creating-a-backup-plan-and-backup)
3. [Create a Restore](#creating-a-restore)
6. [Troubleshooting](#troubleshooting)
## What is Trilio for Kubernetes?
Trilio for Kubernetes is a cloud-native backup and restore application. Being a cloud-native application for Kubernetes, all operations are managed with CRDs (Customer Resource Definitions).
Trilio utilizes Control Plane and Data Plane controllers to carry out the backup and restore operations defined by the associated CRDs. When a CRD is created or modified the controller reconciles the definitions to the cluster.
Trilio gives you the power and flexibility to backup your entire cluster or select a specific namespace(s), label, Helm chart, or Operator as the scope for your backup operations.
In this tutorial, we'll show you how to install and test operation of Trilio for Kubernetes on your VMware Tanzu Kubernetes Grid deployment.
## Prerequisites
* [x] **Confirm Compatibility**
Before installing Trilio for Kubernetes, please review the [compatibility matrix](https://docs.trilio.io/kubernetes/4.0.x/about-trilio-for-kubernetes/compatibility-matrix) to ensure Trilio can function smoothly in your Kubernetes environment.
* [x] **Verify that a CSI Driver which provides Snapshot functionality**
Trilio for Kubernetes requires a compatible Container Storage Interface (CSI) driver that provides the **Snapshot** feature.
Check the [Kubernetes CSI Developer Documentation](https://kubernetes-csi.github.io/docs/drivers.html) to select a driver appropriate for your backend storage solution. See the selected CSI driver's documentation for details on the installation of the driver in your cluster.
Trilio will assume that the selected storage driver is a supported CSI driver when the [`volumesnapshotclass`](https://kubernetes.io/docs/concepts/storage/volume-snapshot-classes/) and [`storageclass`](https://kubernetes.io/docs/concepts/storage/storage-classes/)are utilized.
* [x] **Verify that the Required Custom Resource Definitions (CRD) are installed**
Trilio for Kubernetes requires the following Custom Resource Definitions (CRD) to be installed on your cluster:`VolumeSnapshot`, `VolumeSnapshotContent`, and `VolumeSnapshotClass.`
Installing the Required VolumeSnapshot CRDs
Before attempting to install the VolumeSnapshot CRDs, it is important to confirm that the CRDs are not already present on the system.
To do this, run the following command:
```
kubectl api-resources | grep volumesnapshot
```
If CRDs are already present, the output should be similar to the output displayed below. The second column displays the version of the CRD installed (v1 in this case). Ensure that it is the correct version required by the CSI driver being used.
```
volumesnapshotclasses snapshot.storage.k8s.io/v1 false VolumeSnapshotClass
volumesnapshotcontents snapshot.storage.k8s.io/v1 false VolumeSnapshotContent
volumesnapshots snapshot.storage.k8s.io/v1 true VolumeSnapshot
```
**Installing CRDs**
**Be sure to only install v1 version of VolumeSnapshot CRDs**
1. [Read the external-snapshotter GitHub project documentation](https://github.com/kubernetes-csi/external-snapshotter/tree/master). This is compatible with v1.22+.
2. Run the following commands to install directly, check the repo for the latest version:
```
RELEASE_VERSION=6.3
```
```
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/release-${RELEASE_VERSION}/client/config/crd/snapshot.storage.k8s.io_volumesnapshotclasses.yaml
```
```
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/release-${RELEASE_VERSION}/client/config/crd/snapshot.storage.k8s.io_volumesnapshotcontents.yaml
```
```
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/release-${RELEASE_VERSION}/client/config/crd/snapshot.storage.k8s.io_volumesnapshots.yaml
```
* [x] **Network Access Requirements**
For non-air-gapped environments, the following URLs must be accessed from your Kubernetes cluster.
* [Charts.bitnami.com](http://charts.bitnami.com/)
* [Docker.io](http://docker.io/)
* [charts.helm.sh/stable/](http://charts.helm.sh/stable/)
* [gcr.io](http://gcr.io/)
* [kubernetes.io](http://kubernetes.io/)
* [quay.io](http://quay.io/)
* [github.com](https://github.com/)
* [raw.githubusercontent.com](https://raw.githubusercontent.com/)
* Access to the S3 endpoint if the backup target happens to be S3
* Access to application artifacts registry for image backup/restore
- [x] **Network Port Requirements**
If the Kubernetes cluster's control plane and worker nodes are separated by a firewall, then the firewall must allow traffic on the following port(s)
* 9443
## Verify Prerequisites with the Trilio Preflight Check
{% hint style="success" %}
Make sure your cluster is ready to Install Trilio for Kubernetes by installing the [Preflight Check Plugin](https://docs.trilio.io/kubernetes/4.0.x/krew-plugins/tvk-preflight-checks) and running the [Trilio Preflight Check](https://docs.trilio.io/kubernetes/4.0.x/krew-plugins/tvk-preflight-checks).
Trilio provides a preflight check tool that allows customers to validate their environment for Trilio installation.
The tool generates a report detailing all the requirements and whether they are met or not.
**If you encounter any failures, please send the Preflight Check output to your Trilio Professional Services and Solutions Architect so we may assist you in satisfying any missing requirements before proceeding with the installation.**
{% endhint %}
## **Two options for installation method:**
Trilio for Kubernetes is available as a certified Backup and Recovery solution in the VMware Marketplace for VMware Tanzu Kubernetes Grid (TKG) environments, however currently there is no option to install T4K onto the Tanzu clusters directly from the VMware Marketplace.
1. Use an existing helm repository
2. Use the standard helm install command as per the upstream Kubernetes method
## Helm Installation (Method 1)
Users can download the T4K helm chart .tgz file, add it to their existing helm repository and then install the T4K chart from the same repository. To install using this method, perform the following steps:
1. Log in to [VMware Marketplace](https://marketplace.cloud.vmware.com/services/details/trilio-vault-for-kubernetes-1?slug=true).\
2. From the page displayed, in the version dropdown, select the version of T4K that you wish to install. If you are free to choose, it is recommended to use the latest version.
3. Click **Deploy**.
4. In the Deploy wizard displayed, confirm that the version is correct.\
5. Click **Next**.
6. After reading the Trilio End User License Agreement, place a check/tick in the box to indicate your agreement with the terms.\
7. Click **Next**.
8. Click **Download** to download the .tgz file.\
9. Click **Close** to Exit the **Deploy** wizard.
10. Now add the downloaded .tgz helm chart file to your existing helm repository.
11. Then use the helm install command to pull the image from your existing helm repository.
12. Perform steps to prepare environment for Trilio installation
**Configure Antrea Networking**
Set Antrea NetworkPolicyStats to`true`. To accomplish this, change the cluster context to management and run the following command:
```bash
antrea_config_values=$(kubectl get secret/antrea-data-values \
--namespace tkg-system \
-o json | \
jq -r '.data["values.yaml"]' | \
base64 --decode | \
yq e '.antrea.config.featureGates.NetworkPolicyStats = true' - | \
base64 -w 0)
kubectl patch secret/antrea-data-values \
--dry-run=client \
--namespace tkg-system \
--type merge \
-p "{\"data\":{\"values.yaml\": \"$antrea_config_values\"}}" \
-o yaml > /tmp/antrea-config.yaml
kapp deploy --yes -n tkg-system -a antrea -f /tmp/antrea-config.yaml
```
On the workload cluster, restart the Antrea controller and agent with these commands:
```bash
kubectl rollout restart deployment antrea-controller -n kube-system
kubectl rollout restart daemonset antrea-agent -n kube-system
```
**Install VMware CSI Driver**
First, disable automatic updates for the CSI driver by changing the kubeconfig context to the management cluster and running the following command:
```bash
kubectl patch secret/-vsphere-csi-addon \
-p '{"metadata":{"annotations":{"tkg.tanzu.vmware.com/addon-paused": ""}}}' \
--type=merge
```
Next, get a copy of the Vsphere Config secret and save it to a YAML file:
```bash
kubectl get secrets vsphere-config-secret -o yaml > vsphere-config-secret.yaml
```
Then, remove the `vsphere-csi` package on the workload cluster:
```bash
kubectl delete packageinstalls -n tkg-system vsphere-csi
```
If the secret has been removed by the package uninstall, recreate it with:
```bash
kubectl -n kube-system apply -f vsphere-config-secret.yaml
```
Download the CSI driver YAML:
```bash
wget https://raw.githubusercontent.com/kubernetes-sigs/vsphere-csi-driver/v2.5.0-rc.5/manifests/vanilla/vsphere-csi-driver.yaml
```
If preferred, replace the namespace in this file from `vmware-system-csi` to `kube-system` to run the driver in the `kube-system` namespace.
Apply the driver on the workload cluster:
```bash
kubectl apply -f vsphere-csi-driver.yaml
```
Enable the volume snapshot feature on the workload cluster:
```bash
kubectl patch configmap/internal-feature-states.csi.vsphere.vmware.com \
-n kube-system \
--type merge \
-p '{"data":{"block-volume-snapshot":"true"}}'
```
Download the CSI Snapshot controller install script:
```bash
wget https://github.com/kubernetes-sigs/vsphere-csi-driver/blob/v2.5.0-rc.5/manifests/vanilla/deploy-csi-snapshot-components.sh
```
If preferred, replace the namespace in this file from `vmware-system-csi` to `kube-system` to run the driver in the `kube-system` namespace.
Run the script on the workload cluster:
```bash
bash deploy-csi-snapshot-components.sh
```
After you have installed the CSI driver, a [snapshot class](https://docs.trilio.io/kubernetes/4.0.x/appendix/csi-drivers/installing-volumesnapshot-crds) must be created.
**Install Trilio using Helm Charts**
Start by creating a new namespace on the workload cluster for Trilio, and switch to that new namespace. Run the following command to add Trilio's helm repository and install the Trilio Vault Operator:
```bash
helm repo add trilio-vault-operator https://charts.k8strilio.net/trilio-stable/k8s-triliovault-operator
helm install tvm trilio-vault-operator/k8s-triliovault-operator
```
## Upstream Kubernetes Installation (Method 2)
1. Install the VolumeSnapshot CRDs onto the TKG cluster before performing the T4K installation, by following the steps outlined in the prerequisite section for the [Installation of VolumeSnapshot CRDs](https://docs.trilio.io/kubernetes/4.0.x/appendix/csi-drivers/installing-volumesnapshot-crds).\
\
VolumeSnapshot CRDs are not shipped with the TKG cluster by default, so users must install them onto the TKG cluster before performing the T4K installation. Failure to perform this step first may cause one or more pods not to run successfully.
2. The installation of Trilio for Kubernetes on the TKGm (Tanzu Kubernetes Grid for Multi-cloud) or TKGs (Tanzu Kubernetes Grid Service for vSphere) is the same as Upstream Kubernetes, so follow the upstream [#helm-quickstart-installation](https://docs.trilio.io/kubernetes/4.0.x/upstream-kubernetes#helm-quickstart-installation "mention") guide.
3. Users must follow the instructions to install the VMware vSphere CSI Driver from the [official documentation](https://github.com/kubernetes-sigs/vsphere-csi-driver). T4K requires a CSI driver with volume snapshot capability to enable it to perform the backup operation. The VMware vSphere CSI Driver with v2.5 and above supports the volume snapshot capability.
## Potential Issues and Workarounds
#### **Operator Pod Not Starting**
There may be an initial failure during the operator installation due to the Operator pod not starting as the ReplicaSet (RS) couldn't be created. This may be due to an issue with the Pod Security Policy (PSP).
Please define a ClusterRole as shown below to resolve this issue:
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: psp:privileged
rules:
- apiGroups: ['policy']
resources: ['podsecuritypolicies']
verbs: ['use']
resourceNames:
- vmware-system-privileged
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: all:psp:privileged
roleRef:
kind: ClusterRole
name: psp:privileged
apiGroup: rbac.authorization.k8s.io
subjects:
- kind: Group
name: system:serviceaccounts
apiGroup: rbac.authorization.k8s.io
```
#### Antrea Feature Gates
Following the resolution of the PSP issue, it was necessary to enable NetworkPolicyStats from the `antrea-agent` configuration.
Here are the necessary steps:
1. Edit the `configmap antrea-config-c2g88k9fbh` in the `kube-system` namespace using the following command:
```shell
kubectl edit configmap antrea-config-c2g88k9fbh --namespace kube-system
```
2. Within the `antrea-controller.conf`, set `antrea.config.featureGates.NetworkPolicyStats = true`.
3. Also within `antrea-controller.conf`, set `antrea.config.featureGates.AntreaPolicy = true`.
Please refer to the attachments for more information.
Afterwards, run the following commands to restart deployment and daemonset:
```shell
kubectl rollout restart deployment antrea-controller -n kube-system
kubectl rollout restart daemonset antrea-agent -n kube-system
```
#### Trilio Manager Service Type
Depending upon the network architecture of your cluster, you may need to change the TVM service.type to `LoadBalancer` in order to obtain the EXTERNAL-IP for the ingress service, which will then allow you to access the TVK user interface.
## Access Trilio UI
Independent of which install method you employ, when the installation is complete and all pods are up and running, you can access the Trilio web UI by pointing your browser to the configured Node Port.
Trilio for Kubernetes UI Login
## Licensing Trilio for Kubernetes
To generate and apply the Trilio license, perform the following steps:
{% hint style="info" %}
Though a cluster license enables Trilio features across all namespaces in a cluster, the license should only be created in the Trilio install namespace.
{% endhint %}
1\. A license file must be generated for your specific environment.
a) Navigate to your Trilio Welcome email.
b) Click on the **License** link.
c) On the Trilio for Kubernetes License page, click **Generate License.**
d) On the details confirmation page, copy or download the highlighted text to a file.
{% hint style="info" %}
You can use the download button to save the highlighted text as a local file or use the copy button to copy the text and create your file manually.
{% endhint %}
2\. Once the license file has been created, apply it to a Trilio instance using the command line or UI:
{% tabs %}
{% tab title="Command line" %}
1. Execute the following command:
```
kubectl apply -f -n trilio-system
```
2\. If the previous step is successful, check that output generated is similar to the following:
```
NAMESPACE NAME STATUS MESSAGE CURRENT CPU COUNT GRACE PERIOD END TIME EDITION CAPACITY EXPIRATION TIME MAX CPUS
trilio-system license-sample Active Cluster License Activated successfully. 4 FreeTrial 10 2025-07-08T00:00:00Z 8
```
{% hint style="info" %}
Additional license details can be obtained using the following:
kubectl `get license -o json -m trilio-system`
{% endhint %}
{% endtab %}
{% tab title="Management Console (UI)" %}
{% hint style="info" %}
Prerequisites:
1. Authenticate access to the Management Console (UI). Refer to [ui-authentication-intro](https://docs.trilio.io/kubernetes/4.0.x/advanced-configuration/management-console/ui-authentication-intro "mention").
2. Configure access to the Management Console (UI). Refer to [accessing-the-ui](https://docs.trilio.io/kubernetes/4.0.x/advanced-configuration/management-console/accessing-the-ui "mention").
{% endhint %}
If you have already executed the above prerequisites, then refer to the guide for applying a license in the UI: [#actions-license-update](https://docs.trilio.io/kubernetes/4.0.x/using-trilio/getting-started-with-management-console/navigating-intro/cluster-management-home#actions-license-update "mention")
{% endtab %}
{% endtabs %}
### Upgrading a license
A license upgrade is required when moving from one license type to another.
Trilio maintains only one instance of a license for every installation of Trilio for Kubernetes.
To upgrade a license, run `kubectl apply -f -n ` against a new license file to activate it. The previous license will be replaced automatically.
#### [Learn more about Licensing Trilio for Kubernetes](https://docs.trilio.io/kubernetes/4.0.x/getting-started/licensing)
## Create a Backup Target
The Target CR (Customer Resource) is defined from the Trilio Management Console or from your own self-prepared YAML.
The Target object references the NFS or S3 backup storage share you provide as a *target* for your backups. Trilio will create a validation pod in the namespace where Trilio is installed and attempt to validate the NFS or S3 settings you have defined in the Target CR.
Trilio makes it easy to automatically create your backup Target CRD from the Management Console.
Learn how to [Create a Target from the Management Console](https://docs.trilio.io/kubernetes/4.0.x/using-trilio/getting-started-with-management-console#create-a-target)
Take control of Trilio and define your own self-prepared YAML and apply it to the cluster using the kubectl tool.
#### Example S3 Target
```
kubectl apply -f sample-secret.yaml
```
```
kubectl apply -f demo-s3-target.yaml
```
```
apiVersion: v1
kind: Secret
metadata:
name: sample-secret
type: Opaque
stringData:
accessKey: AKIAS5B35DGFSTY7T55D
secretKey: xWBupfGvkgkhaH8ansJU1wRhFoGoWFPmhXD6/vVDcode
```
```
apiVersion: triliovault.trilio.io/v1
kind: Target
metadata:
name: demo-s3-target
spec:
type: ObjectStore
vendor: AWS
objectStoreCredentials:
region: us-east-1
bucketName: trilio-browser-test
credentialSecret:
name: sample-secret
namespace: TARGET_NAMESPACE
thresholdCapacity: 5Gi
```
* See more [Example Target YAML](https://docs.trilio.io/kubernetes/4.0.x/using-trilio/getting-started-1#step-2-create-a-target)
## Testing Backup and Restore Operation
Trilio is a cloud-native application for Kubernetes, therefore all operations are managed with CRDs (Custom Resource Definitions). We will discuss the purpose of each Trilio CRD and provide examples of how to create these objects Automatically in the Trilio Management Console or from the [kubectl](https://kubernetes.io/docs/reference/kubectl/) tool.
### About Backup Plans and Backups
* The Backup Plan CR is defined from the Trilio Management Console or from your own self-prepared YAML.
The Backup Plan CR must reference the following:
1. Your Application Data (label/helm/operator)
2. Backup Target CR
3. Scheduling Policy CR
4. Retention Policy CR
* A Target CR is defined from the Trilio Management Console or from your own self-prepared YAML. Trilio will test the backup target to ensure it is reachable and writable. Look at Trilio validation pod logs to troubleshoot any backup target creation issues.
* Retention and Schedule Policy CRs are defined from the Trilio Management Console or from your own self-prepared YAML.
* Scheduling Policies allow users to automate the backup of Kubernetes applications on a periodic basis. With this feature, users can create a scheduling policy that includes multiple cron strings to specify the frequency of backups.
* Retention Policies make it easy for users to define the number of backups they want to retain and the rate at which old backups should be deleted. With the retention policy CR, users can use a simple YAML specification to define the number of backups to retain in terms of days, weeks, months, years, or the latest backup. This provides a flexible and customizable way to manage your backup retention policy and ensure you meet your compliance requirements.
* The Backup CR is defined from the Trilio Management Console or from your own self-prepared YAML.
The backup object references the actual backup Trilio creates on the Target. The backup is taken as either a Full or Incremental backup as defined by the user in the Backup CR.
### Creating a Backup Plan and Backup
Trilio makes it easy to automatically create your backup plans and all required target and policy CRDs from the Management Console.
Learn more about [Creating Backups from the Management Console](https://docs.trilio.io/kubernetes/4.0.x/using-trilio/getting-started-with-management-console#create-a-backupplan)
Take control of Trilio, define your self-prepared YAML, and apply it to the cluster using the kubectl tool.
#### Example Namespace Scope BackupPlan:
```
# kubectl apply -f ns-backupplan.yaml
```
```
apiVersion: triliovault.trilio.io/v1
kind: BackupPlan
metadata:
name: ns-backupplan
spec:
backupConfig:
target:
namespace: default
name: demo-s3-target
```
```
kubectl apply -f sample-schedule.yaml
```
```
kind: "Policy"
apiVersion: "triliovault.trilio.io/v1"
metadata:
name: "sample-schedule"
spec:
type: "Schedule"
scheduleConfig:
schedule:
- "0 0 * * *"
- "0 */1 * * *"
- "0 0 * * 0"
- "0 0 1 * *"
- "0 0 1 1 *"
```
```
kubectl apply -f sample-retention.yaml
```
```
apiVersion: triliovault.trilio.io/v1
kind: Policy
metadata:
name: sample-retention
spec:
type: Retention
default: false
retentionConfig:
latest: 2
weekly: 1
dayOfWeek: Wednesday
monthly: 1
dateOfMonth: 15
monthOfYear: March
yearly: 1
```
```
kubectl apply -f sample-backup.yaml
```
```
apiVersion: triliovault.trilio.io/v1
kind: Backup
metadata:
name: sample-backup
spec:
type: Full
backupPlan:
name: sample-application
namespace: default
```
See more [Examples of Backup Plan YAML](https://docs.trilio.io/kubernetes/4.0.x/using-trilio/getting-started-1#step-4-run-example)
### About Restores
A Restore CR (Custom Resource) is defined from the Trilio Management Console or from your own self-prepared YAML. The Restore CR references a backup object which has been created previously from a Backup CR.
In a Migration scenario, the location of the backup should be specified within the desired target as there will be no Backup CR defining the location.
Trilio restores the backup into a specified namespace and upon completion of the restore operation, the application is ready to be used on the cluster.
### Creating a Restore
Trilio makes it easy to automatically create your Restore CRDs from the Management Console.
Learn more about [Creating Restores from the Management Console](https://docs.trilio.io/kubernetes/4.0.x/using-trilio/getting-started-with-management-console#restore-the-backup-application)
Take control of Trilio, define your self-prepared YAML, and apply it to the cluster using the kubectl tool.
```
kubectl apply -f sample-restore.yaml
```
```
apiVersion: triliovault.trilio.io/v1
kind: Restore
metadata:
name: sample-restore
spec:
source:
type: Backup
backup:
name: sample-backup
namespace: default
```
See more [Examples of Restore YAML](https://docs.trilio.io/kubernetes/4.0.x/using-trilio/getting-started-1#step-4-run-example)
## Troubleshooting
Problems? Learn about [Troubleshooting Trilio for Kubernetes](https://docs.trilio.io/kubernetes/4.0.x/support/troubleshooting-guide#troubleshooting-guide)
