TrilioVaultManager
TrilioVaultManager CRD, owned by Trilio Operator for Upstream/Non-Operator Lifecycle Manager (OLM) based environments, manages the lifecycle of Trilio for Kubernetes Application.
The following snippet provides an example on how to use the TrilioVaultManager CRD.
The version number provided below is only for illustrative purposes. Please refer to the Compatibility Matrix to find the latest version to use.
Copy apiVersion : triliovault.trilio.io/v1
kind : TrilioVaultManager
metadata :
labels :
triliovault : triliovault
name : trilio-app
spec :
trilioVaultAppVersion : 0.2.5
helmVersion :
version : v3
tillerNamespace : "kube-system"
applicationScope : Namespaced
restoreNamespaces : [ "kube-system" , "default" , "restoreinme" , "restore-namespace" , "restore" ]
resources :
requests :
memory : 400Mi
In the above CRD restoreNamespaces is optional.
Application scope of Namespaced, will enable additonal namespaces that can be restored into. Application scope of clustered will restrict restores to those specific namespaces only.
Provide Resource Limits for Trilio Pods
Copy apiVersion : triliovault.trilio.io/v1
kind : TrilioVaultManager
metadata :
labels :
triliovault : k8s
name : tvk
spec :
trilioVaultAppVersion : 2.5.0
applicationScope : Cluster
# T4K components configuration, currently supports control-plane, web, exporter, web-backend, ingress-controller, admission-webhook.
# User can configure resources for all componentes and can configure service type and host for the ingress-controller
componentConfiguration :
web-backend :
resources :
requests :
memory : "400Mi"
cpu : "200m"
limits :
memory : "2584Mi"
cpu : "1000m"
ingress-controller :
service :
type : LoadBalancer
host : "trilio.co.in" Backup Target
The Backup Target CRD specifies the backup storage media. Trilio supports either AWS S3 compatible object storage or NFS. A user can configure multiple backup targets and choose the target when an Application CR is created by providing target name and the name of the namespace where it resides. The target credentials can be saved as a secret and refer to the target CR for better security reasons. All backups that are created for that Application will be saved on the backup target specified in Application CR spec. Once a backup target is chosen for an Application, it cannot be changed.
Example 1 - S3 Target: AWS
Sample YAML file for AWS based S3 bucket and credentialSecret
Copy apiVersion : v1
kind : Secret
metadata :
name : sample-secret
type : Opaque
stringData :
accessKey : AKIAS5B35DGFSTY7T55D
secretKey : xWBupfGvkgkhaH8ansJU1wRhFoGoWFPmhXD6/vVD
Copy apiVersion : triliovault.trilio.io/v1
kind : Target
metadata :
name : demo-s3-target
spec :
type : ObjectStore
vendor : AWS
objectStoreCredentials :
region : us-east-1
bucketName : trilio-browser-test
credentialSecret :
name : sample-secret
namespace : TARGET_NAMESPACE
thresholdCapacity : 5Gi Example 2 - S3 Target: Non-AWS
Sample YAML file for non-AWS S3 compatible bucket. The only difference between this spec and AWS S3 spec is the explicit specification on URL.
Copy apiVersion : triliovault.trilio.io/v1
kind : Target
metadata :
name : sample-target
spec :
type : ObjectStore
vendor : Ceph
objectStoreCredentials :
url : "http://bucket.s3.dualstack.region.acme.com/key"
accessKey : "XXXXXXXXXXXXXXXXXXXX"
secretKey : "YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY"
bucketName : "datahub-dev"
region : "us-west-1" Example 3 - S3 Target with SSL Certificate
The following example details how a target can be created when the S3 target has SSL.
Copy apiVersion : v1
kind : Secret
metadata :
name : sample-secret
type : Opaque
data :
accessKey : VFJJTElPTUlOSU8=
ca-bundle.pem: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURpekNDQW5PZ0F3SUJBZ0lKQUk4aGM0ZEMrVE0vTUEwR0NTcUdTSWIzRFFFQkN3VUFNRnd4Q3pBSkJnTlYKQkFZVEFrRlZNUk13RVFZRFZRUUlEQXBUYjIxbExWTjBZWFJsTVNFd0h3WURWUVFLREJoSmJuUmxjbTVsZENCWAphV1JuYVhSeklGQjBlU0JNZEdReEZUQVRCZ05WQkFNTURETTBMalkyTGpFM0xqRTJNREFlRncweU1UQTJNamt3Ck9EUXdORE5hRncweU1qQTJNamt3T0RRd05ETmFNRnd4Q3pBSkJnTlZCQVlUQWtGVk1STXdFUVlEVlFRSURBcFQKYjIxbExWTjBZWFJsTVNFd0h3WURWUVFLREJoSmJuUmxjbTVsZENCWGFXUm5hWFJ6SUZCMGVTQk1kR1F4RlRBVApCZ05WQkFNTURETTBMalkyTGpFM0xqRTJNRENDQVNJd0RRWUpLb1pJaHZjTkFRRUJCUUFEZ2dFUEFEQ0NBUW9DCmdnRUJBS1lpblBOS3NqUzRYM09xcTJKc3NycndLREREUDI4dzZ3MmRRQnRROW1xMHIrOXpRRVBEUk5qWEo3UEcKaDNDT3V6UEtPeUw0WnFSZHdHbmZnTXRiMGdVTWtHVVZwTUxHV1FaRXJvS1pDSGcwOGhHRzRpTWxubWthK0NrUQp2OGNYYnNCOEpZcEd6SXJLdytGaG41NzdYeFRtZzRlK2cxLzZyeE1OMnlhUE9XMUNPdGpZTmpDNkNVeGFHWUt3CnVOQzg2NDJHbjhkR2IxbFZuVUF0SDVXajgvQUlvd1Z0cHB5ZXI0K0liWWJwRVlvN0l2WVd0NGd5OXZhWjJaaHkKS2h1akhpWmFIWVhpYmdMclZLU2VxTWFoOUNRWEVtRHltVHBkRCtieVc1ZnlLdnVoRXlmaWROelNMQjhDclhicwoxODR3MmFPVTUveVp2dndrV1dRK016bnZwL01DQXdFQUFhTlFNRTR3SFFZRFZSME9CQllFRkc0NmJPMklrQW9jCjc2d0J0L3RTTGRwb2NQWXJNQjhHQTFVZEl3UVlNQmFBRkc0NmJPMklrQW9jNzZ3QnQvdFNMZHBvY1BZck1Bd0cKQTFVZEV3UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFBeWlxVis4ZkJmanE4cGFuSmJTNlFnTwpKcDlRZTZpYWloQ2xwaHZCWVJnSVdnUkNrVkJjRmpCcGhEanlxajI2ZFZBSnhJMWFFYUswTXVqSVpaRHp5MXVLCmJRTXA4c0ZJcnUzbU5oZSt5bTBaVFl1eXRVRzVOMHJnQ0hmNWFlbk91T1krem5IdE1PY1BOQlFENGVnZEhlSXkKdVI4bG9FNGRWWkxkSGRLb3RGSjFHQlA2Qy9GWkxiK3NjSzhEM0UxeWhCZWZOUC9pMWh3RVB6Rk5WU2U3VzFOego4cSs3UWNudDRMajJsR2hIdE9KTmlabEdIYW5BbENnSVNoaXB4OU5RK1hUcjc4VHozOFBPL3hOVFN4V0hYTXV0CmEzS2p1UGRtSHNpd0FWenNoOFVSdUtyOHpFQ043Mk5NM25sWkVzdXNPMFo4NHg0dVdxVUhNNFR0YzFLM3ppZz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
secretKey : SEFBTUFBTFVNSEFJQ0hBTE5B
Copy apiVersion : triliovault.trilio.io/v1
kind : Target
metadata :
name : demo-s3-target
spec :
type : ObjectStore
vendor : AWS
objectStoreCredentials :
region : us-east-1
bucketName : trilio-browser-test
credentialSecret :
name : sample-secret
namespace : TARGET_NAMESPACE
thresholdCapacity : 5Gi Example 4 - OVH Object Storage as Target using Swift S3 API
Sample YAML for OVH Object Storage. This is S3 compatible storage can be accessed using Swift S3 APIs and can be configured as a Target.
Copy apiVersion : triliovault.trilio.io/v1
kind : Target
metadata :
name : demo-ovh-s3-target
spec :
type : ObjectStore
vendor : other
objectStoreCredentials :
url : "https://s3.bhs.cloud.ovh.net"
bucketName : demo-ovh-s3-bucket
region : bhs
credentialSecret :
name : sample-ovh-s3-secret
namespace : TARGET_NAMESPACE
thresholdCapacity : 5Gi To configure the OVH S3 Object Storage to use as a Target and to create access key, secret key follow the Configure OVH Object Storage as a Target section.
Example 5 - NFS Target Example
Sample YAML for NFS share.
Copy apiVersion : triliovault.trilio.io/v1
kind : Target
metadata :
name : sample-target
spec :
type : NFS
vendor : Other
nfsCredentials :
nfsExport : 192.168.1.1:/src/nfs/kubedata
nfsOptions : nfsvers=4
thresholdCapacity : 1000Gi
Additional values for the nfsOptions field can be found here
Example 6 - S3 Based Event Target Example
Any backup target can be designated as an event target. An event target is used to share state information between clusters with the shared event target. Two or more clusters can share the same event target.
Sample YAML file for S3-based event target is given below
Copy apiVersion : triliovault.trilio.io/v1
kind : Target
metadata :
annotations :
trilio.io/event-target : "true"
name : s3-event-target
namespace : default
spec :
objectStoreCredentials :
bucketName : aj-test-s3
credentialSecret :
name : s3-cred-secret
namespace : default
region : us-east-1
type : ObjectStore
vendor : AWS
Example 7 - NFS Based Event Target Example
Sample YAML file for NFS-based event target is given below
Copy apiVersion : triliovault.trilio.io/v1
kind : Target
metadata :
annotations :
trilio.io/event-target : "true"
name : nfs-event-target
namespace : default
spec :
nfsCredentials :
nfsExport : 34.66.17.160:/src/nfs/aj
nfsOptions : nfsvers=4
thresholdCapacity : 100Gi
type : NFS
vendor : Other Policy
Trilio provides a Policy Custom Resource Definition through which scheduling, retention, and cleanup policies can be created.
Scheduling Policy
A scheduling policy can be created to automate the capture of applications within a Kubernetes system on a periodic basis. T4K enables users to create a scheduling policy with multiple cron strings defined within it. Each Cron string creates an associated cron job within the Kubernetes system and as a result daily, weekly, yearly, etc. policies can be created independently of each other within the same policy CR.
Below is an example of a scheduling policy CR:
Copy kind: "Policy"
apiVersion: "triliovault.trilio.io/v1"
metadata:
name: "test-all-sch"
spec:
type: "Schedule"
scheduleConfig:
schedule:
- "0 0 * * *"
- "0 */1 * * *"
- "0 0 * * 0"
- "0 0 1 * *"
- "0 0 1 1 *" Retention Policy
The retention policy enables users to define the number of backups to retain and the cadence to delete backups as per compliance requirements. The retention policy CR provides a simple YAML specification to define the number of backups to retain in terms of days, weeks, months, years, latest etc.
Retention Polices are referenced within a BackupPlan and can be added to any of the example YAML files provided below
Example - Retention Policy
Copy apiVersion : triliovault.trilio.io/v1
kind : Policy
metadata :
name : sample-policy
spec :
type : Retention
default : false
retentionConfig :
latest : 2
weekly : 1
dayOfWeek : Wednesday
monthly : 1
dateOfMonth : 15
monthOfYear : March
yearly : 1 Cleanup Policy
By default, Trilio does not delete failed backup job backup images or backup CRs. Users must manually delete failed backups. However, Trilio provides a cleanup policy to clean up failed backups. This policy only cleans up the backup files on the target associated with the failed backup job, not the backup CR itself. Users must use the kubectl command to delete any failed backup CRs from the cluster. If the spec.default is true and the policy is created in the T4K installed namespace, the policy applies to the entire cluster. If the spec.default is false, then the policy applies to the namespace where the policy was created. UI automatically sets the spec.default to false. A cron job that runs every 30 mins and deletes failed backups data based on the value specified by spec.cleanupConfig.backupdays. The policy does not delete any volume snapshots left behind by the failed backup jobs. Since backup CRs have an ownerRef on the volume snapshots, they will be automatically deleted when the failed backup CRs are deleted
An example cleanup policy that cleans up failed backups after five days is provided below.
Copy apiVersion: triliovault.trilio.io/v1
kind: Policy
metadata:
name: sample-cleanup-policy
spec:
type: Cleanup
default: true
cleanupConfig:
backupDays: 5 Continuous Restore Policy
Continuous Restore policy defines the number of consistent sets on a remote site.
Copy apiVersion : triliovault.trilio.io/v1
kind : Policy
metadata :
name : continuous-restore-policy
namespace : default
spec :
continuousRestoreConfig :
consistentSets : 3
default : false
type : ContinuousRestore
Immutability
Trilio provides the ability to create immutable backups at the application level. Once the backup is taken and stored on an immutable target, it can not be altered (overwritten/deleted) until the retention period set through T4K is up.
Immutable Target
To create immutable backups, user needs to create an immutable target as shown in the sample below. The immutable targets work only on object storage targets.
Copy kind : Target
apiVersion : triliovault.trilio.io/v1
metadata :
name : sample-immutable-target
spec :
type : ObjectStore
vendor : AWS
objectStoreCredentials :
# immutable targets works only on object store targets
region : us-east-1
bucketName : trilio-browser-test
credentialSecret :
name : sample-secret
namespace : TARGET_NAMESPACE
objectLockingEnabled : true
thresholdCapacity : 5Gi Retention Policy for immutable backup
After immutable target, user needs to create a retention policy. This sets up the retention period for the backup. Refer the sample Example - Retention Policy .
Immutable Backup
After immutable target and retention policy are set, user needs to create a backup. Once the backup is taken and stored on an immutable target, it can not be altered (overwritten/deleted) until the retention period set through T4K is up. Refer the BackupPlan and Backup samples below.
Example - BackupPlan with Retention Period
Sample for BackupPlan with Retention period
Copy apiVersion : triliovault.trilio.io/v1
kind : BackupPlan
metadata :
name : sample-immutable-backupplan
spec :
backupConfig :
target :
name : sample-immutable-target
namespace : TARGET_NAMESPACE
retentionPolicy :
name : sample-retention-policy
namespace : POLICY_NAMESPACE
schedulePolicy :
fullBackupPolicy :
name : sample-schedule-policy
namespace : POLICY_NAMESPACE
backupPlanComponents :
customSelector :
selectResources :
labelSelector :
- matchLabels :
app : gcp-compute-persistent-disk-csi-driver Example - Immutable Backup
Copy apiVersion : triliovault.trilio.io/v1
kind : Backup
metadata :
name : sample-immutable-backup
spec :
type : Full
backupPlan :
name : sample-immutable-backupplan
namespace : BACKUP_NAMESPACE Hooks
Hooks enable injecting commands into pods/containers before and after a backup via pre/post commands. Hooks enable taking application consistent backups and extending backup workflows.
Note: Hook should be created in the same namespace as that of BackupPlan referencing it resides.
Hook for MySQL
Copy apiVersion : triliovault.trilio.io/v1
kind : Hook
metadata :
name : mysql-hook
spec :
pre :
execAction :
command :
- "bash"
- "-c"
- "mysql --user=root --password=$MYSQL_ROOT_PASSWORD -Bse 'FLUSH TABLES WITH READ LOCK;system ${WAIT_CMD};'"
ignoreFailure : false
maxRetryCount : 1
timeoutSeconds : 10
post :
execAction :
command :
- "bash"
- "-c"
- "mysql --user=root --password=$MYSQL_ROOT_PASSWORD -Bse 'FLUSH LOGS; UNLOCK TABLES;'"
ignoreFailure : false
maxRetryCount : 1
timeoutSeconds : 10 BackupPlan illustrating MySQL Hook
Copy apiVersion : triliovault.trilio.io/v1
kind : BackupPlan
metadata :
name : mysql-backupplan
spec :
backupConfig :
target :
namespace : default
name : demo-s3-target
retentionPolicy :
name : sample-policy
hookConfig :
mode : Sequential
hooks :
- hook :
name : mysql-hook
podSelector :
labels :
- matchLabels :
app : mysql-qa
regex : mysql-qa*
containerRegex : mysql-qa* Hook for Cassandra
Copy apiVersion : triliovault.trilio.io/v1
kind : Hook
metadata :
name : cassandra-hook
spec :
pre :
execAction :
command :
- "bash"
- "-c"
- "nodetool flush -- demodb;"
ignoreFailure : false
maxRetryCount : 1
timeoutSeconds : 10
post :
execAction :
command :
- "bash"
- "-c"
- "nodetool verify -- demodb;"
ignoreFailure : false
maxRetryCount : 1
timeoutSeconds : 10 BackupPlan illustrating Cassandra Hook
Copy apiVersion : triliovault.trilio.io/v1
kind : BackupPlan
metadata :
name : cassandra-backupplan
spec :
backupConfig :
target :
namespace : default
name : demo-s3-target
retentionPolicy :
name : sample-policy
hookConfig :
mode : Sequential
hooks :
- hook :
name : cassandra-hook
podSelector :
labels :
- matchLabels :
app : my-release-cassandra
regex : my-release-cassandra*
containerRegex : my-release-cassandra* Hook for MongoDB
Copy apiVersion : triliovault.trilio.io/v1
kind : Hook
metadata :
name : mongo-hook
spec :
pre :
execAction :
command :
- "/bin/bash"
- "-c"
- "/opt/bitnami/mongodb/bin/mongo --eval 'printjson(db.fsyncLock())' --host 'mongotest-mongodb' --authenticationDatabase admin -u root -p $MONGODB_ROOT_PASSWORD"
ignoreFailure : false
maxRetryCount : 2
timeoutSeconds : 10
post :
execAction :
command :
- "/bin/bash"
- "-c"
- "/opt/bitnami/mongodb/bin/mongo --eval 'printjson(db.fsyncUnlock())' --host 'mongotest-mongodb' --authenticationDatabase admin -u root -p $MONGODB_ROOT_PASSWORD"
ignoreFailure : false
maxRetryCount : 2
timeoutSeconds : 10 BackupPlan illustrating MongoDB Hook
Copy apiVersion : triliovault.trilio.io/v1
kind : BackupPlan
metadata :
name : mongo-backupplan
spec :
backupConfig :
target :
namespace : default
name : demo-s3-target
retentionPolicy :
name : sample-policy
hookConfig :
mode : Sequential
hooks :
- hook :
name : mongo-hook
podSelector :
regex : mongotest-mongodb*
containerRegex : mongodb* Hook for MariaDB
Copy apiVersion : triliovault.trilio.io/v1
kind : Hook
metadata :
name : mariadb-hook
spec :
pre :
execAction :
command :
- "bash"
- "-c"
- "mysql --user=root --password=$MARIADB_ROOT_PASSWORD -Bse 'FLUSH TABLES WITH READ LOCK;system ${WAIT_CMD};'"
ignoreFailure : false
maxRetryCount : 1
timeoutSeconds : 10
post :
execAction :
command :
- "bash"
- "-c"
- "mysql --user=root --password=$MARIADB_ROOT_PASSWORD -Bse 'FLUSH LOGS; UNLOCK TABLES;'"
ignoreFailure : false
maxRetryCount : 1
timeoutSeconds : 10 BackupPlan illustrating MariaDB Hook
Copy apiVersion : triliovault.trilio.io/v1
kind : BackupPlan
metadata :
name : mariadb-backupplan
spec :
backupConfig :
target :
namespace : default
name : demo-s3-target
retentionPolicy :
name : sample-policy
hookConfig :
mode : Sequential
hooks :
- hook :
name : mariadb-hook
podSelector :
labels :
- matchLabels :
app.kubernetes.io/name : mariadb
regex : mariadb*
containerRegex : mariadb* Hook for Redis
Copy apiVersion : triliovault.trilio.io/v1
kind : Hook
metadata :
name : redis-hook
spec :
pre :
execAction :
command :
- "bash"
- "-c"
- "echo 'SAVE' | redis-cli -h my-release-redis-master -a $REDIS_PASSWORD --no-auth-warning"
ignoreFailure : false
maxRetryCount : 1
timeoutSeconds : 10
post :
execAction :
command :
- "bash"
- "-c"
- "echo 'post hook action completed'"
ignoreFailure : false
maxRetryCount : 1
timeoutSeconds : 10 BackupPlan illustrating Redis Hook
Copy apiVersion : triliovault.trilio.io/v1
kind : BackupPlan
metadata :
name : redis-backupplan
spec :
backupConfig :
target :
namespace : default
name : demo-s3-target
retentionPolicy :
name : sample-policy
hookConfig :
mode : Sequential
hooks :
- hook :
name : redis-hook
podSelector :
labels :
- matchLabels :
app : redis
role : master
regex : my-release-redis-master* Hook for PostgreSQL
Copy apiVersion : triliovault.trilio.io/v1
kind : Hook
metadata :
name : postgres-hook
spec :
pre :
execAction :
command :
- "PGPASSWORD=$POSTGRES_PASSWORD; psql -U '$POSTGRES_USER' -c 'CHECKPOINT';"
ignoreFailure : false
maxRetryCount : 1
timeoutSeconds : 10
post :
execAction :
command :
- "bash"
- "-c"
- "echo 'post hook action completed'"
Ignore Failure : false
Max Retry Count : 1
Timeout Seconds : 10 BackupPlan illustrating PostgreSQL Hook
Copy apiVersion : triliovault.trilio.io/v1
kind : BackupPlan
metadata :
name : postgres-backupplan
spec :
backupConfig :
target :
namespace : default
name : demo-s3-target
retentionPolicy :
name : sample-policy
hookConfig :
mode : Sequential
hooks :
- hook :
name : postgres-hook
podSelector :
labels :
- matchLabels :
app.kubernetes.io/name : postgresql
regex : postgres-postgresql*
containerRegex : postgres-postgresql* Hook for InfluxDB
Copy apiVersion : triliovault.trilio.io/v1
kind : Hook
metadata :
name : influxdb-hook
spec :
pre :
execAction :
command :
- "bash"
- "-c"
- "bkpfile=/tmp/snap`date +%Y%m%dT%H%M`; influxd backup $bkpfile"
ignoreFailure : false
maxRetryCount : 1
timeoutSeconds : 10
post :
execAction :
command :
- "bash"
- "-c"
- "echo 'post hook action completed'"
ignoreFailure : false
maxRetryCount : 1
timeoutSeconds : 10 BackupPlan illustrating InfluxDB Hook
Copy apiVersion : triliovault.trilio.io/v1
kind : BackupPlan
metadata :
name : influxdb-backupplan
spec :
backupConfig :
target :
namespace : default
name : demo-s3-target
retentionPolicy :
name : sample-policy
backupPlanComponents :
custom :
- matchLabels :
app : influxdb
hookConfig :
mode : Sequential
hooks :
- hook :
name : influxdb-hook
podSelector :
labels :
- matchLabels :
app : influxdb
regex : influxdb* Hook for Elasticsearch
Copy apiVersion : triliovault.trilio.io/v1
kind : Hook
metadata :
name : es-hook
spec :
pre :
execAction :
command :
- "bash"
- "-c"
- "curl -XPOST 'http://localhost:9200/test/_flush?pretty=true'; curl -H'Content-Type: application/json' -XPUT localhost:9200/test/_settings?pretty -d'{\"index\": {\"blocks.read_only\": true} }'"
ignoreFailure : false
maxRetryCount : 1
timeoutSeconds : 10
post :
execAction :
command :
- "bash"
- "-c"
- "curl -H'Content-Type: application/json' -XPUT localhost:9200/test/_settings?pretty -d'{\"index\": {\"blocks.read_only\": false} }'"
ignoreFailure : false
maxRetryCount : 1
timeoutSeconds : 10 BackupPlan illustrating Elasticsearch Hook
Copy apiVersion : triliovault.trilio.io/v1
kind : BackupPlan
metadata :
name : es-backupplan
spec :
backupConfig :
target :
namespace : default
name : demo-s3-target
retentionPolicy :
name : sample-policy
hookConfig :
mode : Sequential
hooks :
- hook :
name : es-hook
podSelector :
labels :
- matchLabels :
app : master
app.kubernetes.io/name : elasticsearch
regex : my-release-elasticsearch-master*
containerRegex : elasticsearch* Hook for Kafka
Copy apiVersion : triliovault.trilio.io/v1
kind : Hook
metadata :
name : kafka-hook
spec :
pre :
execAction :
command :
- "bash"
- "-c"
- "/opt/bitnami/kafka/bin/kafka-server-stop.sh -daemon /opt/bitnami/kafka/config/server.properties"
ignoreFailure : false
maxRetryCount : 1
timeoutSeconds : 10
post :
execAction :
command :
- "bash"
- "-c"
- "/opt/bitnami/kafka/bin/kafka-server-start.sh -daemon /opt/bitnami/kafka/config/server.properties"
ignoreFailure : false
maxRetryCount : 1
timeoutSeconds : 10 BackupPlan illustrating Kafka Hook
Copy apiVersion : triliovault.trilio.io/v1
kind : BackupPlan
metadata :
name : kafka-backupplan
spec :
backupConfig :
target :
namespace : default
name : demo-s3-target
retentionPolicy :
name : sample-policy
hookConfig :
mode : Sequential
hooks :
- hook :
name : kafka-hook
podSelector :
labels :
- matchLabels :
app.kubernetes.io/name : kafka
regex : my-release-kafka*
containerRegex : kafka* Hook for CockroachDB
Copy apiVersion : triliovault.trilio.io/v1
kind : Hook
metadata :
name : cockroachdb-hook
spec :
pre :
execAction :
command :
- "bash"
- "-c"
- "bkpfile=/tmp/snap`date +%Y%m%dT%H%M`.sql;cockroach dump --dump-all --insecure > $bkpfile"
ignoreFailure : false
maxRetryCount : 1
timeoutSeconds : 10
post :
execAction :
command :
- "bash"
- "-c"
- "echo 'post hook action completed'"
ignoreFailure : false
maxRetryCount : 1
timeoutSeconds : 10 BackupPlan illustrating CockroachDB Hook
Copy apiVersion : triliovault.trilio.io/v1
kind : BackupPlan
metadata :
name : cockroach-helm-backup-plan
spec :
backupConfig :
target :
namespace : default
name : demo-s3-target
retentionPolicy :
name : sample-policy
hookConfig :
mode : Sequential
hooks :
- hook :
name : cockroachdb-hook
podSelector :
labels :
- matchLabels :
app.kubernetes.io/name : cockroachdb
app.kubernetes.io/component : cockroachdb
regex : my-release-cockroachdb-0* BackupPlan
The BackupPlan CRD specifies the backup job. The specification includes the backup schedule, backup target and the resources to backup. Trilio supports three types of resources to backup and an BackupPlan CR may include combination of these resources.
This BackupPlan CR defines a set of resources to backup. Resources can be defined in the form of Helm release, Operators or just bare k8s api resources.
In this release, Trilio supports backup of the following:
Operator-based application instances
Label-based selection of resources
Type: Custom Label - Example 1
The following sample BackupPlan CR specifies a Label-based selection of resources. In this example any resource has a label app with valuegcp-compute-persistent-disk-csi-driver will be backed up.
Copy apiVersion : triliovault.trilio.io/v1
kind : BackupPlan
metadata :
name : sample-application
spec :
backupConfig :
target :
namespace : default
name : sample-target
schedulePolicy :
fullBackupCron :
schedule : "* 0 1 * *"
incrementalCron :
schedule : "* 0 * * *"
backupPlanComponents :
custom :
- matchLabels :
app : gcp-compute-persistent-disk-csi-driver Type: Custom Label - Example 2 - Multiple Labels
The example below explains how a backupPlan CR can be used to protect data using multiple labels.
Copy apiVersion : triliovault.trilio.io/v1
kind : BackupPlan
metadata :
name : sample-application
spec :
backupConfig :
target :
namespace : default
name : sample-target
backupPlanComponents :
custom :
- matchLabels :
app : frontend
- matchLabels :
app : backend Type: Helm - Example 1 - Single Helm Release
The following sample BackupPlan CR that specifies Helm release based resources to backup. Trilio automatically discovers the resources that belong to the release and backups the helm chart as whole. The release that this example backups is mysql-releasename.
Copy apiVersion : triliovault.trilio.io/v1
kind : BackupPlan
metadata :
name : sample-application
spec :
backupConfig :
target :
namespace : default
name : demo-s3-target
schedulePolicy :
fullBackupCron :
schedule : "* 0 1 * *"
incrementalCron :
schedule : "* 0 * * *"
backupPlanComponents :
helmReleases :
- mysql-releasename Type: Helm - Example 2 - Multiple Helm Releases
Copy apiVersion : triliovault.trilio.io/v1
kind : BackupPlan
metadata :
name : sample-application
spec :
backupConfig :
target :
namespace : default
name : demo-s3-target
backupPlanComponents :
helmReleases :
- sample-release
- sample-release1 Type Operator - Example 1
Another type of resource that Trilio supports is an Operator-based applications and the following YAML snippet describes the BackupPlan CR that includes the operator based application.
Example 1a
Copy apiVersion : triliovault.trilio.io/v1
kind : BackupPlan
metadata :
name : sample-application
spec :
backupConfig :
target :
namespace : default
name : demo-s3-target
backupPlanComponents :
operators :
- operatorId : sample-mysqlcluster
customResources :
- groupVersionKind :
group : "mysql.presslabs.org"
version : "v1alpha1"
kind : "MysqlCluster"
objects :
- sample-mysqlcluster
- groupVersionKind :
group : "mysql.presslabs.org"
version : "v1alpha1"
kind : "MysqlBackup"
operatorResourceSelector :
- matchLabels :
app : mysql-operator
release : sample-release
applicationResourceSelector :
- matchLabels :
app.kubernetes.io/managed-by : mysql.presslabs.org
app.kubernetes.io/name : mysql Example 1b
Copy apiVersion : triliovault.trilio.io/v1
kind : BackupPlan
metadata :
name : sample-application
spec :
backupConfig :
target :
namespace : default
name : demo-s3-target
backupPlanComponents :
operators :
- operatorId : etcd-cluster
customResources :
- groupVersionKind :
group : "etcd.database.coreos.com"
version : "v1beta2"
kind : "EtcdCluster"
objects :
- etcd-cluster
operatorResourceSelector : # sa, clusterrole, clusterrolebinding, deployment
- matchLabels :
app : etcd-operator
release : sample-release
applicationResourceSelector : # svc: etcd-cluster, etcd-cluster-client
- matchLabels :
app : etcd
etcd_cluster : etcd-cluster Type Operator - Example 2 - Multiple Operators
The YAML definition below provides an example of how multiple operators can be protected through the BackupPlan CR.
Copy apiVersion : triliovault.trilio.io/v1
kind : BackupPlan
metadata :
name : sample-application
spec :
backupConfig :
target :
namespace : default
name : demo-s3-target
backupPlanComponents :
operators :
- operatorId : sample-mysqlcluster
customResources :
- groupVersionKind :
group : "mysql.presslabs.org"
version : "v1alpha1"
kind : "MysqlCluster"
objects :
- sample-mysqlcluster
- groupVersionKind :
group : "mysql.presslabs.org"
version : "v1alpha1"
kind : "MysqlBackup"
operatorResourceSelector :
- matchLabels :
app : mysql-operator
release : sample-release
applicationResourceSelector :
- matchLabels :
app.kubernetes.io/managed-by : mysql.presslabs.org
app.kubernetes.io/name : mysql
- operatorId : sample-fluxcd-helm-release
customResources :
- groupVersionKind :
group : "helm.fluxcd.io"
kind : "HelmRelease"
version : "v1"
objects :
- redis
operatorResourceSelector :
- matchLabels :
app : helm-operator
release : helm-operator
applicationResourceSelector :
- matchLabels :
app : redis
release : redis Type: Operator - Example 3 - Helm based operator
Operator based applications can also be protected by providing the helm release name for the Operator resources.
Example 3a
Copy apiVersion : triliovault.trilio.io/v1
kind : BackupPlan
metadata :
name : sample-application
spec :
backupConfig :
target :
namespace : default
name : demo-s3-target
backupPlanComponents :
operators :
- operatorId : sample-mysqlcluster
customResources :
- groupVersionKind :
group : "mysql.presslabs.org"
version : "v1alpha1"
kind : "MysqlCluster"
objects :
- sample-mysqlcluster
- groupVersionKind :
group : "mysql.presslabs.org"
version : "v1alpha1"
kind : "MysqlBackup"
helmRelease : sample-release
applicationResourceSelector :
- matchLabels :
app.kubernetes.io/managed-by : mysql.presslabs.org
app.kubernetes.io/name : mysql Example 3b
Copy apiVersion : triliovault.trilio.io/v1
kind : BackupPlan
metadata :
name : sample-application
spec :
backupConfig :
target :
namespace : default
name : demo-s3-target
backupPlanComponents :
operators :
- operatorId : sample-fluxcd-helm-release
customResources :
- groupVersionKind :
group : "helm.fluxcd.io"
kind : "HelmRelease"
version : "v1"
objects :
- redis
helmRelease : helm-operator
applicationResourceSelector :
- matchLabels :
app : redis
release : redis Type: Operator - Example 4 - Helm based Operator without Operator Custom Resources
Operator Example where the Operator resources are defined via the helm release. No Operator custom resources are being backed up in this example.
Example 4a
Copy apiVersion : triliovault.trilio.io/v1
kind : BackupPlan
metadata :
name : sample-application
spec :
backupConfig :
target :
namespace : default
name : demo-s3-target
backupPlanComponents :
operators :
- operatorId : sample-mysqlcluster
helmRelease : sample-release
applicationResourceSelector :
- matchLabels :
app.kubernetes.io/managed-by : mysql.presslabs.org
app.kubernetes.io/name : mysql Example 4b
Copy apiVersion : triliovault.trilio.io/v1
kind : BackupPlan
metadata :
name : sample-application
spec :
backupConfig :
target :
namespace : default
name : demo-s3-target
backupPlanComponents :
operators :
- operatorId : sample-fluxcd-helm-release
helmRelease : helm-operator
applicationResourceSelector :
- matchLabels :
app : redis
release : redis Type: Operator - Example 5 - OLM operators
OLM or Openshift Operators can also be protected by providing the subscription name and namespace in operator resources.
Example 5a. backups CockroachDB operator subscription named cockroachdb-certified along with its custom resource named crdb-tls-example of the kind CrdbCluster
Example 5a
Copy apiVersion: triliovault.trilio.io/v1
kind: BackupPlan
metadata:
name: sample-application
spec:
backupConfig:
target:
name: demo-s3-target
namespace: default
backupPlanComponents:
operators:
- customResources:
- groupVersionKind:
group: crdb.cockroachlabs.com
kind: CrdbCluster
version: v1alpha1
objects:
- crdb-tls-example
operatorId: crdb
subscription:
name: cockroachdb-certified
namespace: openshift-operators Example 5b. backups Openshift Pipelines operator subscription named openshift-pipelines-operator-rh along with its custom resources apply-manifests and update-deployment of the kind Task , and build-and-deploy of kind Pipeline
Example 5b
Copy apiVersion: triliovault.trilio.io/v1
kind: BackupPlan
metadata:
name: sample-application
namespace: pipelines-tutorial
spec:
backupConfig:
target:
name: sample-target
namespace: pipelines-tutorial
backupPlanComponents:
operators:
- customResources:
- groupVersionKind:
group: tekton.dev
kind: Task
version: v1
objects:
- apply-manifests
- update-deployment
- groupVersionKind:
group: tekton.dev
kind: Pipeline
version: v1
objects:
- build-and-deploy
operatorId: pipelines
subscription:
name: openshift-pipelines-operator-rh
namespace: openshift-operators Type: All - Example with all 3 Application Types -1
The BackupPlan CRD is an extremely flexible CRD within which multiple application components can be specified.
Copy apiVersion: triliovault.trilio.io/v1
kind: BackupPlan
metadata:
name: sample-application
spec:
backupConfig:
target:
namespace: default
name: demo-s3-target
backupPlanComponents:
custom:
- matchLabels:
triliobackupall: all
helmReleases:
- sample-release
operators:
- operatorId: sample-mysqlcluster
customResources:
- groupVersionKind:
group: "mysql.presslabs.org"
version: "v1alpha1"
kind: "MysqlCluster"
objects:
- sample-mysqlcluster
- groupVersionKind:
group: "mysql.presslabs.org"
version: "v1alpha1"
kind: "MysqlBackup"
operatorResourceSelector:
- matchLabels:
app: mysql-operator
release: sample-release
applicationResourceSelector:
- matchLabels:
app.kubernetes.io/managed-by: mysql.presslabs.org
app.kubernetes.io/name: mysql Type: Namespace - Example 1
For namespace level backup, BackupPlan components are optional
Copy apiVersion: triliovault.trilio.io/v1
kind: BackupPlan
metadata:
name: ns-backupplan-1
spec:
backupConfig:
target:
namespace: default
name: demo-s3-target
Type: ClusterBackupPlan - Example 1
ClusterBackupPlan is used to protect multiple namespaces in the cluster. User can specify multiple namespaces to be backed up.
Copy apiVersion: triliovault.trilio.io/v1
kind: ClusterBackupPlan
metadata:
name: sample-clusterbplan
spec:
backupConfig:
target:
name: sample-target
namespace: TARGET_NAMESPACE
retentionPolicy:
name: sample-retention-policy
namespace: POLICY_NAMESPACE
schedulePolicy:
fullBackupPolicy:
name: sample-schedule-policy
namespace: POLICY_NAMESPACE
backupComponents:
- namespace: NS_1
- namespace: NS_2 Type: Inclusion/Exclusion
User can provide specific resource which needs to be included or excluded while doing backup. Resources can be specified either by GVKO or by the Kind directly.
Copy apiVersion: triliovault.trilio.io/v1
kind: BackupPlan
metadata:
name: sample-backupplan
spec:
backupConfig:
target:
name: sample-target
namespace: TARGET_NAMESPACE
includeResources:
labelSelector:
- matchLabels:
triliobackupall: trilio
excludeResources:
labelSelector:
- matchLabels:
triliobackupall: trilio Type: Encryption
Setup Master Encryption key using this
Create a secret for encrypting the backups.
Copy apiVersion: v1
kind: Secret
metadata:
name: sample-secret
type: Opaque
data:
encryptKey: bXllbmNyeXB0aW9ua2V5
Copy apiVersion: triliovault.trilio.io/v1
kind: BackupPlan
metadata:
name: sample-application
spec:
encryption:
encryptionSecret:
name: sample-secret
namespace: BACKUPPLAN_NAMESPACE
backupConfig:
target:
name: sample-target
retentionPolicy:
name: sample-retention-policy
backupPlanComponents:
helmReleases:
- mysql Backup
The Backup CRD takes a backup of the resources specified in the BackupPlan spec. It takes either a Full backup or an Incremental Backup. The first backup of the Application will always be a Full backup, even if the user specifies their backup type as Incremental.
Note: BackupPlan and Backup CR should be created in the same namespace.
The incremental backup includes all the YAML files and delta changes to PV data. Incremental backups are not complete by themselves and rely on all the previous incremental backups and the full backup.
Example 1 - Full Backup
The following sample Backup YAML for Full backup
Copy apiVersion: triliovault.trilio.io/v1
kind: Backup
metadata:
name: sample-backup
spec:
type: Full
backupPlan:
name: sample-application
namespace: default Example 2 - Incremental Backup
Sample Backup YAML for incremental backup
Copy apiVersion: triliovault.trilio.io/v1
kind: Backup
metadata:
name: sample-backup
spec:
type: Incremental
backupPlan:
name: sample-application
namespace: default
The scheduleType the field is to specify if the backup should follow the backup schedule provided in the backupPlan or if it should be a one-time backup only.
Example 3 - ClusterBackup
Sample Backup YAML for ClusterBackup. This CR is used to back up multiple namespaces and should be used with ClusterBackupPlan. It takes either a Full backup or an Incremental Backup. The first backup will always be a Full backup, even if the user specifies their backup type as Incremental.
Copy apiVersion: triliovault.trilio.io/v1
kind: ClusterBackup
metadata:
name: sample-clusterbackup
spec:
clusterBackupPlan:
name: sample-clusterbplan
type: Full Cancel In-Progress Backup or ClusterBackup:
To cancel any in-progress backup or cluster backup, user can add triliovault.trilio.io/mark-for-cancel: "true" annotation to the backup yaml:
Copy apiVersion: triliovault.trilio.io/v1
kind: Backup
metadata:
annotations:
triliovault.trilio.io/mark-for-cancel: "true"
name: mysql-backup-1
namespace: default
spec:
backupPlan:
apiVersion: triliovault.trilio.io/v1
kind: BackupPlan
name: mysql-bplan
namespace: default
resourceVersion: "4473"
uid: eb33a8cc-b9e6-40f8-a6a2-60e62e2472aa
type: Full User can only cancel in-progress backup and canceling completed or available backups will be blocked.
Restore
The Restore CRD specifies the backup that resources need to be restored from. Resources can be restored to the same namespace or a different namespace. If a backup target with existing backups are created to a different cluster, those backups can be restored to the cluster. A migration or disaster recovery use case can be implemented using this functionality.
Example 1 - Restore from a specific backup
Sample YAML file to restore from a specific backup. The backup is identified by name sample-backup.
Copy apiVersion: triliovault.trilio.io/v1
kind: Restore
metadata:
name: sample-restore
spec:
source:
type: Backup
backup:
name: sample-backup
namespace: default Example 2 - Restore last successful backup
Sample YAML that restores latest backup of a BackupPlan.
Copy apiVersion: triliovault.trilio.io/v1
kind: Restore
metadata:
name: sample-restore
spec:
source:
type: BackupPlan
backupPlan:
name: sample-application
namespace: default Example 3 - Skip Restore
Trilio provides the capability to Skip the restoration of objects if they already exist in the namespace. This is achieved by using the skipIfAlreadyExists field within the Restore custom resource
Copy apiVersion: triliovault.trilio.io/v1
kind: Restore
metadata:
name: sample-restore
spec:
source:
type: Backup
backup:
name: sample-backup
namespace: default
skipIfAlreadyExists: true Example 4 - Patch Restore
Trilio provides the capability to Patch resources if they already existing in the namespace during the restore
Copy apiVersion: triliovault.trilio.io/v1
kind: Restore
metadata:
name: sample-restore
spec:
source:
type: Backup
backup:
name: sample-backup
namespace: default
patchIfAlreadyExists: true Example 5 -Restore From Specific Location - Migration Scenario
Copy apiVersion: triliovault.trilio.io/v1
kind: Restore
metadata:
name: sample-restore
spec:
source:
type: Location
location: e90ce943-a0ce-11ea-93b2-42010a8e0038/66777359-a0d1-11ea-93b2-42010a8e0038
target:
name: sample-target
namespace: default Example 6 - Restore with Transformations (StorageClass)
Copy apiVersion: triliovault.trilio.io/v1
kind: Restore
metadata:
name: sample-restore
spec:
source:
type: Backup
backup:
name: sample-backup
namespace: default
transformComponents:
helm:
- release: mysql
transformName: t1
set:
- key: persistence.storageClass
value: insert.storage.class.here
custom:
- transformName: t3
resources:
groupVersionKind:
group: ""
version: v1
kind: PersistentVolumeClaim
objects:
- insert-pvc1-to-be-replaced
- insert-pvc2-to-be-replaced
jsonPatches:
- op: replace
path: "/spec/storageClassName"
value: insert.updated.storage.class.here Example 7 - Restore with Transformations (NodePort)
Copy apiVersion: triliovault.trilio.io/v1
kind: Restore
metadata:
name: sample-restore
spec:
source:
type: Backup
backup:
name: sample-backup
namespace: default
transformComponents:
custom:
- transformName: t1
resources:
groupVersionKind:
group: ""
version: v1
kind: Service
objects:
- insert-svc-name-to-be-updated
jsonPatches:
- op: replace
path: "/spec/ports/0/nodePort"
value: 31366 Example 8 - Restore with Exclusions
Exclude the specific resources from the backup by defining them with a labelSelector or gvkSelector in the excludeResourceSelector field.
Copy apiVersion: triliovault.trilio.io/v1
kind: Restore
metadata:
name: sample-restore
spec:
source:
type: Backup
backup:
name: sample-backup
namespace: default
excludeResourceSelector:
labelSelector:
- matchLabels:
run: "nginx"
gvkSelector:
- groupVersionKind:
group: ""
version: "v1"
kind: "ConfigMap" Example 9 - Restore with Inclusions
Select the resources for restoration using the resourceSelector field. This allows the user to restore only the specified resources, rather than the entire backup.
Copy apiVersion: triliovault.trilio.io/v1
kind: Restore
metadata:
name: sample-restore
spec:
source:
type: Backup
backup:
name: sample-backup
namespace: default
resourceSelector:
labelSelector:
- matchLabels:
app: "nginx"
gvkSelector:
- groupVersionKind:
version: "v1"
kind: "ServiceAccount"
objects:
- "default" Example 10 - Restore from BackupPlan
Restore from BackupPlan enables users to restore the last successful backup based on a BackupPlan by providing only the name of the BackupPlan.
Copy apiVersion: triliovault.trilio.io/v1
kind: Restore
metadata:
name: sample-restore
spec:
source:
type: BackupPlan
backupPlan:
name: sample-backupplan-helm3
namespace: default Example 11 - ClusterRestore
ClusterRestore enables users to restore the last successful ClusterBackup based on a ClusterBackupPlan. It is used to restore multiple namespaces protected by the ClusterBackup. The ClusterBackup is identified by name sample-clusterbackup. It also provides a flag to cleanup in case of a failure.
Copy apiVersion: triliovault.trilio.io/v1
kind: ClusterRestore
metadata:
name: sample-clusterrestore
spec:
source:
type: ClusterBackup
clusterBackup:
name: sample-clusterbackup
globalConfig:
restoreFlags:
skipIfAlreadyExists: true
components:
- backupNamespace: NS_1
restoreNamespace: NS_RES_1
- backupNamespace: NS_2
restoreNamespace: NS_RES_2
cleanupOnFailure: true Example 12 - Encryption - Restore From Specific Location with encryption
Restore with encryption key is used to decrypt the encrypted data as shown in Type: ClusterBackupPlan - Example 1 .
Copy apiVersion: triliovault.trilio.io/v1
kind: Restore
metadata:
name: sample-restore
spec:
source:
type: Location
location: e90ce943-a0ce-11ea-93b2-42010a8e0038/66777359-a0d1-11ea-93b2-42010a8e0038
target:
name: sample-target
encryption:
encryptionSecret:
name: sample-secret
namespace: SECRET_NAMESPACE 