Known Issues and Workarounds

This page describes issues that may be encountered and potential solutions to those issues.

Deprecated Documentation

This document is deprecated and no longer supported. For accurate, up-to-date information, please refer to the documentation for the latest version of Trilio.

# Known Issues and Workarounds

Target Creation Failure

kubectl apply -f <backup_target.yaml> is not successful and the resource goes into Unavailable state.

Possible reasons for the target marked Unavailable could be not having enough permissions or lack of network connectivity. Target resource creation launches a validation pod to check if Trilio can connect and write to the Target. Trilio then cleans the pod after the validation process completes, which clears any logs that the validation pod has. Users should repeat the same operation and actively check the logs on the validation pods.

Kubernetes

kubectl get pods -A | grep validator
kubectl logs <validator-pod> -n <namespace>

OpenShift

oc get pods -A | grep validator
oc logs <validator-pod> -n <namespace

Backup Failures

Please follow the following steps to analyze backup failures.

kubectl get jobs
oc get jobs

To troubleshoot backup issues, it is important to know the different phases of jobs.triliovault pod.

1. <backupname>-snapshot: Trilio leverages CSI snapshot functionality to take PV snapshot. If the backup fails in this step user should try a manual snapshot of the PV to make sure all required drivers are present, and this operation works without Trilio.

2. <backupname>-upload: In this phase, Trilio uploads metadata and data to the Target. If the backup fails in this phase, we need to look into the following logs for error.

Kubernetes

kubectl get pods -A | grep k8s-triliovault-control-plane
kubectl logs <controller POD>

OpenShift

oc get pods -A | grep k8s-triliovault-control-plane
oc logs <controller POD>

Restore Failures

The most common reasons for restore failures are resource conflicts. Users may try to restore the backup to the same namespace as the original application and restored application names conflicts with an existing application. Please check the logs to analyze the root cause of the issue.

Kubernetes

kubectl get pods -A | grep k8s-triliovault-control-plane
kubectl logs <controller POD>

OpenShift

oc get pods -A | grep k8s-triliovault-control-plane
oc logs <controller POD>

Webhooks are installed

Problem Description: If a Trilio operation is failing with an error Internal error occurred: failed or service "k8s-triliovault-webhook-service" not found

Error from server (InternalError): error when creating "targetmigration.yaml": Internal error occurred: failed
calling webhook "mapplication.kb.io": Post https://k8s-triliovault-webhook-service.default.svc:443/mutate?timeout=30s: service "k8s-triliovault-webhook-service" not found

Solution : This might be due to installation of T4K in multiple Namespaces. Even a simple clean up of those installations will not have webhook instances cleared. Please run the following commands to get all the webhooks instances.

kubectl get validatingwebhookconfigurations  -A | grep trilio
kubectl get mutatingwebhookconfigurations -A | grep trilio

Delete duplicate entries if any.

Restore Failed

Problem Description: If you are not able to restore an application into different namespace (OpenShift)

Error:
Permission denied: could not bind to address 0.0.0.0:80

Solution: User may not have sufficient permissions to restore the application into the namespace.

oc adm policy add-scc-to-user anyuid -z default -n <NameSpace>

Uninstall Trilio

To uninstall Trilio and start fresh, delete all Trilio CRD's.

On Kubernetes

kubectl delete crds $(kubectl get crds | grep trilio | awk '{print $1}')

On OpenShift

oc delete crds $(kubectl get crds | grep trilio | awk '{print $1}')