Enabling T4O 4.1 or older backups when using NFS backup target

Trilio for OpenStack generates a base64 hash value for every NFS backup target connected to the T4O solution. This enables T4O to mount multiple NFS backup targets to the same T4O installation.

The mountpoints are generated utilizing a hash value inside the mountpoint, providing a unique mount for every NFS backup target.

This mountpoint is then used inside the incremental backups to point to the qcow2 backing files. The backing file path is required as a full path and can not be set as a relative path. This is a limitation of the qcow2 format.

With T4O 4.2 there was a significant change in how the hash value gets calculated. T4O 4.1 and prior calculated the hash value out of the complete NFS path provided as shown in the example below.

# echo -n 10.10.2.20:/Trilio_Backup | base64
MTAuMTAuMi4yMDovVHJpbGlvX0JhY2t1cA==

T4O 4.2 is now only considering the NFS directory part for the hash value as shown below.

# echo -n /Trilio_Backup | base64
L1RyaWxpb19CYWNrdXA=

It is therefore necessary to make older backups taken by T4O 4.1 or prior available for T4O 4.2 to restore.

This can be done by one of two methods:

1. Rebase all backups and change the backing file path

2. Make the old mount point available again and point it to the new one using mount bind

Rebase all backups

This method is taking a significant amount of time depending on the number of backups, that need to be rebased. It is therefore recommended to determine the required time through a test workload.

Trilio provides a script that takes care of the rebase procedure available from the following repository.

solutions/backing_file_update.sh at master · trilioData/solutionsGitHub

Rebase script for T4O backups

Login to the triliovault_wlm_workloads container with nova user and copy the script to the tmp directory inside the container

The nova user is required, as this user owns the backup files, and using any other user will change the ownership and lead to unrestorable backups.

The script is requiring the complete path to the workload directory on the backup target.

./backing_file_update.sh /var/triliovault-mounts/<base64>/workload_<workload_id>

This needs to be repeated until all workloads have been rebased.

Use mount bind with the old mount point

This method is a temporary solution, which is to be kept in place until all workloads have gone through a complete retention cycle.

Once all Workloads only contain backups created by T4O 4.2 it is no longer required to keep the mount bind active.

This method is generating a second mount point based on the old hash value calculation and then mounting the new mount point to the old one. By doing this mount bind both mount points will be available and point to the same backup target.

To use this method the following information needs to be available:

• old mount path

• new mount path

Examples of how to calculate the base64 hash value are shown below.

Old mountpoint hash value:

# echo -n 10.10.2.20:/Trilio_Backup | base64
MTAuMTAuMi4yMDovVHJpbGlvX0JhY2t1cA==

New mountpoint hash value:

# echo -n /Trilio_Backup | base64
L1RyaWxpb19CYWNrdXA=

The mount bind needs to be done for all Trilio appliances and Datamover services.

The mount bind needs to be done on all the Compute nodes and all the wlm containers on the Controller node

Controller Nodes

To activate the mount bind on the controller node, access each of the WLM containers (triliovault_wlm_api, triliovault_wlm_scheduler, triliovault_wlm_workloads, and triliovault_wlm_cron), and execute the following sequence of actions. It's crucial to replicate these steps on all other overcloud controller nodes.

You do not need to do mount bind inside the triliovault_datamover_api and horizon containers

1. Create the old mountpoint directory mkdir -p old_mount_path

2. run mount --bind command mount --bind <new_mount_path> <old_mount_path>

3. set permissions for mountpoint chmod 777 <old_mount_path>

It is recommended to use df -h to identify the current mountpoint as all the supported OpenStack distros may have different mount path.

An example is given below.

mkdir -p /var/lib/nova/triliovault-mounts/MTcyLjMwLjEuOTovbW50L3Job3NwdGFyZ2V0bmZz
mount --bind /var/lib/nova/triliovault-mounts/L21udC9yaG9zcHRhcmdldG5mcw\=\=/ /var/lib/nova/triliovault-mounts/MTcyLjMwLjEuOTovbW50L3Job3NwdGFyZ2V0bmZz
chmod 777 /var/lib/nova/triliovault-mounts/MTcyLjMwLjEuOTovbW50L3Job3NwdGFyZ2V0bmZz

Compute Nodes

The following steps need to be done on all the overcloud compute nodes themselves and not inside of a container.

To enable the mount bind on the Compute node follow the below steps:

1. Create the old mountpoint directory mkdir -p old_mount_path

2. Run mount --bind command mount --bind <new_mount_path> <old_mount_path>

3. Set permissions for mountpoint chmod 777 <old_mount_path>

It is recommended to use df -h to identify the current mountpoint as all the supported OpenStack distros may have different mount path.

An example is given below.

mkdir -p /var/lib/nova/triliovault-mounts/MTcyLjMwLjEuOTovbW50L3Job3NwdGFyZ2V0bmZz
mount --bind /var/lib/nova/triliovault-mounts/L21udC9yaG9zcHRhcmdldG5mcw\=\=/ /var/lib/nova/triliovault-mounts/MTcyLjMwLjEuOTovbW50L3Job3NwdGFyZ2V0bmZz
chmod 777 /var/lib/nova/triliovault-mounts/MTcyLjMwLjEuOTovbW50L3Job3NwdGFyZ2V0bmZz