Enabling T4O 4.1 or older backups when using NFS backup target

Trilio for OpenStack is generating a base64 hash value for every NFS backup target connected to the T4O solution. This enables T4O to mount multiple NFS backup targets to the same T4O installation.

The mountpoints are generated utilizing a hash value inside the mountpoint, providing a unique mount for every NFS backup target.

This mountpoint is then used inside the incremental backups to point to the qcow2 backing files. The backing file path is required as a full path and can not be set as a relative path. This is a limitation of the qcow2 format.

T4O 4.2 has changed how the hash value gets calculated. T4O 4.1 and prior calculated the hash value out of the complete NFS path provided as shown in the example below.

# echo -n 10.10.2.20:/Trilio_Backup | base64
MTAuMTAuMi4yMDovVHJpbGlvX0JhY2t1cA==

T4O 4.2 is now only considering the NFS directory part for the hash value as shown below.

# echo -n /Trilio_Backup | base64
L1RyaWxpb19CYWNrdXA=

It is therefore necessary to make older backups taken by T4O 4.1 or prior available for TVO4.2 to restore.

This can be done by one of two methods:

1. Rebase all backups and change the backing file path

2. Make the old mount point available again and point it to the new one using mount bind

Rebase all backups

This method is taking a significant amount of time depending on the number of backups, that need to be rebased. It is therefore recommended to determine the required time through a test workload.

Trilio is providing a script, that is taking care of the rebase procedure. This script can be downloaded from the following location.

solutions/backing_file_update.sh at master · trilioData/solutionsGitHub

Rebase script for T4O backups

Copy and use the script from the Trilio appliance as the nova user.

The nova user is required, as this user owns the backup files, and using any other user will change the ownership and lead to unrestorable backups.

The script is requiring the complete path to the workload directory on the backup target.

./backing_file_update.sh /var/triliovault-mounts/<base64>/workload_<workload_id>

This needs to be repeated until all workloads have been rebased.

Use mount bind with the old mount point

This method is a temporary solution, which is to be kept in place until all workloads have gone through a complete retention cycle.

Once all Workloads only contain backups created by T4O 4.2 it is no longer required to keep the mount bind active.

This method is generating a second mount point based on the old hash value calculation and then mounting the new mount point to the old one. By doing this mount bind both mount points will be available and point to the same backup target.

To use this method the following information needs to be available:

• old mount path

• new mount path

Examples of how to calculate the base64 hash value are shown below.

Old mountpoint hash value:

# echo -n 10.10.2.20:/Trilio_Backup | base64
MTAuMTAuMi4yMDovVHJpbGlvX0JhY2t1cA==

New mountpoint hash value:

# echo -n /Trilio_Backup | base64
L1RyaWxpb19CYWNrdXA=

The mount bind needs to be done for all Trilio appliances and Datamover services.

Trilio Appliance

To enable the mount bind on the Trilio appliance follow these steps:

1. Create the old mountpoint directory mkdir -p old_mount_path

2. run mount --bind command mount --bind <new_mount_path> <old_mount_path>

3. set permissions for mountpoint chmod 777 <old_mount_path>

It is recommended to use df -h to identify the current mountpoint as RHOSP, TripleO and Kolla Ansible OpenStack are using a different path than Ansible OpenStack or Canonical OpenStack.

An example is given below.

mkdir /var/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=/
mount --bind /var/triliovault-mounts/L21udC90dmF1bHQvdHZtNA==/ /var/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=
chmod 777 /var/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=/

RHOSP/TripleO Datamover

The following steps need to be done on the overcloud compute node. They don't need to be done inside any container.

To enable the mount bind on the Trilio appliance follow these steps:

1. Create the old mountpoint directory mkdir -p old_mount_path

2. run mount --bind command mount --bind <new_mount_path> <old_mount_path>

3. set permissions for mountpoint chmod 777 <old_mount_path>

mkdir /var/lib/nova/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=/
mount --bind /var/lib/nova/triliovault-mounts/L21udC90dmF1bHQvdHZtNA==/ /var/lib/nova/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=
chmod 777 /var/lib/nova/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=/

Kolla Ansible OpenStack Datamover

The following steps need to be done on the overcloud compute node. They don't need to be done inside any container.

To enable the mount bind on the Trilio appliance follow these steps:

1. Create the old mountpoint directory mkdir -p old_mount_path

2. run mount --bind command mount --bind <new_mount_path> <old_mount_path>

3. set permissions for mountpoint chmod 777 <old_mount_path>

mkdir /var/trilio/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=/
mount --bind /var/trilio/triliovault-mounts/L21udC90dmF1bHQvdHZtNA==/ /var/trilio/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=
chmod 777 /var/trilio/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=/

Ansible OpenStack Datamover

The following steps need to be done on the overcloud compute node. They don't need to be done inside any container.

To enable the mount bind on the Trilio appliance follow these steps:

1. Create the old mountpoint directory mkdir -p old_mount_path

2. run mount --bind command mount --bind <new_mount_path> <old_mount_path>

3. set permissions for mountpoint chmod 777 <old_mount_path>

mkdir /var/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=/
mount --bind /var/triliovault-mounts/L21udC90dmF1bHQvdHZtNA==/ /var/trilio/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=
chmod 777 /var/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=/

Canonical Openstack WLM & Datamover containers

Canonical OpenStack does the creation of the mountpoint and the mount bind through JuJu using the following commands.

To create the mountpoint, if it doesn't already exist:

juju exec [-m <model>] --application trilio-data-mover "sudo -u nova mkdir /var/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=/"
juju exec [-m <model>] --application trilio-wlm "sudo -u nova mkdir /var/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=/

To create the mount bind

juju exec [-m <model>] --application trilio-data-mover "sudo mount --bind /var/triliovault-mounts/L21udC90dmF1bHQvdHZtNA==/ /var/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=/"
juju exec [-m <model>] --application trilio-wlm "sudo mount --bind /var/triliovault-mounts/L21udC90dmF1bHQvdHZtNA==/ /var/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=/"