Enabling TVO 4.1 or older backups when using NFS backup target
TrilioVault for OpenStack is generating a base64 hash value for every NFS backup target connected to the TVO solution.
This enables TVO to mount multiple NFS backup targets to the same TVO installation.
The mountpoints are generated utilizing a hash value inside the mountpoint, providing a unique mount for every NFS backup target.
This mountpoint is then used inside the incremental backups to point to the qcow2 backing files.
The backing file path is required as a full path and can not be set as a relative path.
This is a limitation of the qcow2 format.
TVO 4.2 has changed how the hash value gets calculated.
TVO 4.1 and prior calculated the hash value out of the complete NFS path provided as shown in the example below.
# echo -n 10.10.2.20:/Trilio_Backup | base64
MTAuMTAuMi4yMDovVHJpbGlvX0JhY2t1cA==
TVO 4.2 is now only considering the NFS directory part for the hash value as shown below.
# echo -n /Trilio_Backup | base64
L1RyaWxpb19CYWNrdXA=
It is therefore necessary to make older backups taken by TVO 4.1 or prior available for TVO4.2 to restore.
This can be done by one of two methods:
- 1.Rebase all backups and change the backing file path
- 2.Make the old mount point available again and point it to the new one using mount bind
This method is taking a significant amount of time depending on the number of backups, that need to be rebased. It is therefore recommended to determine the required time through a test workload.
Trilio is providing a script, that is taking care of the rebase procedure.
This script can be downloaded from the following location.
solutions/backing_file_update.sh at master · trilioData/solutions
GitHub
Rebase script for TVO backups
Copy and use the script from the TrilioVault appliance as the nova user.
The nova user is required, as this user owns the backup files, and using any other user will change the ownership and lead to unrestorable backups.
The script is requiring the complete path to the workload directory on the backup target.
./backing_file_update.sh /var/triliovault-mounts/<base64>/workload_<workload_id>
This needs to be repeated until all workloads have been rebased.
This method is a temporary solution, which is to be kept in place until all workloads have gone through a complete retention cycle.
Once all Workloads only contain backups created by TVO 4.2 it is no longer required to keep the mount bind active.
This method is generating a second mount point based on the old hash value calculation and then mounting the new mount point to the old one.
By doing this mount bind both mount points will be available and point to the same backup target.
To use this method the following information needs to be available:
- old mount path
- new mount path
Examples of how to calculate the base64 hash value are shown below.
Old mountpoint hash value:
# echo -n 10.10.2.20:/Trilio_Backup | base64
MTAuMTAuMi4yMDovVHJpbGlvX0JhY2t1cA==
New mountpoint hash value:
# echo -n /Trilio_Backup | base64
L1RyaWxpb19CYWNrdXA=
The mount bind needs to be done for all TrilioVault appliances and Datamover services.
To enable the mount bind on the TrilioVault appliance follow these steps:
- 1.Create the old mountpoint directory
mkdir -p old_mount_path - 2.run mount --bind command
mount --bind <new_mount_path> <old_mount_path> - 3.set permissions for mountpoint
chmod 777 <old_mount_path>
It is recommended to use df -h to identify the current mountpoint as RHOSP, TripleO and Kolla Ansible OpenStack are using a different path than Ansible OpenStack or Canonical OpenStack.
An example is given below.
mkdir /var/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=/
mount --bind /var/triliovault-mounts/L21udC90dmF1bHQvdHZtNA==/ /var/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=
chmod 777 /var/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=/
The following steps need to be done on the overcloud compute node. They don't need to be done inside any container.
To enable the mount bind on the TrilioVault appliance follow these steps:
- 1.Create the old mountpoint directory
mkdir -p old_mount_path - 2.run mount --bind command
mount --bind <new_mount_path> <old_mount_path> - 3.set permissions for mountpoint
chmod 777 <old_mount_path>
mkdir /var/lib/nova/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=/
mount --bind /var/lib/nova/triliovault-mounts/L21udC90dmF1bHQvdHZtNA==/ /var/lib/nova/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=
chmod 777 /var/lib/nova/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=/
The following steps need to be done on the overcloud compute node. They don't need to be done inside any container.
To enable the mount bind on the TrilioVault appliance follow these steps:
- 1.Create the old mountpoint directory
mkdir -p old_mount_path - 2.run mount --bind command
mount --bind <new_mount_path> <old_mount_path> - 3.set permissions for mountpoint
chmod 777 <old_mount_path>
mkdir /var/trilio/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=/
mount --bind /var/trilio/triliovault-mounts/L21udC90dmF1bHQvdHZtNA==/ /var/trilio/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=
chmod 777 /var/trilio/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=/
The following steps need to be done on the overcloud compute node. They don't need to be done inside any container.
To enable the mount bind on the TrilioVault appliance follow these steps:
- 1.Create the old mountpoint directory
mkdir -p old_mount_path - 2.run mount --bind command
mount --bind <new_mount_path> <old_mount_path> - 3.set permissions for mountpoint
chmod 777 <old_mount_path>
mkdir /var/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=/
mount --bind /var/triliovault-mounts/L21udC90dmF1bHQvdHZtNA==/ /var/trilio/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=
chmod 777 /var/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=/
Canonical OpenStack does the creation of the mountpoint and the mount bind through JuJu using the following commands.
To create the mountpoint, if it doesn't already exist:
juju [-m <model>] run --application trilio-data-mover "sudo -u nova mkdir /var/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=/"
juju [-m <model>] run --application trilio-wlm "sudo -u nova mkdir /var/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=/
To create the mount bind
juju [-m <model>] run --application trilio-data-mover "sudo mount --bind /var/triliovault-mounts/L21udC90dmF1bHQvdHZtNA==/ /var/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=/"
juju [-m <model>] run --application trilio-wlm "sudo mount --bind /var/triliovault-mounts/L21udC90dmF1bHQvdHZtNA==/ /var/triliovault-mounts/MTkyLjE2OC4xLjM1Oi9tbnQvdHZhdWx0L3R2bTQ=/"