Upgrade Trilio Appliance
Generic Prerequisites
Please ensure to complete the upgrade of all the Trilio components on the Openstack controller & compute nodes before starting the rolling upgrade of Trilio.
The mentioned Gemfury repository should be accessible from Trilio VM.
Either 4.2 GA OR any hotfix patch against 4.2 should be already deployed for performing upgrades mentioned in the current document.
Please ensure the following points before starting the upgrade process:
No snapshot OR restore to be running.
Global job-scheduler should be disabled.
wlm-cron should be disabled and any lingering process should be killed.
Deactivating the wlm-cron service
The following sets of commands will disable the wlm-cron service and verify that it is has been completly shut-down.
[root@TVM2 ~]# pcs resource disable wlm-cron
[root@TVM2 ~]# systemctl status wlm-cron
● wlm-cron.service - workload's scheduler cron service
Loaded: loaded (/etc/systemd/system/wlm-cron.service; disabled; vendor preset : disabled)
Active: inactive (dead)
Jun 11 08:27:06 TVM2 workloadmgr-cron[11115]: 11-06-2021 08:27:06 - INFO - 1...t
Jun 11 08:27:07 TVM2 workloadmgr-cron[11115]: 140686268624368 Child 11389 ki...5
Jun 11 08:27:07 TVM2 workloadmgr-cron[11115]: 11-06-2021 08:27:07 - INFO - 1...5
Jun 11 08:27:07 TVM2 workloadmgr-cron[11115]: Shutting down thread pool
Jun 11 08:27:07 TVM2 workloadmgr-cron[11115]: 11-06-2021 08:27:07 - INFO - S...l
Jun 11 08:27:07 TVM2 workloadmgr-cron[11115]: Stopping the threads
Jun 11 08:27:07 TVM2 workloadmgr-cron[11115]: 11-06-2021 08:27:07 - INFO - S...s
Jun 11 08:27:07 TVM2 workloadmgr-cron[11115]: All threads are stopped succes...y
Jun 11 08:27:07 TVM2 workloadmgr-cron[11115]: 11-06-2021 08:27:07 - INFO - A...y
Jun 11 08:27:09 TVM2 systemd[1]: Stopped workload's scheduler cron service.
Hint: Some lines were ellipsized, use -l to show in full.
[root@TVM2 ~]# pcs resource show wlm-cron
Resource: wlm-cron (class=systemd type=wlm-cron)
Meta Attrs: target-role=Stopped
Operations: monitor interval=30s on-fail=restart timeout=300s (wlm-cron-monito r-interval-30s)
start interval=0s on-fail=restart timeout=300s (wlm-cron-start-int erval-0s)
stop interval=0s timeout=300s (wlm-cron-stop-interval-0s)
[root@TVM2 ~]# ps -ef | grep -i workloadmgr-cron
root 15379 14383 0 08:27 pts/0 00:00:00 grep --color=auto -i workloadmgr -cron
Steps to upgrade Trilio having Internet access
Download the
hf_upgrade.shscript on all T4O nodes where the upgrade is to be done.
cd /opt/
wget https://raw.githubusercontent.com/trilioData/triliovault-cfg-scripts/4.3.2/TVOAppliance/hf_upgrade.sh
chmod +x hf_upgrade.shYou can check the usage of this script here.
Run the following command to download and Install the upgraded packages
./hf_upgrade.sh --all
OR
./hf_upgrade.sh -a
Run the following command to enable the wlm-cron service
pcs resource enable wlm-cron
Steps to upgrade Trilio having No Internet access
Here the packages need to be downloaded on a separate host which has internet access.
The script ./hf_upgrade.sh can be used with the below-mentioned option to download the required package
Download the
hf_upgrade.shscript and use the script to download the upgraded packages. You can check the usage of this script here.\cd /<tvo_packages_download_path>/ wget https://raw.githubusercontent.com/trilioData/triliovault-cfg-scripts/4.3.2/TVOAppliance/hf_upgrade.sh chmod +x hf_upgrade.sh # Download the upgraded packages ./hf_upgrade.sh --downloadonlyCopy the downloaded
4.3-offlinePkgs.tar.gzpackage andhf_upgrade.shscript to all the Trilio nodes\scp <tvo_packages_download_path>/* root@<TrilioVault_node_IP>:/<path_to_upgrade_package>/Now access the appliance VMs and install the copied offline packages on each of the Trilio appliances by running the below-mentioned command:\
cd <path_to_upgrade_package>/ ./hf_upgrade.sh --installonlyOnce the installation is done, run the following command to enable the wlm-cron service\
pcs resource enable wlm-cron
Post upgrade steps
Verify all wlm services on all Trilio nodes
systemctl status tvault-config wlm-workloads wlm-api wlm-scheduler pcs status (on primary node) systemctl status wlm-cron (on primary node) systemctl status tvault-object-store (only if Trilio configured with S3 backend storage)Make sure all wlm services are up and running from python version 3.8.x
Check the mount point using “df -h” command
Grafana dashboard shows the correct wlm service status on all T4O nodes.
Enable Global Job Scheduler
Additional check for wlm-cron on the primary node
ps -ef | grep workloadmgr-cron | grep -v grep
# Above command should show only 2 processes running; sample below
[root@tvm6 ~]# ps -ef | grep workloadmgr-cron | grep -v grep
nova 8841 1 2 Jul28 ? 00:40:44 /home/stack/myansible/bin/python3 /home/stack/myansible/bin/workloadmgr-cron --config-file=/etc/workloadmgr/workloadmgr.conf
nova 8898 8841 0 Jul28 ? 00:07:03 /home/stack/myansible/bin/python3 /home/stack/myansible/bin/workloadmgr-cron --config-file=/etc/workloadmgr/workloadmgr.conf[RHOSP, TripleO and Kolla only] Verify nova UID/GID for nova user on the Appliance
Red Hat OpenStack, TripleO, and Kolla Ansible Openstack are using the nova UID/GID of 42436 inside their containers instead of 162:162 which is the standard in other Openstack environments.
Please verify that the nova UID/GID on the Trilio Appliance is still 42436, to do so follow the below document provided here:
Last updated
Was this helpful?