Search…
TVO-4.2
Deployment Guide
TrilioVault Appliance Administration Guide
Admin Guide
Troubleshooting
Upgrading on Canonical Openstack
Upgrading TrilioVault 4.0 to 4.1 using JuJu Charms
For the major upgrade from 4.0 to 4.1 use the JuJu charms upgrade path.
The charms will always install the latest version available of TVO 4.1.
This will only work when upgrading from 4.0 to 4.1.
The following charms exist:
The documentation of the charms can be found here:
TrilioVault Data Protection — charm-deployment-guide 0.0.1.dev426 documentation
The following steps have been tested and verified within Trilio environments.
There have been cases where these steps updated all packages inside the LXC containers, leading to failures in basic OpenStack services.
It is recommended to run each of these steps in dry-run first.
When any other packages but TrilioVault packages are getting updated, stop the upgrade procedure and contact your TrilioVault customer success manager.
Upgrading the TrilioVault packages
TrilioVault is releasing hotfixes, which require updating the packages inside the containers.
These hotfixes can not be installed using the Juju charms as they don't require an update to the charms.
Generic Pre-requisites
- 1.Either 4.1 GA, 4.2 GA or any hotfix patch against 4.1/4.2 should be already deployed for performing upgrades mentioned in the current document.
- 2.No snapshot OR restore to be running.
- 3.Global job scheduler should be disabled.
- 4.wlm-cron should be disabled ( Following commands are to be run on MAAS node)
- 1.If trilio-wlm is HA enabled, set the cluster configuration to maintenance mode ( this command will fail for single node deployment)
juju run [-m <model>] --unit trilio-wlm/leader "sudo crm configure property maintenance-mode=true" - 2.Stop wlm-cron service
juju run [-m <model>] --application trilio-wlm "sudo systemctl stop wlm-cron" - 3.Ensure that no stale wlm-cron processes are there
juju run [-m <model>] --application trilio-wlm "sudo ps -ef | grep [w]orkloadmgr-cron"If any stale process is found, that needs to be killed manually.
- 5.The mentioned gemfury repository should be accessible from trilio units.
Upgrade package on trilio units
The deployed TrilioVault version is controlled by the
triliovault-pkg-source charm configuration option.
For each trilio charm it should be pointing to below gemfury repository.
This can be checked via juju [-m ] config triliovault-pkg-source command output.1
deb [trusted=yes] https://apt.fury.io/triliodata-4-2/ /
Copied!
This is the preferred, recommended and tested method to update the packages is through the Juju command line.
Run below commands form MASS node
On Ubuntu Bionic environments
1
juju [-m <model>] run --application trilio-data-mover 'sudo apt-get update && sudo apt-get install -y --only-upgrade -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" tvault-contego s3-fuse-plugin'
2
juju [-m <model>] run --application trilio-dm-api 'sudo apt-get update && sudo apt-get install -y --only-upgrade -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" python3-dmapi'
3
juju [-m <model>] run --application trilio-horizon-plugin 'sudo apt-get update && sudo apt-get install -y --only-upgrade -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" tvault-horizon-plugin python-workloadmgrclient'
4
juju [-m <model>] run --application trilio-wlm 'sudo apt-get update && sudo apt-get install -y --only-upgrade -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" workloadmgr python3-workloadmgrclient python3-contegoclient'
Copied!
On other (not Ubuntu Bionic) environments
1
juju [-m <model>] run --application trilio-wlm 'sudo apt-get update && sudo apt-get install -y --only-upgrade -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" workloadmgr python3-workloadmgrclient python3-contegoclient'
2
juju [-m <model>] run --application trilio-horizon-plugin 'sudo apt-get update && sudo apt-get install -y --only-upgrade -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" python3-tvault-horizon-plugin python3-workloadmgrclient'
3
juju [-m <model>] run --application trilio-dm-api 'sudo apt-get update && sudo apt-get install -y --only-upgrade -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" python3-dmapi'
4
juju [-m <model>] run --application trilio-data-mover 'sudo apt-get update && sudo apt-get install -y --only-upgrade -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" python3-tvault-contego python3-s3-fuse-plugin'
Copied!
Check the trilio units status in
juju status [-m ] | grep trilio output. All the trilio units will be with new package.1
trilio-data-mover <package version> active 3 trilio-data-mover jujucharms 8 ubuntu
2
trilio-dm-api <package version> active 1 trilio-dm-api jujucharms 5 ubuntu
3
trilio-horizon-plugin <package version> active 1 trilio-horizon-plugin jujucharms 4 ubuntu
4
trilio-wlm <package version> active 3 trilio-wlm jujucharms 7 ubuntu
Copied!
Update the DB schema
Run the below command to update the schema
1
juju run [-m <model>] --unit trilio-wlm/leader "alembic -c /etc/workloadmgr/alembic.ini upgrade heads"
Copied!
Check the schema head with below command. It should point to latest schema head.
1
juju run [-m <model>] --unit trilio-wlm/leader "alembic -c /etc/workloadmgr/alembic.ini current"
Copied!
Enable mount-bind for NFS
TVO 4.2 is changing how the mount point is calculated. It is required to setup a mount bind to make TVO 4.1 or older backups available.
Post-Upgrade steps
- 1.If the trilio-wlm nodes are HA enabled:
- 1.Make sure the wlm-cron services are down after the pkg upgrade. Run the following command for the same:
juju run [-m <model>] --application trilio-wlm "sudo systemctl stop wlm-cron" - 2.Unset the cluster maintenance mode
juju run [-m <model>] --unit trilio-wlm/leader "sudo crm configure property maintenance-mode=false"
- 2.Make sure the wlm-cron service up and running on any one node.
juju run [-m <model>] --application trilio-wlm "sudo systemctl status wlm-cron" - 3.Set the Global Job Scheduler to the original state.
Troubleshooting
If any trilio unit get into error state with message
hook failed: "update-status" Follow below steps
1
Login to trilio unit and run "sudo dpkg --configure -a"
2
It will ask for user input, hit enter and log out from the unit.
3
From mass node run command "juju resolve <trilio unit name>"
Copied!
Last modified 2mo ago