Ubuntu/Debian based Openstack enviroments

Generic Pre-requisites

1. Please ensure the following points before starting the upgrade process:

   1. No snapshot OR restore to be running.

   2. Global job scheduler should be disabled.

2. The mentioned gemfury repository should be accessible from TVault VM.

Rolling upgrade on Openstack Ansible

Repo Updates

Add gemfury repo on each dmapi, horizon containers & compute node(s) to get updated packages.

Create a file /etc/apt/sources.list.d/fury.list and add below line in it.

deb [trusted=yes] https://apt.fury.io/triliodata-4-0/ /

Use the below commands to get a list of updated packages available on the configured repositories.

apt-get update
apt list --upgradable 

Upgrade tvault-datamover-api package

• Login to dmapi container from the controller node using the below command.

lxc-attach -n controller_dmapi

• Take a backup of the following file on each dmapi container(s).

tar -czvf dmapi_config.tar.gz /etc/dmapi

• Add the gemfury repo and upgrade the dmapi package using the below command. Select the appropriate package depending on the python version used.

apt install python3-dmapi --upgrade

• Restore the backed-up config files

tar -xzvf dmapi_config.tar.gz -C /

• Now restart and check the service tvault-datamover-api on controller.

systemctl restart tvault-datamover-api
systemctl status tvault-datamover-api

Upgrade Horizon plugin

• For the Horizon plugin upgrade, we need to upgrade below two packages.

python3-tvault-horizon-plugin python3-workloadmgrclient

• Login to the horizon container from the controller node using the below command.

  lxc-attach -n controller_horizon_container-ead7cc60

• Add the gemfury repo and upgrade the tvault-horizon-plugin & workloadmgrclient packages using the below command. Select the appropriate package depending on the python version used.

apt install python3-tvault-horizon-plugin --upgrade
apt install python3-workloadmgrclient --upgrade

• Restart the apache2 service and verify the workloadmgrclient version using the below commands.

systemctl restart apache2
workloadmgr --version

Upgrade the tvault-contego package

NFS Storage Backend

• Take a backup of following file on each compute node(s) for nfs storage backend.

tar -czvf  contego_config.tar.gz /etc/tvault-contego/ 

• Upgrade the tvault-contego package using below command.Select the appropriate package depending on python version used.

apt install python3-tvault-contego --upgrade

• Restore the backed-up config files

 tar -xzvf  contego_config.tar.gz -C /

• Now restart and check the service tvault-contego on compute node(s).

systemctl restart tvault-contego
systemctl status tvault-contego

Please make sure all configuration files are unchanged.

S3 Storage Backend

• Take a backup of following file on each compute node(s) for s3 storage backend.

tar -czvf  contego_config.tar.gz /etc/tvault-contego/ /etc/tvault-object-store/

• Upgrade the tvault-contego package using below command. Select the appropriate package depending on the python version used.

apt install python3-tvault-contego python3-s3-fuse-plugin --upgrade

Note*:- If you get prompt while installing, choose the default selection.

• Restore the backed-up config files

 tar -xzvf  contego_config.tar.gz -C /

• Now restart and check the service tvault-contego on compute node(s).

systemctl restart tvault-object-store
systemctl restart tvault-contego 
systemctl status tvault-object-store
systemctl status tvault-contego

Please make sure all configuration files are unchanged.

Rolling upgrade on Kolla Openstack

Upgrade tvault-datamover-api package (trilio_datamover_api Container)

Container trilio_datamover_api needs to be redeployed. Follow below steps on all controller nodes :

• Backup Existing conf files/folders

Copy folders /etc/kolla/nova-api & /etc/kolla/trilio-datamover-api/ to some local location.
Eg. cp -r /etc/kolla/nova-api $HOME/4.0-BK/

• Stop and Remove existing container trilio_datamover_api

docker stop trilio_datamover_api
docker rm trilio_datamover_api

• Pull trilio_datamover_api container image:

## Login to docker using credentials: triliodocker/triliopassword
docker login docker.io

## Pull Trilio Datamover Api container using following command. You need to edit OS_PLATFORM, TVAULT_VERSION,
 OPENSTACK_RELEASE_NAME.

<OS_PLATFORM> could be ubuntu/centos.
<OPENSTACK_RELEASE_NAME> could would be train
<TVAULT_VERSION>    : For exact GA build version check section 1. i) Plan for deployment

docker pull docker.io/trilio/<OS_PLATFORM>-source-trilio-datamover-api:<TVAULT_VERSION>-<OPENSTACK_RELEASE_NAME>

Example command for train openstack on ubuntu platform with triliovault 4.0 release:
docker pull docker.io/trilio/ubuntu-source-trilio-datamover-api:4.0.106-train

• Run datamover api container.

You need to edit OS_PLATFORM, TVAULT_VERSION, OPENSTACK_RELEASE_NAME. 
<OS_PLATFORM> could be ubuntu/centos.
<OPENSTACK_RELEASE_NAME> would be train
<TVAULT_VERSION> : For exact TVAULT build version check section 1. i) Plan for deployment

docker run --network host --name trilio_datamover_api -d --restart always -v /etc/kolla/trilio-datamover-api/nova.conf:/etc/nova/nova.conf \
-v /etc/kolla/trilio-datamover-api/dmapi.conf:/etc/dmapi/dmapi.conf \
-v /var/log/kolla/:/var/log/kolla/ \
trilio/<OS_PLATFORM>-source-trilio-datamover-api:<TVAULT_VERSION>-<OPENSTACK_RELEASE_NAME> /var/lib/kolla/venv/bin/python /usr/bin/dmapi-api 

• Verify deployment

Make sure 'trilio_datamover_api' named container running well and not in 'restarting' state on all controller nodes. 
'docker ps' command can be used for the same.

# docker ps | grep trilio_datamover_api
e603ea78094f  trilio/ubuntu-source-trilio-datamover-api:4.0.106-train   "dumb-init --single-…"   5 minutes ago Up 5 minutes
trilio_datamover_api

Upgrade Horizon Plugin Package

• Login to horizon container (on controller node)

docker exec -it horizon /bin/bash

• Add gemfury repo on horizon to get updated packages.

  Create a file /etc/apt/sources.list.d/fury.list and add below line in it.

  deb [trusted=yes] https://apt.fury.io/triliodata-4-0/ /

  Use below commands to get list of updated packages available on repo's.

  apt-get update
  apt list --upgradable

• For Horizon plugin upgrade, following packages need to be upgraded.

python3-tvault-horizon-plugin python3-workloadmgrclient

• Run following commands to upgrade the tvault-horizon-plugin & workloadmgrclient packages. Select the appropriate package depending on python version used.

dpkg -l | grep -i tvault
  ii  python3-tvault-horizon-plugin  4.0.92    all   tVault plugin for  OpenStack Horizon Dashboard
  ii  python3-workloadmgrclient      4.0.92   all  tVault Workload Manager Client

apt install python3-tvault-horizon-plugin --upgrade
apt install python3-workloadmgrclient --upgrade

dpkg -l | grep -i tvault
   ii  python3-tvault-horizon-plugin  4.0.115                             all          tVault plugin for  OpenStack Horizon Dashboard
   ii  python3-workloadmgrclient      4.0.115                             all          tVault Workload Manager Client

• Restart docker container (from controller node) and verify the workloadmgrclient version (inside horizon container).

docker restart horizon
docker exec -it horizon /bin/bash
workloadmgr --version

Upgrade Tvault-Contego (trilio_datamover Container)

Container trilio_datamover needs to be redeployed. Follow below steps on all Compute nodes :

• Backup Existing conf files/folders

Copy folder /etc/kolla/trilio-datamover to some local location.
Eg. cp -r /etc/kolla/trilio-datamover $HOME/4.0-BK/

• Stop & remove existing trilio_datamover container

docker stop trilio_datamover
docker rm trilio_datamover

• Pull Trilio Datamover container image using the following command:

## Login to docker using credentials: triliodocker/triliopassword
docker login docker.io

## Pull Trilio Datamover Api container using following command. You need to edit OS_PLATFORM, TVAULT_VERSION,
OPENSTACK_RELEASE_NAME.

<OS_PLATFORM> could be ubuntu/centos.
<OPENSTACK_RELEASE_NAME> would be train
<TVAULT_VERSION> : For exact TVAULT build version check section 1. i) Plan for deployment

docker pull docker.io/trilio/<OS_PLATFORM>-source-trilio-datamover:<TVAULT_VERSION>-<OPENSTACK_RELEASE_NAME>

Example command for train openstack on ubuntu platform with Trilio 4.0 release:
docker pull docker.io/trilio/ubuntu-source-trilio-datamover:4.0.106-train

• Run datamover container.

-- In following docker run command you need to edit OS_PLATFORM, TVAULT_VERSION,
OPENSTACK_RELEASE_NAME.

<OS_PLATFORM> could be ubuntu/centos.
<OPENSTACK_RELEASE_NAME> would be train
<TVAULT_VERSION> : For exact TVAULT build version check section 1. i) Plan for deployment

Note: If your cloud does not use 'ceph' storage for nova/cinder, remove '/etc/ceph' volume mount option 
from below commands.

##If backup storage type is 'NFS'
docker run --privileged --network host --name trilio_datamover -d \
--restart always \
-v /etc/kolla/trilio-datamover/tvault-contego.conf:/etc/tvault-contego/tvault-contego.conf \
-v /etc/kolla/trilio-datamover/nova.conf:/etc/nova/nova.conf \
-v /dev:/dev:rw \
-v /etc/ceph:/etc/ceph:ro \
-v nova_compute:/var/lib/nova/:rw \
-v /var/log/kolla/:/var/log/kolla/ \
-v iscsi_info:/etc/iscsi:rw -v /var/run/libvirt \
trilio/<OS_PLATFORM>-source-trilio-datamover:<TVAULT_VERSION>-<OPENSTACK_RELEASE_NAME> /opt/tvault/start_datamover_nfs

• Verify Deployment of trilio_datamover

Make sure 'trilio_datamover' named container running well and not in 'restarting' state on all compute nodes.
   'docker ps' command can be used for the same.
# docker ps | grep trilio_datamover
3e62813e5ade        trilio/ubuntu-source-trilio-datamover:4.0.115-train   "dumb-init --single-…"   7 hours ago         Up 7 hours                              trilio_datamover