# Upgrading on RHOSP

## 0] Pre-requisites

Please ensure the following points are met before starting the upgrade process:

* No Snapshot or Restore is running
* Global job scheduler is disabled
* wlm-cron is disabled on the Trilio Appliance

#### Deactivating the wlm-cron service

The following sets of commands will disable the wlm-cron service and verify that it is has been completely shut-down.

```
[root@TVM2 ~]# pcs resource disable wlm-cron
[root@TVM2 ~]# systemctl status wlm-cron
● wlm-cron.service - workload's scheduler cron service
   Loaded: loaded (/etc/systemd/system/wlm-cron.service; disabled; vendor preset                                                                                                                                                                                                                                                                                                                                                           : disabled)
   Active: inactive (dead)

Jun 11 08:27:06 TVM2 workloadmgr-cron[11115]: 11-06-2021 08:27:06 - INFO - 1...t
Jun 11 08:27:07 TVM2 workloadmgr-cron[11115]: 140686268624368 Child 11389 ki...5
Jun 11 08:27:07 TVM2 workloadmgr-cron[11115]: 11-06-2021 08:27:07 - INFO - 1...5
Jun 11 08:27:07 TVM2 workloadmgr-cron[11115]: Shutting down thread pool
Jun 11 08:27:07 TVM2 workloadmgr-cron[11115]: 11-06-2021 08:27:07 - INFO - S...l
Jun 11 08:27:07 TVM2 workloadmgr-cron[11115]: Stopping the threads
Jun 11 08:27:07 TVM2 workloadmgr-cron[11115]: 11-06-2021 08:27:07 - INFO - S...s
Jun 11 08:27:07 TVM2 workloadmgr-cron[11115]: All threads are stopped succes...y
Jun 11 08:27:07 TVM2 workloadmgr-cron[11115]: 11-06-2021 08:27:07 - INFO - A...y
Jun 11 08:27:09 TVM2 systemd[1]: Stopped workload's scheduler cron service.
Hint: Some lines were ellipsized, use -l to show in full.
[root@TVM2 ~]# pcs resource show wlm-cron
 Resource: wlm-cron (class=systemd type=wlm-cron)
  Meta Attrs: target-role=Stopped
  Operations: monitor interval=30s on-fail=restart timeout=300s (wlm-cron-monito                                                                                                                                                                                                                                                                                                                                                           r-interval-30s)
              start interval=0s on-fail=restart timeout=300s (wlm-cron-start-int                                                                                                                                                                                                                                                                                                                                                           erval-0s)
              stop interval=0s timeout=300s (wlm-cron-stop-interval-0s)
[root@TVM2 ~]# ps -ef | grep -i workloadmgr-cron
root     15379 14383  0 08:27 pts/0    00:00:00 grep --color=auto -i workloadmgr                                                                                                                                                                                                                                                                                                                                                           -cron

```

## 1] \[On Undercloud node] Clone latest Trilio repository and upload Trilio puppet module

{% hint style="warning" %}
All commands need to be run as user 'stack' on undercloud node
{% endhint %}

### 1.1] Clone Trilio cfg scripts repository

```
cd /home/stack
mv triliovault-cfg-scripts triliovault-cfg-scripts-old
git clone -b TVO/4.2.8 https://github.com/trilioData/triliovault-cfg-scripts.git
cd triliovault-cfg-scripts/redhat-director-scripts/<RHOSP_RELEASE_DIRECTORY>/
```

Separate directories are created as per Redhat OpenStack release under 'triliovault-cfg-scripts/redhat-director-scripts/' directory. Use all scripts/templates from respective directory. For ex, if your RHOSP release is 13, then use scripts/templates from 'triliovault-cfg-scripts/redhat-director-scripts/rhosp13' directory only.

{% hint style="info" %}
Available RHOSP\_RELEASE\_\_\_DIRECTORY values are:

rhosp13\
rhosp16.1\
rhosp16.2\
rhosp17.0
{% endhint %}

### 1.2] If backup target type is 'Ceph based S3' with SSL:

If your backup target is ceph S3 with SSL and SSL certificates are self signed or authorized by private CA, then user needs to provide CA chain certificate to validate the SSL requests. For that, user needs to rename his ca chain cert file to 's3-cert.pem' and copy it into the puppet directory of the right release.

```
cp s3-cert.pem /home/stack/triliovault-cfg-scripts/redhat-director-scripts/<RHOSP_RELEASE_DIRECTORY>/puppet/trilio/files/
```

## 2] Upload Trilio puppet module

```
cd /home/stack/triliovault-cfg-scripts/redhat-director-scripts/<RHOSP_RELEASE_DIRECTORY>/scripts/
./upload_puppet_module.sh

## Output of above command looks like following for RHOSP13, RHOSP16.1 and RHOSP16.2
Creating tarball...
Tarball created.
Creating heat environment file: /home/stack/.tripleo/environments/puppet-modules-url.yaml
Uploading file to swift: /tmp/puppet-modules-8Qjya2X/puppet-modules.tar.gz
+-----------------------+---------------------+----------------------------------+
| object                | container           | etag                             |
+-----------------------+---------------------+----------------------------------+
| puppet-modules.tar.gz | overcloud-artifacts | 368951f6a4d39cfe53b5781797b133ad |
+-----------------------+---------------------+----------------------------------+

## Output of above command looks like following for RHOSP17.0
Creating tarball...
Tarball created.
renamed '/tmp/puppet-modules-P3duCg9/puppet-modules.tar.gz' -> '/var/lib/tripleo/artifacts/overcloud-artifacts/puppet-modules.tar.gz'
Creating heat environment file: /home/stack/.tripleo/environments/puppet-modules-url.yaml

## Above command creates following file.
ls -ll /home/stack/.tripleo/environments/puppet-modules-url.yaml

```

## 3] Update overcloud roles data file to include Trilio services

Trilio has two services as explained below.\
You need to add these two services to your roles\_data.yaml.\
If you do not have customized roles\_data file, you can find your default roles\_data.yaml file at `/usr/share/openstack-tripleo-heat-templates/roles_data.yaml` on undercloud.

You need to find that role\_data file and edit it to add the following Trilio services.

**i) Trilio Datamover Api Service:**

Service Entry in roles\_data yaml: `OS::TripleO::Services::TrilioDatamoverApi`

{% hint style="info" %}
This service needs to be co-located with database and keystone services. That said, you need to add this service on the same role as of keystone and database service.
{% endhint %}

Typically this service should be deployed on controller nodes where keystone and database runs.\
If you are using RHOSP's pre-defined roles, you need to add`OS::TripleO::Services::TrilioDatamoverApi`service to `Controller role`.

**ii) Trilio Datamover Service:**\
Service Entry in roles\_data yaml: `OS::TripleO::Services::TrilioDatamover`\
This service should be deployed on role where `nova-compute` service is running.

If you are using RHOSP's pre-defined roles, you need to add our `OS::TripleO::Services::TrilioDatamover` service to `Compute role`.

If you have defined your custom roles, then you need to identify the role name where in 'nova-compute' service is running and then you need to add 'OS::TripleO::Services::TrilioDatamover' service to that role.

**iii) Trilio Horizon Service:**

This service needs to share the same role as the OpenStack Horizon server.\
In the case of the pre-defined roles will the Horizon service run on the role Controller.\
\
Add the following to the identified role `OS::TripleO::Services::TrilioHorizon`

## 4] Prepare latest Trilio container images

{% hint style="warning" %}
All commands need to be run as user 'stack'
{% endhint %}

Trilio containers are pushed to 'RedHat Container Registry'. Registry URL is '[registry.connect.redhat.com](http://registry.connect.redhat.com/)'.\
The Trilio container URLs are as follows:

{% hint style="info" %}
Refer to the word **\<HOTFIX-TAG-VERSION>** as ***4.2.8** in the below sections*
{% endhint %}

### 4.1] available container images

#### RHOSP 13

```
Trilio Datamover container:       registry.connect.redhat.com/trilio/trilio-datamover:<HOTFIX-TAG-VERSION>-rhosp13
Trilio Datamover Api Container:   registry.connect.redhat.com/trilio/trilio-datamover-api:<HOTFIX-TAG-VERSION>-rhosp13
Trilio horizon plugin:            registry.connect.redhat.com/trilio/trilio-horizon-plugin:<HOTFIX-TAG-VERSION>-rhosp13
```

#### RHOSP 16.1

```
Trilio Datamover container:       registry.connect.redhat.com/trilio/trilio-datamover:<HOTFIX-TAG-VERSION>-rhosp16.1
Trilio Datamover Api Container:   registry.connect.redhat.com/trilio/trilio-datamover-api:<HOTFIX-TAG-VERSION>-rhosp16.1
Trilio horizon plugin:            registry.connect.redhat.com/trilio/trilio-horizon-plugin:<HOTFIX-TAG-VERSION>-rhosp16.1
```

#### RHOSP 16.2

```
Trilio Datamover container:       registry.connect.redhat.com/trilio/trilio-datamover:<HOTFIX-TAG-VERSION>-rhosp16.2
Trilio Datamover Api Container:   registry.connect.redhat.com/trilio/trilio-datamover-api:<HOTFIX-TAG-VERSION>-rhosp16.2
Trilio horizon plugin:            registry.connect.redhat.com/trilio/trilio-horizon-plugin:<HOTFIX-TAG-VERSION>-rhosp16.2
```

#### RHOSP 17.0

```
Trilio Datamover container:       registry.connect.redhat.com/trilio/trilio-datamover:<HOTFIX-TAG-VERSION>-rhosp17.0
Trilio Datamover Api Container:   registry.connect.redhat.com/trilio/trilio-datamover-api:<HOTFIX-TAG-VERSION>-rhosp17.0
Trilio horizon plugin:            registry.connect.redhat.com/trilio/trilio-horizon-plugin:<HOTFIX-TAG-VERSION>-rhosp17.0
```

There are three registry methods available in RedHat Openstack Platform.

1. Remote Registry
2. Local Registry
3. Satellite Server

### 4.2] Remote Registry

{% hint style="info" %}
Please refer to the [following overview](#4-1-available-container-images) to see which containers are available.
{% endhint %}

Follow this section when 'Remote Registry' is used.

For this method it is not necessary to pull the containers in advance.\
It is only necessary to populate the trilio\_env.yaml file with the Trilio container URLs from Redhat registry.

Populate the trilio\_env.yaml with container URLs for:

* Trilio Datamover container
* Trilio Datamover api container
* Trilio Horizon Plugin

{% hint style="info" %}
trilio\_env.yaml will be available in\
\_\_`triliovault-cfg-scripts/redhat-director-scripts/<RHOSP_RELEASE_DIRECTORY>/environments`
{% endhint %}

#### Example

```
# For RHOSP13
$ grep 'Image' trilio_env.yaml
   DockerTrilioDatamoverImage: registry.connect.redhat.com/trilio/trilio-datamover:<HOTFIX-TAG-VERSION>-rhosp13
   DockerTrilioDmApiImage: registry.connect.redhat.com/trilio/trilio-datamover-api:<HOTFIX-TAG-VERSION>-rhosp13
   DockerHorizonImage: registry.connect.redhat.com/trilio/trilio-horizon-plugin:<HOTFIX-TAG-VERSION>-rhosp13

# For RHOSP16.1
$ grep 'Image' trilio_env.yaml
   DockerTrilioDatamoverImage: registry.connect.redhat.com/trilio/trilio-datamover:<HOTFIX-TAG-VERSION>-rhosp16.1
   DockerTrilioDmApiImage: registry.connect.redhat.com/trilio/trilio-datamover-api:<HOTFIX-TAG-VERSION>-rhosp16.1
   ContainerHorizonImage: registry.connect.redhat.com/trilio/trilio-horizon-plugin:<HOTFIX-TAG-VERSION>-rhosp16.1
   
# For RHOSP16.2
$ grep 'Image' trilio_env.yaml
   DockerTrilioDatamoverImage: registry.connect.redhat.com/trilio/trilio-datamover:<HOTFIX-TAG-VERSION>-rhosp16.2
   DockerTrilioDmApiImage: registry.connect.redhat.com/trilio/trilio-datamover-api:<HOTFIX-TAG-VERSION>-rhosp16.2
   ContainerHorizonImage: registry.connect.redhat.com/trilio/trilio-horizon-plugin:<HOTFIX-TAG-VERSION>-rhosp16.2

# For RHOSP17.0
$ grep 'Image' trilio_env.yaml
   DockerTrilioDatamoverImage: registry.connect.redhat.com/trilio/trilio-datamover:<HOTFIX-TAG-VERSION>-rhosp17.0
   DockerTrilioDmApiImage: registry.connect.redhat.com/trilio/trilio-datamover-api:<HOTFIX-TAG-VERSION>-rhosp17.0
   ContainerHorizonImage: registry.connect.redhat.com/trilio/trilio-horizon-plugin:<HOTFIX-TAG-VERSION>-rhosp17.0

```

### 4.2] Local Registry

{% hint style="info" %}
Please refer to the [this overview](#4-1-available-container-images) to see which containers are available.
{% endhint %}

Follow this section when 'local registry' is used on the undercloud.

In this case it is necessary to push the Trilio containers to the undercloud registry.\
Trilio provides shell scripts which will pull the containers from 'registry.connect.redhat.com' and push them to the undercloud and updates the trilio\_env.yaml.

#### RHOSP13 example

```
cd /home/stack/triliovault-cfg-scripts/redhat-director-scripts/<RHOSP_RELEASE_DIRECTORY>/scripts/

./prepare_trilio_images.sh <undercloud_ip> <container_tag>

# Example:
./prepare_trilio_images.sh 192.168.13.34 <HOTFIX-TAG-VERSION>-rhosp13

## Verify changes
# For RHOSP13
$ grep '4.2.6-rhosp13' ../environments/trilio_env.yaml
   DockerTrilioDatamoverImage: 172.25.2.2:8787/trilio/trilio-datamover:<HOTFIX-TAG-VERSION>-rhosp13
   DockerTrilioDmApiImage: 172.25.2.2:8787/trilio/trilio-datamover-api:<HOTFIX-TAG-VERSION>-rhosp13
   DockerHorizonImage: 172.25.2.2:8787/trilio/trilio-horizon-plugin:<HOTFIX-TAG-VERSION>-rhosp13
```

#### RHOSP 16.1, RHOSP16.2 and RHOSP17.0 example

```
cd /home/stack/triliovault-cfg-scripts/redhat-director-scripts/<RHOSP_RELEASE_DIRECTORY>/scripts/

sudo ./prepare_trilio_images.sh <UNDERCLOUD_REGISTRY_HOSTNAME> <CONTAINER_TAG> 

## Run following command to find 'UNDERCLOUD_REGISTRY_HOSTNAME'. 
-- In the below example 'trilio-undercloud.ctlplane.localdomain' is <UNDERCLOUD_REGISTRY_HOSTNAME>
$ openstack tripleo container image list | grep keystone
| docker://trilio-undercloud.ctlplane.localdomain:8787/rhosp-rhel8/openstack-keystone:16.0-82                       |
| docker://trilio-undercloud.ctlplane.localdomain:8787/rhosp-rhel8/openstack-barbican-keystone-listener:16.0-84   

## 'CONTAINER_TAG' format for RHOSP16.1: <<HOTFIX-TAG-VERSION>>-rhosp16.1
## 'CONTAINER_TAG' format for RHOSP16.2: <<HOTFIX-TAG-VERSION>>-rhosp16.2
## 'CONTAINER_TAG' format for RHOSP17.0: <<HOTFIX-TAG-VERSION>>-rhosp17.0

## Example
sudo ./prepare_trilio_images.sh trilio-undercloud.ctlplane.localdomain <HOTFIX-TAG-VERSION>-rhosp16.1
```

The changes can be verified using the following commands.

```
(undercloud) [stack@undercloud redhat-director-scripts/<RHOSP_RELEASE_DIRECTORY>]$ openstack tripleo container image list | grep trilio
| docker://undercloud.ctlplane.localdomain:8787/trilio/trilio-datamover:<HOTFIX-TAG-VERSION>-rhosp16.1                       |                        |
| docker://undercloud.ctlplane.localdomain:8787/trilio/trilio-datamover-api:<HOTFIX-TAG-VERSION>-rhosp16.1                   |                   |
| docker://undercloud.ctlplane.localdomain:8787/trilio/trilio-horizon-plugin:<HOTFIX-TAG-VERSION>-rhosp16.1                  |

-----------------------------------------------------------------------------------------------------

(undercloud) [stack@undercloud redhat-director-scripts]$ grep 'Image' ../environments/trilio_env.yaml
   DockerTrilioDatamoverImage: undercloud.ctlplane.localdomain:8787/trilio/trilio-datamover:<HOTFIX-TAG-VERSION>-rhosp16.1
   DockerTrilioDmApiImage: undercloud.ctlplane.localdomain:8787/trilio/trilio-datamover-api:<HOTFIX-TAG-VERSION>-rhosp16.1
   ContainerHorizonImage: undercloud.ctlplane.localdomain:8787/trilio/trilio-horizon-plugin:<HOTFIX-TAG-VERSION>-rhosp16.1
```

### 4.3] Red Hat Satellite Server

{% hint style="info" %}
Please refer to the [following overview](#4-1-available-container-images) to see which containers are available.
{% endhint %}

Follow this section when a Satellite Server is used for the container registry.

Pull the Trilio containers on the Red Hat Satellite using the given [Red Hat registry URLs.](https://docs.trilio.io/openstack/tvo-4.2/configuring-and-installing-triliovault/installing-on-rhosp#prepare-for-deployment)

Populate the trilio\_env.yaml with container urls.

#### RHOSP 13 example

```
$ grep '<HOTFIX-TAG-VERSION>-rhosp13' ../environments/trilio_env.yaml
   DockerTrilioDatamoverImage: <SATELLITE_REGISTRY_URL>/trilio/trilio-datamover:<HOTFIX-TAG-VERSION>-rhosp13
   DockerTrilioDmApiImage: <SATELLITE_REGISTRY_URL>/trilio/trilio-datamover-api:<HOTFIX-TAG-VERSION>-rhosp13
   DockerHorizonImage: <SATELLITE_REGISTRY_URL>/trilio/trilio-horizon-plugin:<HOTFIX-TAG-VERSION>-rhosp13
```

#### RHOSP 16.1, RHOSP 16.2 and RHOSP 17.0

```
## For RHOSP16.1
$ grep 'Image' trilio_env.yaml
   DockerTrilioDatamoverImage: <SATELLITE_REGISTRY_URL>/trilio/trilio-datamover:<HOTFIX-TAG-VERSION>-rhosp16.1
   DockerTrilioDmApiImage: <SATELLITE_REGISTRY_URL>/trilio/trilio-datamover-api:<HOTFIX-TAG-VERSION>-rhosp16.1
   ContainerHorizonImage: <SATELLITE_REGISTRY_URL>/trilio/trilio-horizon-plugin:<HOTFIX-TAG-VERSION>-rhosp16.1

## For RHOSP16.2
$ grep 'Image' trilio_env.yaml
   DockerTrilioDatamoverImage: <SATELLITE_REGISTRY_URL>/trilio/trilio-datamover:<HOTFIX-TAG-VERSION>-rhosp16.2
   DockerTrilioDmApiImage: <SATELLITE_REGISTRY_URL>/trilio/trilio-datamover-api:<HOTFIX-TAG-VERSION>-rhosp16.2
   ContainerHorizonImage: <SATELLITE_REGISTRY_URL>/trilio/trilio-horizon-plugin:<HOTFIX-TAG-VERSION>-rhosp16.2

## For RHOSP7.0
$ grep 'Image' trilio_env.yaml
   DockerTrilioDatamoverImage: <SATELLITE_REGISTRY_URL>/trilio/trilio-datamover:<HOTFIX-TAG-VERSION>-rhosp17.0
   DockerTrilioDmApiImage: <SATELLITE_REGISTRY_URL>/trilio/trilio-datamover-api:<HOTFIX-TAG-VERSION>-rhosp17.0
   ContainerHorizonImage: <SATELLITE_REGISTRY_URL>/trilio/trilio-horizon-plugin:<HOTFIX-TAG-VERSION>-rhosp17.0

```

## 5. Verify Trilio environment details

It is recommended to re-populate the backup target details in the freshly downloaded trilio\_env.yaml file.\
This will ensure that parameters that have been added since the last update/installation of Trilio are available and will be filled out too.

Locations of the trilio\_env.yaml:

```
RHOSP13: /home/stack/triliovault-cfg-scripts/redhat-director-scripts/rhosp13/environments/trilio_env.yaml
RHOSP16.1: /home/stack/triliovault-cfg-scripts/redhat-director-scripts/rhosp16.1/environments/trilio_env.yaml
RHOSP16.2: /home/stack/triliovault-cfg-scripts/redhat-director-scripts/rhosp16.2/environments/trilio_env.yaml
RHOSP17.0: /home/stack/triliovault-cfg-scripts/redhat-director-scripts/rhosp17.0/environments/trilio_env.yaml
```

For more details about the trilio\_env.yaml please check [here](https://docs.trilio.io/openstack/tvo-4.2/configuring-and-installing-triliovault/installing-on-rhosp#5-provide-environment-details-in-trilio-env-yaml).

## 6] Configure multi-IP NFS

{% hint style="info" %}
This section is only required when the multi-IP feature for NFS is required.

This feature allows to set the IP to access the NFS Volume per datamover instead of globally.
{% endhint %}

On Undercloud node, change directory

```
cd triliovault-cfg-scripts/common/
```

Edit file '`triliovault_nfs_map_input.yml`' in the current directory and provide compute host and NFS share/ip map.

Get the compute hostnames from the following command. Check ‘Name' column. Use exact hostnames in `'triliovault_nfs_map_input.yml`' file.

{% hint style="info" %}
Run this command on undercloud by sourcing 'stackrc'.
{% endhint %}

```
(undercloud) [stack@ucqa161 ~]$ openstack server list
+--------------------------------------+-------------------------------+--------+----------------------+----------------+---------+
| ID                                   | Name                          | Status | Networks             | Image          | Flavor  |
+--------------------------------------+-------------------------------+--------+----------------------+----------------+---------+
| 8c3d04ae-fcdd-431c-afa6-9a50f3cb2c0d | overcloudtrain1-controller-2  | ACTIVE | ctlplane=172.30.5.18 | overcloud-full | control |
| 103dfd3e-d073-4123-9223-b8cf8c7398fe | overcloudtrain1-controller-0  | ACTIVE | ctlplane=172.30.5.11 | overcloud-full | control |
| a3541849-2e9b-4aa0-9fa9-91e7d24f0149 | overcloudtrain1-controller-1  | ACTIVE | ctlplane=172.30.5.25 | overcloud-full | control |
| 74a9f530-0c7b-49c4-9a1f-87e7eeda91c0 | overcloudtrain1-novacompute-0 | ACTIVE | ctlplane=172.30.5.30 | overcloud-full | compute |
| c1664ac3-7d9c-4a36-b375-0e4ee19e93e4 | overcloudtrain1-novacompute-1 | ACTIVE | ctlplane=172.30.5.15 | overcloud-full | compute |
+--------------------------------------+-------------------------------+--------+----------------------+----------------+---------+
```

Edit input map file and fill all the details. Refer to the [this page](https://docs.trilio.io/openstack/tvo-4.2/deployment/multi-ip-nfs-backup-target-mapping-file-configuration) for details about the structure.

`vi triliovault_nfs_map_input.yml`

Update pyyaml on the undercloud node only

{% hint style="warning" %}
If pip isn't available please install pip on the undercloud.
{% endhint %}

```
## On Python3 env 
sudo pip3 install PyYAML==5.1 3 

## On Python2 env 
sudo pip install PyYAML==5.1
```

Expand the map file to create a one-to-one mapping of the compute nodes and the nfs shares.

```
## On Python3 env 
python3 ./generate_nfs_map.py 
 
## On Python2 env 
python ./generate_nfs_map.py
```

The result will be in file - '`triliovault_nfs_map_output.yml`'

Validate output map file

Open file '`triliovault_nfs_map_output.yml`

`vi triliovault_nfs_map_output.yml`

available in the current directory and validate that all compute nodes are covered with all the necessary NFS shares.

Append this output map file to '`triliovault-cfg-scripts`/`redhat-director-scripts/<RHOSP_RELEASE_DIRECTORY>/environments/trilio_nfs_map.yaml`'

```
grep ':.*:' triliovault_nfs_map_output.yml >> ../redhat-director-scripts/<RHOSP_RELEASE_DIRECTORY>/environments/trilio_nfs_map.yaml
```

Validate the changes in file '`triliovault-cfg-scripts`/`redhat-director-scripts/<RHOSP_RELEASE_DIRECTORY>/environments/trilio_nfs_map.yaml`'

Include the environment file (`trilio_nfs_map.yaml`) in overcloud deploy command with '-e' option as shown below.

```
-e /home/stack/triliovault-cfg-scripts/redhat-director-scripts/<RHOSP_RELEASE_DIRECTORY>/environments/trilio_nfs_map.yaml
```

Ensure that `MultiIPNfsEnabled` is set to true `in` trilio\_env.yaml file and that nfs is used as backup target.

## 7] Update Overcloud Trilio components

Use the following heat environment file and roles data file in overcloud deploy command:

1. `trilio_env.yaml`
2. `roles_data.yaml`
3. Use correct Trilio endpoint map file as per available Keystone endpoint configuration
   1. Instead of `tls-endpoints-public-dns.yaml` file, use `environments/trilio_env_tls_endpoints_public_dns.yaml`
   2. Instead of `tls-endpoints-public-ip.yaml` file, use`environments/trilio_env_tls_endpoints_public_ip.yaml`
   3. Instead of `tls-everywhere-endpoints-dns.yaml` file, use`environments/trilio_env_tls_everywhere_dns.yaml`

To include new environment files use '-e' option and for roles data file use '-r' option.\
An example overcloud deploy command is shown below:

```
openstack overcloud deploy --templates \
  -e /home/stack/templates/node-info.yaml \
  -e /home/stack/templates/overcloud_images.yaml \
  -e /home/stack/triliovault-cfg-scripts/redhat-director-scripts/rhosp16.1/environments/trilio_env.yaml \
  -e /usr/share/openstack-tripleo-heat-templates/environments/ssl/enable-tls.yaml \
  -e /usr/share/openstack-tripleo-heat-templates/environments/ssl/inject-trust-anchor.yaml \
  -e /home/stack/triliovault-cfg-scripts/redhat-director-scripts/rhosp16.1/environments/trilio_env_tls_endpoints_public_dns.yaml \
  -e /home/stack/triliovault-cfg-scripts/redhat-director-scripts/rhosp16.1/environments/trilio_nfs_map.yaml \
  --ntp-server 192.168.1.34 \
  --libvirt-type qemu \
  --log-file overcloud_deploy.log \
  -r /home/stack/templates/roles_data.yaml
```

## 8] Verify deployment

{% hint style="danger" %}
If the containers are in restarting state or not listed by the following command then your deployment is not done correctly. Please recheck if you followed the complete documentation.
{% endhint %}

### 8.1] On Controller node

Make sure Trilio dmapi and horizon containers are in a running state and no other Trilio container is deployed on controller nodes.\
When the role for these containers is not "controller" check on respective nodes according to configured roles\_data.yaml.

```
[root@overcloud-controller-0 heat-admin]# podman ps | grep trilio
26fcb9194566  rhosptrainqa.ctlplane.localdomain:8787/trilio/trilio-datamover-api:<HOTFIX-TAG-VERSION>-rhosp16.2        kolla_start           5 days ago  Up 5 days ago         trilio_dmapi
094971d0f5a9  rhosptrainqa.ctlplane.localdomain:8787/trilio/trilio-horizon-plugin:<HOTFIX-TAG-VERSION>-rhosp16.2       kolla_start           5 days ago  Up 5 days ago         horizon
```

### 8.2] On Compute node

Make sure Trilio datamover container is in running state and no other Trilio container is deployed on compute nodes.

```
[root@overcloud-novacompute-0 heat-admin]# podman ps | grep trilio
b1840444cc59  rhosptrainqa.ctlplane.localdomain:8787/trilio/trilio-datamover:<HOTFIX-TAG-VERSION>-rhosp16.2                    kolla_start           5 days ago  Up 5 days ago         trilio_datamover
```

### 8.3] On the node with Horizon service

Make sure horizon container is in running state. Please note that 'Horizon' container is replaced with Trilio Horizon container. This container will have latest OpenStack horizon + Trilio's horizon plugin.

```
[root@overcloud-controller-0 heat-admin]# podman ps | grep horizon
094971d0f5a9  rhosptrainqa.ctlplane.localdomain:8787/trilio/trilio-horizon-plugin:<HOTFIX-TAG-VERSION>-rhosp16.2       kolla_start           5 days ago  Up 5 days ago         horizon
```

If Trilio Horizon container is in restarted state on RHOSP 16.1.8/RHSOP 16.2.4 then use below workaroud

```
## Either of the below workarounds should be performed on all the controller nodes where issue occurs for horizon pod.

option-1: Restart the memcached service on controller using systemctl (command: systemctl restart tripleo_memcached.service)

option-2: Restart the memcached pod (command: podman restart memcached)
```

## 9] Enable mount-bind for NFS

T4O 4.2 has changed the calculation of the mount point.\
It is necessary to set up the mount-bind to make T4O 4.1 or older backups available for T4O 4.2

Please follow [this documentation](https://docs.trilio.io/openstack/tvo-4.2/deployment/enabling-tvo-4.1-or-older-backups-when-using-nfs-backup-target) to set up the mount bind for RHOSP.