> For the complete documentation index, see [llms.txt](https://docs.trilio.io/openstack/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.trilio.io/openstack/upgrading-to-t4o-6.x-from-older-supported-versions/upgradetrilio/upgrading-on-kolla.md). # Upgrading on Kolla-Ansible {% hint style="info" %} This page covers two upgrade paths on Kolla-Ansible. Pick the one that matches your source release: * [**Upgrading from T4O 6.2.0 to 6.2.1**](#upgrading-from-t4o-6.2.0-to-6.2.1-epoxy-and-flamingo) — both releases use the Dynamic Mount Service (DMS), so this is a straightforward script-driven upgrade with no backup target migration required. * [**Upgrading from an older, pre-DMS release (T4O 5.x or early 6.x)**](#upgrading-from-an-older-pre-dms-release-to-t4o-6.2.1) — T4O 6.2 replaced the per-backup-target object-store containers with DMS, so this path also migrates existing backup targets and workloads in addition to the usual image/config update. {% endhint %} ## Upgrading from T4O 6.2.0 to 6.2.1 (Epoxy and Flamingo) ### 1] Prerequisites Before upgrading, ensure the following: * No Snapshot or Restore is running * The global job scheduler is disabled ### 2] Back up scripts and clone the target release Rename your existing `triliovault-cfg-scripts` directory to keep it available for reference/rollback, then clone the 6.2.1 release into a new directory: ``` mv triliovault-cfg-scripts triliovault-cfg-scripts-6.2.0 git clone -b {{ trilio_branch }} https://github.com/trilioData/triliovault-cfg-scripts.git triliovault-cfg-scripts-6.2.1 cd triliovault-cfg-scripts-6.2.1/kolla-ansible/ansible/ ``` ### 3] Activate the kolla virtual environment The venv path depends on the base distro of your Kolla deployment: ``` # Kolla Rocky (Epoxy and Flamingo) export venv_path="/opt/kolla-venv" # Kolla Ubuntu (Epoxy and Flamingo) export venv_path="/root/kolla-venv" source $venv_path/bin/activate ``` ### 4] Run the upgrade preparation script Run `prepare_install_upgrade.py` in `upgrade` mode to update the Trilio global variables, passwords, site.yml content, and inventory entries for your OpenStack release in the corresponding Kolla-ansible files. ``` # Epoxy python3 scripts/prepare_install_upgrade.py --os-release 2025.1 --mode upgrade --venv-path $venv_path --inventory-file --globals-file /etc/kolla/globals.yml # Flamingo python3 scripts/prepare_install_upgrade.py --os-release 2025.2 --mode upgrade --venv-path $venv_path --inventory-file --globals-file /etc/kolla/globals.yml ``` * `--os-release`:arrow\_right: Use `2025.1` for Epoxy or `2025.2` for Flamingo. * `--mode`:arrow\_right: Use `upgrade` when upgrading an existing Trilio installation. * `--venv-path`:arrow\_right: The kolla virtual environment path exported in the previous step. * `--inventory-file`:arrow\_right: Path to your cloud’s kolla-ansible inventory file. * `--globals-file`:arrow\_right: Path to your cloud’s `globals.yml` file. {% hint style="info" %} Before modifying any file, the script creates a timestamped backup directory in the current working directory (`./backup/YYYYMMDD_HHMMSS/`) and copies `/etc/kolla/globals.yml`, `/etc/kolla/passwords.yml`, `site.yml`, and the inventory file into it before making changes. The exact backup path is printed in the script header — copy a file back from there if you need to restore it. {% endhint %} In `upgrade` mode the script: * **globals.yml** — replaces the Trilio block; previously set credentials are preserved automatically, and the Trilio version tags are bumped to the target release * **passwords.yml** — skipped if all 5 Trilio password keys are already present (passwords are never regenerated) * **site.yml** — replaces the Trilio plays block with fresh content for the release * **inventory** — fully replaces the Trilio inventory block * **nova-cell volumes** — no-op if already patched * **Horizon settings** — no-op if the file already exists ### 5] Update Trilio version tags in globals.yml Open `/etc/kolla/globals.yml` and confirm `triliovault_tag` and `triliovault_dms_tag` are set to the values for 6.2.1, listed on the [Resources](/openstack/about-trilio-for-openstack/artifacts.md) page: ``` vi /etc/kolla/globals.yml ``` Cloud admin details and passwords populated during the original installation do not need to change. ### 6] Pull updated container images ``` ansible -i control -m shell -a "docker login -u -p docker.io" --become kolla-ansible pull -i --tags triliovault ``` ### 7] Deploy the upgrade ``` kolla-ansible deploy -i --tags triliovault ``` {% hint style="info" %} Post deployment for multipath enabled environment, log into respective datamover container and add uxsock\_timeout with value as 60000 (i.e. 60 sec) in /etc/multipath.conf. Restart datamover container {% endhint %} ### 8] Verify the upgrade ``` docker ps | grep trilio docker inspect triliovault_wlm_api | grep -i image workloadmgr backup-target-list ``` ## Upgrading from an older, pre-DMS release to T4O 6.2.1 {% hint style="info" %} Use this path when upgrading from an older, pre-DMS Trilio release (T4O 5.x or early 6.x) directly to T4O 6.2.1 on Kolla-Ansible (Epoxy/Flamingo). T4O 6.2 replaced the per-backup-target object-store containers with the Dynamic Mount Service (DMS), so upgrading from an older release requires migrating existing backup targets and workloads in addition to the usual image/config update. {% endhint %} ### 1] Prerequisites Before upgrading, ensure the following: * No Snapshot or Restore is running * The global job scheduler is disabled ### 2] Back up scripts and clone the target release Rename your existing `triliovault-cfg-scripts` directory to keep it available for reference/rollback, then clone the target release into a new directory: ``` mv triliovault-cfg-scripts triliovault-cfg-scripts-{{ current_version }} git clone -b {{ trilio_branch }} https://github.com/trilioData/triliovault-cfg-scripts.git triliovault-cfg-scripts-{{ target_version }} cd triliovault-cfg-scripts-{{ target_version }}/kolla-ansible/ansible/ ``` **Example**, upgrading from 5.2.7 to 6.2.1: ``` mv triliovault-cfg-scripts triliovault-cfg-scripts-5.2.7 git clone -b 6.2.1 https://github.com/trilioData/triliovault-cfg-scripts.git triliovault-cfg-scripts-6.2.1 cd triliovault-cfg-scripts-6.2.1/kolla-ansible/ansible/ ``` ### 3] Collect the existing backup target configuration Run this **before** making any further changes. Older releases don't expose `workloadmgr backup-target-list`, so this script reads the backup target configuration directly out of `globals.yml` and saves it for use when recreating the backup target in step 10. ``` bash scripts/collect_backup_targets.sh -g /etc/kolla/globals.yml -o existing_backup_targets.yaml ``` {% hint style="warning" %} `existing_backup_targets.yaml` contains the S3 secret key in plain text. Treat it as sensitive, and delete it once step 10 completes successfully. {% endhint %} ### 4] Activate the kolla virtual environment The venv path depends on the base distro of your Kolla deployment. For example: ``` export venv_path="/opt/kolla-venv" source $venv_path/bin/activate ``` ### 5] Run the upgrade preparation script Run `prepare_install_upgrade.py` in `upgrade` mode to update the Trilio global variables, passwords, site.yml content, and inventory entries for your OpenStack release in the corresponding Kolla-ansible files. ``` python3 scripts/prepare_install_upgrade.py \ --os-release <2025.1 or 2025.2> \ --mode upgrade \ --venv-path $venv_path \ --inventory-file \ --globals-file /etc/kolla/globals.yml ``` * `--os-release`:arrow\_right: Use `2025.1` for Epoxy or `2025.2` for Flamingo. * `--mode`:arrow\_right: Use `upgrade` when upgrading an existing Trilio installation. * `--venv-path`:arrow\_right: The kolla virtual environment path exported in the previous step. * `--inventory-file`:arrow\_right: Path to your cloud’s kolla-ansible inventory file. * `--globals-file`:arrow\_right: Path to your cloud’s `globals.yml` file. {% hint style="info" %} Before modifying any file, the script creates a timestamped backup directory in the current working directory (`./backup/YYYYMMDD_HHMMSS/`) and copies `/etc/kolla/globals.yml`, `/etc/kolla/passwords.yml`, `site.yml`, and the inventory file into it before making changes. The exact backup path is printed in the script header — copy a file back from there if you need to restore it. {% endhint %} In `upgrade` mode the script: * **globals.yml** — replaces the Trilio block; previously set credentials are preserved automatically, and the Trilio version tags are bumped to the target release * **passwords.yml** — skipped if all 5 Trilio password keys are already present (passwords are never regenerated) * **site.yml** — replaces the Trilio plays block with fresh content for the release * **inventory** — fully replaces the Trilio inventory block * **nova-cell volumes** — no-op if already patched * **Horizon settings** — no-op if the file already exists ### 6] Set Trilio version tags in globals.yml Open `/etc/kolla/globals.yml` and set `triliovault_tag` and `triliovault_dms_tag` to the values for your target release, listed on the [Resources](/openstack/about-trilio-for-openstack/artifacts.md) page: ``` vi /etc/kolla/globals.yml ``` Cloud admin details and passwords populated during the original installation do not need to change. ### 7] Pull updated container images ``` ansible -i control -m shell -a "docker login -u -p docker.io" --become kolla-ansible pull -i --tags triliovault ``` ### 8] Deploy the upgrade ``` kolla-ansible deploy -i --tags triliovault ``` {% hint style="info" %} Post deployment for multipath enabled environment, log into respective datamover container and add uxsock\_timeout with value as 60000 (i.e. 60 sec) in /etc/multipath.conf. Restart datamover container {% endhint %} ### 9] Clean up legacy object-store containers and mounts Once the 6.2.1 DMS-based containers are deployed, remove the old per-backup-target object-store containers and mounts left over from the pre-DMS release: ``` ansible-playbook -i cleanup_backup_targets.yml --tags upgrade-and-clean-backup-target-mounts ``` This playbook: * Stops and removes all `triliovault_object_store*` Docker containers on WLM and datamover nodes * Lazy-unmounts S3/FUSE and NFS mounts under `/var/trilio/triliovault-mounts` * Removes the `/etc/kolla/triliovault-object-store` config directory and any empty mount-point subdirectories * Removes the old backup target variables (`triliovault_backup_target`, `triliovault_nfs_*`, `triliovault_s3_*`, `multi_ip_nfs_enabled`, `triliovault_copy_ceph_s3_ssl_cert`) from `/etc/kolla/globals.yml` on the ansible control node Verify cleanup: ``` # No old object-store containers (expected: no output) docker ps -a | grep triliovault_object_store # No hung mounts — run on each compute and control node (expected: no output) findmnt -r | grep triliovault-mounts # No stale empty directories (expected: empty) ls /var/trilio/triliovault-mounts/ ``` ### 10] Recreate the backup target in T4O 6.2.1 Using the configuration collected in step 3, recreate the backup target against the new DMS-based schema. The playbook runs on a single controller node and executes `workloadmgr` inside the `triliovault_wlm_api` container. The backup target name is auto-generated from its type: `nfs-backup-target-1` or `s3-backup-target-1`. **NFS backup target:** ``` ansible-playbook -i create_backup_target_62.yml ``` **S3 backup target** — the secret key is read automatically from `existing_backup_targets.yaml` (collected in step 3); no export or extra-var needed: ``` ansible-playbook -i create_backup_target_62.yml ``` ### 11] Import existing workloads and update backing files Workloads created under the pre-DMS release need to be re-registered in the database and have their backing files rebased onto the new DMS-mounted backup target. #### 11.1] Update existing workload and policy data ``` docker cp /etc/kolla/triliovault-cloudrc triliovault_wlm_api:/opt/ docker exec -it triliovault_wlm_api bash source /opt/triliovault-cloudrc workloadmgr workload-inplace-db-upgrade ``` This command updates all workload and policy records in the database to match the current release version. #### 11.2] Mount the backup target (S3 backup target only) {% hint style="info" %} This step is required only when the backup target is S3. Skip it for NFS backup targets. {% endhint %} Still inside the `triliovault_wlm_api` container: ``` target_id=`workloadmgr backup-target-list -f yaml | grep ID | awk '{ print $2 }'` token=`openstack token issue -f yaml | grep ^id | awk '{ print $2 }'` mountpath=`workloadmgr backup-target-list -f yaml | awk -F': ' '/Backend Mountpath/{print $2}'` secret_ref=`workloadmgr backup-target-list -f yaml | awk -F': ' '/Secret Ref/{print $2}'` workloadmgr dms-mount --job-id 10001 --target-id $target_id --target-type s3 --token $token --mount-path $mountpath --secret-ref $secret_ref ``` #### 11.3] Update the backing file for each workload (S3 backup target only) {% hint style="info" %} This step is required only when the backup target is S3. Skip it for NFS backup targets. {% endhint %} {% hint style="success" %} `backing_file_update.sh` is not publicly available — request it from your Trilio Customer Success Manager/Engineer via the Customer Portal. See [Switching NFS Backing file](/openstack/advanced-configuration/switch-nfs-backing-file.md) for full script details and logging. {% endhint %} Copy the script into the `triliovault_wlm_api` container: ``` docker cp backing_file_update.sh triliovault_wlm_api:/tmp/ ``` Then, back inside the container, run it once per workload against its mount path: ``` cd /tmp ./backing_file_update.sh /var/trilio/triliovault-mounts//workload_ ``` Example: ``` ./backing_file_update.sh /var/trilio/triliovault-mounts/Y2VwaHF1aW5jeS50cmlsaW9kYXRhLmRlbW8vdHJpbGlvLWF1dG9tYXRpb24=/workload_6edc1e04-2c3e-459a-9daf-829debc1c228 ``` ### 12] Verify the upgrade ``` docker ps | grep trilio docker inspect triliovault_wlm_api | grep -i image workloadmgr backup-target-list ``` ### 13] Recreate cloud admin trust (if required) If the cloud admin trust needs to be re-established after the upgrade, run: ``` ansible-playbook -i {{ kolla inventory file }} $venv_path/share/kolla-ansible/ansible/roles/triliovault/tasks/wlm_cloud_trust.yml -e "@/etc/kolla/globals.yml" ```