Installing on Kolla Openstack
This page lists all steps required to deploy Trilio components on Kolla-ansible deployed OpenStack cloud.
Please ensure that the Trilio Appliance has been updated to the latest maintenance release before continuing the installation.
Refer to the below-mentioned acceptable values for the placeholderstriliovault_tagand ```**kolla_base_distro`** , in this document as per the Openstack environment:
Openstack Version | triliovault_tag | kolla_base_distro |
|---|---|---|
Victoria | 4.2.6-victoria | ubuntu
centos |
Wallaby | 4.2.6-wallaby | ubuntu
centos |
Yoga | 4.2.6-yoga | ubuntu
centos |
Backup target storage is used to store backup images taken by Trilio and details needed for configuration:
Following backup target types are supported by Trilio. Select one of them and get it ready before proceeding to the next step.
a) NFS
Need NFS share path
b) Amazon S3
- S3 Access Key
- Secret Key
- Region
- Bucket name
c) Other S3 compatible storage (Like, Ceph based S3)
- S3 Access Key
- Secret Key
- Region
- Endpoint URL (Valid for S3 other than Amazon S3)
- Bucket name
Clone triliovault-cfg-scripts GitHub repository on Kolla ansible server at '/root' or any other directory of your preference. Afterwards, copy Trilio Ansible role into Kolla-ansible roles directory
git clone -b TVO/4.2.6 https://github.com/trilioData/triliovault-cfg-scripts.git
cd triliovault-cfg-scripts/kolla-ansible/
# For Centos and Ubuntu
cp -R ansible/roles/triliovault /usr/local/share/kolla-ansible/ansible/roles/
## For Centos and Ubuntu
- Take backup of globals.yml
cp /etc/kolla/globals.yml /opt/
- Append Trilio global variables to globals.yml
cat ansible/triliovault_globals.yml >> /etc/kolla/globals.yml
Append
triliovault_passwords.yml to /etc/kolla/passwords.yml. Passwords are empty. Set these passwords manually in the /etc/kolla/passwords.yml.## For Centos and Ubuntu
- Take backup of passwords.yml
cp /etc/kolla/passwords.yml /opt/
- Append Trilio global variables to passwords.yml
cat ansible/triliovault_passwords.yml >> /etc/kolla/passwords.yml
- Edit '/etc/kolla/passwords.yml', go to end of the file and set trilio passwords.
# For Centos and Ubuntu
- Take backup of site.yml
cp /usr/local/share/kolla-ansible/ansible/site.yml /opt/
# If the OpenStack release is ‘yoga' append below Trilio code to site.yml
cat ansible/triliovault_site_yoga.yml >> /usr/local/share/kolla-ansible/ansible/site.yml
# If the OpenStack release is other than 'yoga' append below Trilio code to site.yml
cat ansible/triliovault_site.yml >> /usr/local/share/kolla-ansible/ansible/site.yml
For example:
If your inventory file name path '/root/multinode' then use following command.
cat ansible/triliovault_inventory.txt >> /root/multinode
This step is only required when the multi-IP NFS feature is used to connect different datamovers to the same NFS volume through multiple IPs
On kolla-ansible server node, change directory
cd triliovault-cfg-scripts/common/
Edit file '
triliovault_nfs_map_input.yml' in the current directory and provide compute host and NFS share/ip map.If IP addresses are used in the kolla-ansible inventory file then you should use same IP addresses in 'triliovault_nfs_map_input.yml' file too. If you used hostnames there then you need to use same hostnames here in nfs map input file.
Compute host names or IP addresses that you are using in nfs map input file here should match with kolla-ansible inventory file entries.
vi triliovault_nfs_map_input.ymlUpdate
PyYAML on the kolla-ansible server node onlypip3 install -U pyyaml
Expand the map file to create one to one mapping of compute and nfs share.
python ./generate_nfs_map.py
Result will be in file - '
triliovault_nfs_map_output.yml'Validate output map file
Open file '
triliovault_nfs_map_output.ymlvi triliovault_nfs_map_output.ymlavailable in the current directory and validate that all compute nodes are covered with all necessary nfs shares.
Append this output map file to 'triliovault_globals.yml'
File Path: '/home/stack/triliovault-cfg-scripts/kolla-ansible/ansible/triliovault_globals.yml’
cat triliovault_nfs_map_output.yml >> ../kolla-ansible/ansible/triliovault_globals.yml
Ensure to set multi_ip_nfs_enabled in __ triliovault_globals.yml file to yes
Edit
/etc/kolla/globals.yml file to fill Trilio backup target and build details.
You will find the Trilio related parameters at the end of globals.yml file.
Details like Trilio build version, backup target type, backup target details, etc need to be filled out.Following is the list of parameters that the usr needs to edit.
Parameter | Defaults/choices | comments |
|---|---|---|
triliovault_tag | <triliovault_tag > | Use the triliovault tag as per your Kolla openstack version. Exact tag is mentioned in the 1st step |
horizon_image_full | Uncomment | By default, Trilio Horizon container would not get deployed. Uncomment this parameter to deploy Trilio Horizon container instead of Openstack Horizon container. |
triliovault_docker_username | <dockerhub-login-username> | Default docker user of Trilio
(read permission only).
Get the Dockerhub login credentials from Trilio Sales/Support team |
triliovault_docker_password | <dockerhub-login-password> | Password for default docker user of Trilio
Get the Dockerhub login credentials from Trilio Sales/Support team |
triliovault_docker_registry | Default value: docker.io | Edit this value if a different container registry for Trilio containers is to be used.
Containers need to be pulled from docker.io and pushed to chosen registry first. |
triliovault_backup_target |
| nfs if the backup target is NFSamazon_s3 if the backup target is Amazon S3other_s3_compatible if the backup target type is S3 but not amazon S3. |
multi_ip_nfs_enabled | yes
no
default: no | This parameter is valid only If you want to use multiple IP/endpoints based NFS share/shares as backup target for TriloVault. |
triliovault_nfs_shares | <NFS-IP/FQDN>:/<NFS path> | NFS share path example: ‘192.168.122.101:/nfs/tvault’ |
triliovault_nfs_options | 'nolock,soft,timeo=180,
intr,lookupcache=none'.
for Cohesity nfs: 'nolock,soft,timeo=600,intr,lookupcache=none,nfsvers=3,retrans=10' | -These parameter set NFS mount options.
-Keep default values, unless a special requirement exists. |
triliovault_s3_access_key | S3 Access Key | Valid for amazon_s3 and |
triliovault_s3_secret_key | S3 Secret Key | Valid for amazon_s3 and other_s3_compatible |
triliovault_s3_region_name |
| Valid for amazon_s3 and other_s3_compatibleIf s3 storage doesn't have region parameter keep default |
triliovault_s3_bucket_name | S3 Bucket name | Valid for amazon_s3 and other_s3_compatible |
triliovault_s3_endpoint_url | S3 Endpoint URL | Valid for other_s3_compatible only |
triliovault_s3_ssl_enabled |
| Valid for other_s3_compatible onlySet true for SSL enabled S3 endpoint URL |
triliovault_s3_ssl_cert_file_name | s3-cert.pem | Valid for other_s3_compatible only with SSL enabled and self signed certificatesOR issued by a private authority.
In this case, copy the ceph s3 ca chain file to/etc/kolla/config/triliovault/directory on ansible server. Create this directory if it does not exist already.
|
triliovault_copy_ceph_s3_ssl_cert |
| Valid for other_s3_compatible onlySet to True when:
SSL enabled with self-signed certificates or issued by a private authority. |
In the case of a different registry than docker hub, Trilio containers need to be pulled from docker.io and pushed to preferred registries.
Following are the triliovault container image URLs for 4.2 releases**.**
Replace
kolla_base_distro and triliovault_tag variables with their values.\This {{ kolla_base_distro }} variable can be either 'centos' or 'ubuntu' depends on your base OpenStack distro
Trilio supports the Source-based containers from the OpenStack Yoga release **** onwards.
Below are the Source-based OpenStack deployment images
1. docker.io/trilio/kolla-{{ kolla_base_distro }}-trilio-datamover:{{ triliovault_tag }}
2. docker.io/trilio/kolla-{{ kolla_base_distro }}-trilio-datamover-api:{{ triliovault_tag }}
3. docker.io/trilio/kolla-{{ kolla_base_distro }}-trilio-horizon-plugin:{{ triliovault_tag }}
## EXAMPLE from Kolla Ubuntu source based OpenStack
docker.io/trilio/kolla-ubuntu-trilio-datamover:{{ triliovault_tag }}
docker.io/trilio/kolla-ubuntu-trilio-datamover-api:{{ triliovault_tag }}
docker.io/trilio/kolla-ubuntu-trilio-horizon-plugin:{{ triliovault_tag }}
Below are the Binary-based OpenStack deployment images
1. docker.io/trilio/kolla-{{ kolla_base_distro }}-trilio-datamover:{{ triliovault_tag }}
2. docker.io/trilio/kolla-{{ kolla_base_distro }}-trilio-datamover-api:{{ triliovault_tag }}
3. docker.io/trilio/{{ kolla_base_distro }}-binary-trilio-horizon-plugin:{{ triliovault_tag }}
## EXAMPLE from Kolla Ubuntu binary based OpenStack
docker.io/trilio/kolla-ubuntu-trilio-datamover:{{ triliovault_tag }}
docker.io/trilio/kolla-ubuntu-trilio-datamover-api:{{ triliovault_tag }}
docker.io/trilio/ubuntu-binary-trilio-horizon-plugin:{{ triliovault_tag }}
To enable Trilio's Snapshot mount feature it is necessary to make the Trilio Backup target available to the nova-compute and nova-libvirt containers.
Edit
/usr/local/share/kolla-ansible/ansible/roles/nova-cell/defaults/main.yml and find nova_libvirt_default_volumes variables. Append the Trilio mount bind /var/trilio:/var/trilio:shared to the list of already existing volumes.For a default Kolla installation, will the variable look as follows afterward:
nova_libvirt_default_volumes:
- "{{ node_config_directory }}/nova-libvirt/:{{ container_config_directory }}/:ro"
- "/etc/localtime:/etc/localtime:ro"
- "{{ '/etc/timezone:/etc/timezone:ro' if ansible_os_family == 'Debian' else '' }}"
- "/lib/modules:/lib/modules:ro"
- "/run/:/run/:shared"
- "/dev:/dev"
- "/sys/fs/cgroup:/sys/fs/cgroup"
- "kolla_logs:/var/log/kolla/"
- "libvirtd:/var/lib/libvirt"
- "{{ nova_instance_datadir_volume }}:/var/lib/nova/"
- "
{% if enable_shared_var_lib_nova_mnt | bool %}/var/lib/nova/mnt:/var/lib/nova/mnt:shared{% endif %}
"
- "nova_libvirt_qemu:/etc/libvirt/qemu"
- "{{ kolla_dev_repos_directory ~ '/nova/nova:/var/lib/kolla/venv/lib/python' ~ distro_python_version ~ '/site-packages/nova' if nova_dev_mode | bool else '' }
- "/var/trilio:/var/trilio:shared"
Next, find the variable
nova_compute_default_volumes in the same file and append the mount bind /var/trilio:/var/trilio:shared to the list.After the change will the variable look for a default Kolla installation as follows:
nova_compute_default_volumes:
- "{{ node_config_directory }}/nova-compute/:{{ container_config_directory }}/:ro"
- "/etc/localtime:/etc/localtime:ro"
- "{{ '/etc/timezone:/etc/timezone:ro' if ansible_os_family == 'Debian' else '' }}"
- "/lib/modules:/lib/modules:ro"
- "/run:/run:shared"
- "/dev:/dev"
- "kolla_logs:/var/log/kolla/"
- "
{% if enable_iscsid | bool %}iscsi_info:/etc/iscsi{% endif %}"
- "libvirtd:/var/lib/libvirt"
- "{{ nova_instance_datadir_volume }}:/var/lib/nova/"
- "{% if enable_shared_var_lib_nova_mnt | bool %}/var/lib/nova/mnt:/var/lib/nova/mnt:shared{% endif %}
"
- "{{ kolla_dev_repos_directory ~ '/nova/nova:/var/lib/kolla/venv/lib/python' ~ distro_python_version ~ '/site-packages/nova' if nova_dev_mode | bool else '' }}"
- "/var/trilio:/var/trilio:shared"
In case of using Ironic compute nodes one more entry need to be adjusted in the same file.
Find the variable
nova_compute_ironic_default_volumes and append trilio mount /var/trilio:/var/trilio:shared to the list.After the changes the variable will looks like the following:
nova_compute_ironic_default_volumes:
- "{{ node_config_directory }}/nova-compute-ironic/:{{ container_config_directory }}/:ro"
- "/etc/localtime:/etc/localtime:ro"
- "{{ '/etc/timezone:/etc/timezone:ro' if ansible_os_family == 'Debian' else '' }}"
- "kolla_logs:/var/log/kolla/"
- "{{ kolla_dev_repos_directory ~ '/nova/nova:/var/lib/kolla/venv/lib/python' ~ distro_python_version ~ '/site-packages/nova' if nova_dev_mode | bool else '' }}"
- "/var/trilio:/var/trilio:shared"
Activate the login into dockerhub for Trilio tagged containers.
Please get the Dockerhub login credentials from Trilio Sales/Support team
ansible -i multinode control -m shell -a "docker login -u <docker-login-username> -p <docker-login-password> docker.io"
Pull the Trilio container images from the dockerhub based on the existing inventory file. In the example is the inventory file named
multinode.kolla-ansible -i multinode pull --tags triliovault
All that is left, is to run the deploy command using the existing inventory file. In the example is the inventory file named 'multinode'.
This is just an example command. You need to use your cloud deploy command.
kolla-ansible -i multinode deploy
Post deployment for multipath enabled environment, log into respective datamover container and add uxsock_timeout with value as 60000 (i.e. 60 sec) in /etc/multipath.conf. Restart datamover container
Verify on the nodes that are supposed to run the Trilio containers, that those are available and healthy.
The example is shown for 4.2.6 maintenance release from Kolla Victoria CentOS setup.
[[email protected] ~]# docker ps | grep datamover-api
cd9d0ccc19b6 trilio/kolla-centos-trilio-datamover-api:4.2.6-victoria "dumb-init --single-…" 21 hours ago Up 21 hours triliovault_datamover_api
[[email protected] ~]# docker ps | grep datamover
fae5e4f2e04a trilio/kolla-centos-trilio-datamover:4.2.6-victoria "dumb-init --single-…" 21 hours ago Up 21 hours triliovault_datamover
[[email protected] ~]# docker ps | grep horizon
f019ef071d3c trilio/centos-binary-trilio-horizon-plugin:4.2.6-victoria "dumb-init --single-…" 21 hours ago Up 21 hours (unhealthy) horizon
The example is shown for 4.2.6 maintenance release from Kolla Yoga Ubuntu source based setup.
[email protected]:~# docker ps | grep triliovault_datamover_api
5e9f87240a25 trilio/kolla-ubuntu-trilio-datamover-api:4.2.6-yoga "dumb-init --single-…" 23 hours ago Up 23 hours triliovault_datamover_api
[email protected]:~# docker ps | grep horizon
4cd644f0486c trilio/kolla-ubuntu-trilio-horizon-plugin:4.2.6-yoga "dumb-init --single-…" 23 hours ago Up 23 hours (healthy) horizon
[email protected]:~# docker ps | grep triliovault_datamover
7b6001ef43b9 trilio/kolla-ubuntu-trilio-datamover:4.2.6-yoga "dumb-init --single-…" 23 hours ago Up 23 hours triliovault_datamover
The example is shown for 4.2.6 maintenance release from Kolla Yoga Ubuntu binary based setup.
[email protected]:~# docker ps | grep triliovault_datamover_api
686b1aff0165 trilio/kolla-ubuntu-trilio-datamover-api:4.2.6-yoga "dumb-init --single-…" 3 hours ago Up 3 hours triliovault_datamover_api
[email protected]:~# docker ps | grep horizon
d49ac6f52af4 trilio/ubuntu-binary-trilio-horizon-plugin:4.2.6-yoga "dumb-init --single-…" 3 hours ago Up 3 hours (healthy) horizon
[email protected]:~# docker ps | grep triliovault_datamover
c5a01651ddc7 trilio/kolla-ubuntu-trilio-datamover:4.2.6-yoga "dumb-init --single-…" 3 hours ago Up 3 hours triliovault_datamover
To see all TriloVault containers running on a specific node use the docker ps command.
docker ps -a | grep trilio
To check the startup logs use the docker logs <container name> command.
docker logs trilio_datamover_api
docker logs trilio_datamover
Verify that the Trilio Appliance is configured. The Horizon tabs are only shown, when a configured Trilio appliance is available.
Verify that the Trilio horizon container is installed and in a running state.
docker ps | grep horizon
- Trilio datamover api service logs on datamover api node
/var/log/kolla/triliovault-datamover-api/dmapi.log
- Trilio datamover service logs on datamover node
/var/log/kolla/triliovault-datamover/tvault-contego.log
Note: This step needs to be done on Trilio Appliance node. Not on OpenStack node.Pre-requisite:
You should have already launched Trilio appliance VM
In Kolla openstack distribution, 'nova' user id on nova-compute docker container is set to '42436'. The 'nova' user id on the Trilio nodes need to be set the same. Do the following steps on all Trilio nodes:
- 1.Download the shell script that will change the user id
- 2.Assign executable permissions
- 3.Execute the script
- 4.Verify that 'nova' user and group id has changed to '42436'
- 5.After this step, you can proceed to 'Configuring Trilio' section.
## Download the shell script
$ curl -O https://raw.githubusercontent.com/trilioData/triliovault-cfg-scripts/master/common/nova_userid.sh
## Assign executable permissions
$ chmod +x nova_userid.sh
## Execute the shell script to change 'nova' user and group id to '42436'
$ ./nova_userid.sh
## Ignore any errors and verify that 'nova' user and group id has changed to '42436'
$ id nova
uid=42436(nova) gid=42436(nova) groups=42436(nova),990(libvirt),36(kvm)
Last modified 1d ago