Installing on Ansible Openstack Train

Installing Trilio Datamover-API container

The installation of the Datamover-API, short dmapi, requires to create a new container, in which all necessary packages and the Trilio dmapi code are loaded.

Create the dmapi container on Controller nodes

Create lxc container for hosting dmapi service on controller nodes with below commands.

## On Ubuntu
lxc-create -t ubuntu -n controller_dmapi       #Create the controller_dmapi container 

## On CentOS
lxc-create -t centos -n controller_dmapi       #Create the controller_dmapi container

lxc-start -n controller_dmapi                  #Start the controller_dmapi container

Prepare the dmapi container for installation

Add nova user and required directory on container controller_dmapi.

lxc-attach -n controller_dmapi      # Login to container
mkdir /etc/nova                     # create require /etc/nova directory
useradd nova                        # create nova user
id nova                             # check user added on container and aslo confirm the group nova added on container

Add required packages on container controller_dmapi.

## On Ubuntu
sudo apt-get install software-properties-common
sudo add-apt-repository cloud-archive:train
sudo apt-get install python3-dev
sudo apt-get install curl
sudo apt-get install python3-nova

## On CentOs
yum install centos-release-openstack-train
yum install python-devel
yum install python-nova

Copy nova.conf file from nova-api container to /etc/nova directory in dmapi container. Run the below command on controller nodes:

cp /var/lib/lxc/controller_nova_api_container/rootfs/etc/nova/nova.conf /var/lib/lxc/controller_dmapi/rootfs/etc/nova/

Create a new interface with specific ip for dmapi container.

Edit /var/lib/lxc/controller_dmapi/config and add below section as per network bridge available on the controller node.

lxc.net.1.type = veth
lxc.net.1.name = ens4
lxc.net.1.link = br-mgmt
lxc.net.1.hwaddr = 00:16:3e:de:52:31
lxc.net.1.flags = up
lxc.net.1.mtu = 1500
lxc.net.1.ipv4.address = 172.26.0.104/22

Restart the container with the below commands.

lxc-stop -n controller_dmapi
lxc-start -n controller_dmapi

Install the dmapi service

Download and run the tvault-installation script inside the container.

wget https://<ipaddress-of-triliovault-controller>/tvault-contego-install.sh --no-check-certificate
chmod +x tvault-contego-install.sh

The script to be executed inside dmapi container, after the following changes have been done: Comment the 2 lines below and add a line below NOVA_VERSION = 20, as nova-manage doesn't work in Ansible Openstack.

# NOVA_VERSION=`nova-manage version | awk -F. '{print $1}'`
# NOVA_VERSION_ROCKY=`nova-manage version`
NOVA_VERSION = 20

Run the script

./tvault-contego-install.sh --install

Verify the installation and set necessary configuration

Verify in dmapi.conf domain name for the nova service user under keystone section. Check field values for project_domain_nameand user_domain_name and update those if not in keystone section

[keystone_authtoken]
 memcached_servers =
 signing_dir =
 cafile =
 project_domain_name = Default
 project_name = service
 user_domain_name = Default
 password = ************
 username = nova
 auth_url = http://172.26.0.3:5000
 auth_type = password
 user_domain_id = default
 project_domain_id = default
 auth_uri =
 memcached_servers =

If SSL is enabled then add the following section in dmapi.conf:

[oslo_messaging_rabbit]
ssl = True
rpc_conn_pool_size = 30

Verify below entries are there in keystone policy.json file

file path : /var/lib/lxc/controller_keystone_container/rootfs/etc/keystone

"cloud_admin": "rule:admin_required and (is_admin_project:True or domain_id:CLOUD_DOMAIN_ID or project_id:SERVICE_PROJECT_ID)"

Once verified above checks, start the dmapi service.

service tvault-datamover-api start
service tvault-datamover-api status

Install Trilio Datamover on Compute nodes

Prepare the installation

Activate the virtual environment on the compute node.

source /openstack/venvs/nova-20.0.3.dev2/bin/activate

After activating the virtual environment, find out the location of compute.filters file.

(nova-20.0.3.dev2) root@compute:~# find / -name compute.filters
/openstack/venvs/nova-20.0.3.dev2/etc/nova/rootwrap.d/compute.filters

Download the installation script.

wget https://<ipaddress-of-triliovault-controller>/tvault-contego-install.sh --no-check-certificate
chmod +x tvault-contego-install.sh

Modify install script to use the same location for creating trilio filters.

1797              elif [ "$value" == 2 ]; then
1798                  NOVA_COMPUTE_FILTERS_FILE="/openstack/venvs/nova-20.0.3.dev2/etc/nova/rootwrap.d/compute.filters"
1799                  create_filter_file /openstack/venvs/nova-20.0.3.dev2/etc/nova/rootwrap.d/trilio.filters

Also comment the 2 lines below and add a line below NOVA_VERSION = 20, as nova-manage doesn't work in Ansible Openstack.

# NOVA_VERSION=`nova-manage version | awk -F. '{print $1}'`
# NOVA_VERSION_ROCKY=`nova-manage version`
NOVA_VERSION = 20

Install the Trilio datamover service

Run install script and if you get prompt while installing, choose the default selection.

If non-default values are selected, it will overwrite the current configuration and will impact nova-compute service.

./tvault-contego-install.sh --install

Make sure ExecStart value look like below in /etc/systemd/system/tvault-contego.service file.

ExecStart=/openstack/venvs/nova-20.0.3.dev2/bin/python3 /usr/bin/tvault-contego --config-file=/etc/nova/nova.conf --config-file=/etc/tvault-contego/tvault-contego.conf

Use below commands to restart and verify the service.

systemctl daemon-reload
service tvault-contego start
service tvault-contego status

Use below command and check if nfs/s3 storage is mounted or not.

df -h

Install Horizon Plugin into Horizon service

List running containers on controller nodes and login to horizon container using the below command.

lxc-ls
 lxc-attach -n <name of horizon container>

Install curl package on the Horizon container if not present.

## On Ubuntu
apt-get install curl

## On CentOS
yum install curl

Activate virtual environment on horizon container

source /openstack/venvs/horizon-20.0.3.dev2/bin/activate

Download script to install horizon plugin on horizon container and run install script

wget https://<ipaddress-of-triliovault-controller>/tvault-horizon-plugin-install.sh --no-check-certificate
chmod +x tvault-horizon-plugin-install.sh
./tvault-horizon-plugin-install.sh

Install script will ask for the dashboard folder, provide below path

/openstack/venvs/horizon-20.0.3.dev2/lib/python3.6/site-packages/

Verify installation using below commands

pip list | grep tvault

Haproxy configuration on controller nodes

Refer to the keystone haproxy settings for dmapi haproxy.

Haproxy config file can be found here: /etc/haproxy/haproxy.cfg

A sample configuration is shown below.

frontend datamover_service-front-1
        bind trainubuntu.triliodata.demo:8784 ssl crt /etc/ssl/private/haproxy.pem ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS
        option httplog
        option forwardfor except 127.0.0.0/8
        option http-server-close
        reqadd X-Forwarded-Proto:\ https
        mode http
        default_backend datamover_service-back

frontend datamover_service-front-2
        bind 172.26.0.3:8784
        option httplog
        option forwardfor except 127.0.0.0/8
        option http-server-close
        mode http
        default_backend datamover_service-back


backend datamover_service-back
        mode http
        balance leastconn
        stick store-request src
        stick-table type ip size 256k expire 30m
        option forwardfor
        option httplog
        option httpchk HEAD / HTTP/1.0\r\nUser-agent:\ osa-haproxy-healthcheck
        server controller_dmapi  172.26.0.104:8784 check port 8784 inter 12000 rise 1 fall 1

Check the syntax of the file and restart the service.

haproxy -c -f /etc/haproxy/haproxy.conf
systemctl restart haproxy