Post Installation Health-Check

Search…
After the installation and configuration of TrilioVault for RHV did succeed the following steps can be done to verify that the TrilioVault installation is healthy.
Verify the TrilioVault Appliance services are up
TrilioVault is using 3 main services on the TrilioVault Appliance:
wlm-api
wlm-scheduler
wlm-workloads
Those can be verified to be up and running using the systemctl status command.
1
systemctl status wlm-api
2
######
3
● wlm-api.service - Cluster Controlled wlm-api
4
   Loaded: loaded (/etc/systemd/system/wlm-api.service; disabled; vendor preset: disabled)
5
  Drop-In: /run/systemd/system/wlm-api.service.d
6
           └─50-pacemaker.conf
7
   Active: active (running) since Wed 2020-04-22 09:17:05 UTC; 1 day 2h ago
8
 Main PID: 21265 (python)
9
    Tasks: 1
10
   CGroup: /system.slice/wlm-api.service
11
           └─21265 /home/rhv/myansible/bin/python /usr/bin/workloadmgr-api --config-file=/etc/workloadmgr/workloadmgr.conf
Copied!
1
systemctl status wlm-scheduler
2
######
3
● wlm-scheduler.service - Cluster Controlled wlm-scheduler
4
   Loaded: loaded (/etc/systemd/system/wlm-scheduler.service; disabled; vendor preset: disabled)
5
  Drop-In: /run/systemd/system/wlm-scheduler.service.d
6
           └─50-pacemaker.conf
7
   Active: active (running) since Wed 2020-04-22 09:17:17 UTC; 1 day 2h ago
8
 Main PID: 21512 (python)
9
    Tasks: 1
10
   CGroup: /system.slice/wlm-scheduler.service
11
           └─21512 /home/rhv/myansible/bin/python /usr/bin/workloadmgr-scheduler --config-file=/etc/workloadmgr/workloadmgr.conf
Copied!
1
systemctl status wlm-workloads
2
######
3
● wlm-workloads.service - workloadmanager workloads service
4
   Loaded: loaded (/etc/systemd/system/wlm-workloads.service; enabled; vendor preset: disabled)
5
   Active: active (running) since Wed 2020-04-22 09:15:43 UTC; 1 day 2h ago
6
 Main PID: 20079 (python)
7
    Tasks: 33
8
   CGroup: /system.slice/wlm-workloads.service
9
           ├─20079 /home/rhv/myansible/bin/python /usr/bin/workloadmgr-workloads --config-file=/etc/workloadmgr/workloadmgr.conf
10
           ├─20180 /home/rhv/myansible/bin/python /usr/bin/workloadmgr-workloads --config-file=/etc/workloadmgr/workloadmgr.conf
11
           [...]
12
           ├─20181 /home/rhv/myansible/bin/python /usr/bin/workloadmgr-workloads --config-file=/etc/workloadmgr/workloadmgr.conf
13
           ├─20233 /home/rhv/myansible/bin/python /usr/bin/workloadmgr-workloads --config-file=/etc/workloadmgr/workloadmgr.conf
14
           ├─20236 /home/rhv/myansible/bin/python /usr/bin/workloadmgr-workloads --config-file=/etc/workloadmgr/workloadmgr.conf
15
           └─20237 /home/rhv/myansible/bin/python /usr/bin/workloadmgr-workloads --config-file=/etc/workloadmgr/workloadmgr.conf
Copied!
Check the TrilioVault pacemaker and nginx cluster
The second component to check the TrilioVault Appliance's health is the nginx and pacemaker cluster.
1
pcs status
2
######
3
Cluster name: triliovault
4
​
5
WARNINGS:
6
Corosync and pacemaker node names do not match (IPs used in setup?)
7
Stack: corosync
8
Current DC: om_tvm (version 1.1.19-8.el7_6.1-c3c624ea3d) -
9
partition with quorum
10
Last updated: Wed Dec 5 12:25:02 2018
11
Last change: Wed Dec 5 09:20:08 2018 by root via cibadmin on om_tvm
12
1 node configured
13
4 resources configured
14
​
15
Online: [ om_tvm ]
16
Full list of resources:
17
virtual_ip (ocf::'heartbeat:IPaddr2): Started om_tvm
18
wlm-api (systemd:wlm-api): Started om_tvm
19
wlm-scheduler (systemd:wlm-scheduler): Started om_tvm
20
Clone Set: lb_nginx-clone [lb_nginx]
21
Started: [ om_tvm ]
22
Daemon Status:
23
corosync: active/enabled
24
pacemaker: active/enabled
25
pcsd: active/enabled
Copied!
Verify API connectivity from the RHV-Manager
The RHV-Manager is doing all API calls towards the TrilioVault Appliance.
Therefore it is helpful to do a quick API connectivity check using curl.
The following curl command lists the available workload-types and verfifies that the connection is available and working:
1
curl -k -XGET https://30.30.1.11:8780/v1/admin/workload_types/detail -H "Content-Type: application/json" -H "X-OvirtAuth-User: [email protected]" -H "X-OvirtAuth-Password: password"
2
######
3
{"workload_types": [{"status": "available", "user_id": "[email protected]", "name": "Parallel", "links": [{"href": "https://myapp/v1/admin/workloadtypes/2ddd528d-c9b4-4d7e-8722-cc395140255a", "rel": "self"}, {"href": "https://myapp/admin/workloadtypes/2ddd528d-c9b4-4d7e-8722-cc395140255a", "rel": "bookmark"}], "created_at": "2020-04-02T15:38:51.000000", "updated_at": "2020-04-02T15:38:51.000000", "metadata": [], "is_public": true, "project_id": "admin", "id": "2ddd528d-c9b4-4d7e-8722-cc395140255a", "description": "Parallel workload that snapshots VM in the specified order"}, {"status": "available", "user_id": "[email protected]", "name": "Serial", "links": [{"href": "https://myapp/v1/admin/workloadtypes/f82ce76f-17fe-438b-aa37-7a023058e50d", "rel": "self"}, {"href": "https://myapp/admin/workloadtypes/f82ce76f-17fe-438b-aa37-7a023058e50d", "rel": "bookmark"}], "created_at": "2020-04-02T15:38:47.000000", "updated_at": "2020-04-02T15:38:47.000000", "metadata": [], "is_public": true, "project_id": "admin", "id": "f82ce76f-17fe-438b-aa37-7a023058e50d", "description": "Serial workload that snapshots VM in the specified order"}]}
Copied!
Verify the ovirt-imageio services are up and running
TrilioVault is extending the already exiting ovirt-imageio services.
The installation of these extensions does check if the ovirt-services come up.
Still it is a good call to verify again afterwards:
RHV 4.3.X
On the RHV-Manager check the ovirt-imageio-proxy service:
1
systemctl status ovirt-imageio-proxy
2
######
3
● ovirt-imageio-proxy.service - oVirt ImageIO Proxy
4
   Loaded: loaded (/usr/lib/systemd/system/ovirt-imageio-proxy.service; enabled; vendor preset: disabled)
5
   Active: active (running) since Wed 2020-04-08 05:05:25 UTC; 2 weeks 1 days ago
6
 Main PID: 1834 (python)
7
   CGroup: /system.slice/ovirt-imageio-proxy.service
8
           └─1834 bin/python proxy/ovirt-imageio-proxy
Copied!
On the RHV-Host check the ovirt-imageio-daemon service:
1
systemctl status ovirt-imageio-daemon
2
######
3
● ovirt-imageio-daemon.service - oVirt ImageIO Daemon
4
   Loaded: loaded (/usr/lib/systemd/system/ovirt-imageio-daemon.service; enabled; vendor preset: disabled)
5
   Active: active (running) since Wed 2020-04-08 04:40:50 UTC; 2 weeks 1 days ago
6
 Main PID: 1442 (python)
7
    Tasks: 4
8
   CGroup: /system.slice/ovirt-imageio-daemon.service
9
           └─1442 /opt/ovirt-imageio/bin/python daemon/ovirt-imageio-daemon
Copied!
RHV 4.4.X
On the RHV-Manager check the ovirt-imageio-proxy service:
1
systemctl status ovirt-imageio
2
######
3
● ovirt-imageio.service - oVirt ImageIO Daemon
4
   Loaded: loaded (/usr/lib/systemd/system/ovirt-imageio.service; enabled; vend>
5
   Active: active (running) since Tue 2021-03-02 09:18:30 UTC; 5 months 11 days>
6
 Main PID: 1041 (ovirt-imageio)
7
    Tasks: 3 (limit: 100909)
8
   Memory: 22.0M
9
   CGroup: /system.slice/ovirt-imageio.service
10
           └─1041 /usr/libexec/platform-python -s /usr/bin/ovirt-imageio
Copied!
On the RHV-Host check the ovirt-imageio-daemon service:
1
systemctl status ovirt-imageio
2
######
3
● ovirt-imageio.service - oVirt ImageIO Daemon
4
   Loaded: loaded (/usr/lib/systemd/system/ovirt-imageio.service; enabled; vend>
5
   Active: active (running) since Tue 2021-03-02 09:01:57 UTC; 5 months 11 days>
6
 Main PID: 51766 (ovirt-imageio)
7
    Tasks: 4 (limit: 821679)
8
   Memory: 19.8M
9
   CGroup: /system.slice/ovirt-imageio.service
10
           └─51766 /usr/libexec/platform-python -s /usr/bin/ovirt-imageio
Copied!
Verify the NFS Volume is correctly mounted
TrilioVault mounts the NFS Backup Target to the TrilioVault Appliance and RHV-Hosts.
To verify those are correctly mounted it is recommended to do the following checks.
First df -h looking for /var/triliovault-mounts/<hash-value>
1
df -h
2
######
3
Filesystem                                      Size  Used Avail Use% Mounted on
4
devtmpfs                                         63G     0   63G   0% /dev
5
tmpfs                                            63G   16K   63G   1% /dev/shm
6
tmpfs                                            63G   35M   63G   1% /run
7
tmpfs                                            63G     0   63G   0% /sys/fs/cgroup
8
/dev/mapper/rhvh-rhvh--4.3.8.1--0.20200126.0+1  7.1T  3.7G  6.8T   1% /
9
/dev/sda2                                       976M  198M  712M  22% /boot
10
/dev/mapper/rhvh-var                             15G  1.9G   12G  14% /var
11
/dev/mapper/rhvh-home                           976M  2.6M  907M   1% /home
12
/dev/mapper/rhvh-tmp                            976M  2.6M  907M   1% /tmp
13
/dev/mapper/rhvh-var_log                        7.8G  230M  7.2G   4% /var/log
14
/dev/mapper/rhvh-var_log_audit                  2.0G   17M  1.8G   1% /var/log/audit
15
/dev/mapper/rhvh-var_crash                      9.8G   37M  9.2G   1% /var/crash
16
30.30.1.4:/rhv_backup                           2.0T  5.3G  1.9T   1% /var/triliovault-mounts/MzAuMzAuMS40Oi9yaHZfYmFja3Vw
17
30.30.1.4:/rhv_data                             2.0T   37G  2.0T   2% /rhev/data-center/mnt/30.30.1.4:_rhv__data
18
tmpfs                                            13G     0   13G   0% /run/user/0
19
30.30.1.4:/rhv_iso                              2.0T   37G  2.0T   2% /rhev/data-center/mnt/30.30.1.4:_rhv__iso
Copied!
Secondly do a read / write / delete test as the user vdsm:kvm (uid = 36 / gid = 36) from the TrilioVault Appliance and the RHV-Host.
1
su vdsm
2
######
3
[[email protected] MzAuMzAuMS40Oi9yaHZfYmFja3Vw]$ touch foo
4
[[email protected] MzAuMzAuMS40Oi9yaHZfYmFja3Vw]$ ll
5
total 24
6
drwxr-xr-x  3 vdsm kvm 4096 Apr  2 17:27 contego_tasks
7
-rw-r--r--  1 vdsm kvm    0 Apr 23 12:25 foo
8
drwxr-xr-x  2 vdsm kvm 4096 Apr  2 15:38 test-cloud-id
9
drwxr-xr-x 10 vdsm kvm 4096 Apr 22 11:00 workload_1540698c-8e22-4dd1-a898-8f49cd1a898c
10
drwxr-xr-x  9 vdsm kvm 4096 Apr  8 15:21 workload_51517816-6d5a-4fce-9ac7-46ee1e09052c
11
drwxr-xr-x  6 vdsm kvm 4096 Apr 22 11:30 workload_77fb42d2-8d34-4b8d-bfd5-4263397b636c
12
drwxr-xr-x  5 vdsm kvm 4096 Apr 23 06:15 workload_85bf16ed-d4fd-49a6-a753-98c5ca6e906b
13
[[email protected] MzAuMzAuMS40Oi9yaHZfYmFja3Vw]$ rm foo
14
[[email protected] MzAuMzAuMS40Oi9yaHZfYmFja3Vw]$ ll
15
total 24
16
drwxr-xr-x  3 vdsm kvm 4096 Apr  2 17:27 contego_tasks
17
drwxr-xr-x  2 vdsm kvm 4096 Apr  2 15:38 test-cloud-id
18
drwxr-xr-x 10 vdsm kvm 4096 Apr 22 11:00 workload_1540698c-8e22-4dd1-a898-8f49cd1a898c
19
drwxr-xr-x  9 vdsm kvm 4096 Apr  8 15:21 workload_51517816-6d5a-4fce-9ac7-46ee1e09052c
20
drwxr-xr-x  6 vdsm kvm 4096 Apr 22 11:30 workload_77fb42d2-8d34-4b8d-bfd5-4263397b636c
21
drwxr-xr-x  5 vdsm kvm 4096 Apr 23 06:15 workload_85bf16ed-d4fd-49a6-a753-98c5ca6e906b
22
[[email protected] MzAuMzAuMS40Oi9yaHZfYmFja3Vw]$
Copied!
Deployment Guide - Previous
Installation of RHV extensions
Next - Deployment Guide
Uninstall TrilioVault
Last modified 5mo ago
Export as PDF
Copy link
Contents
Verify the TrilioVault Appliance services are up
Check the TrilioVault pacemaker and nginx cluster
Verify API connectivity from the RHV-Manager
Verify the ovirt-imageio services are up and running
RHV 4.3.X
RHV 4.4.X
Verify the NFS Volume is correctly mounted