T4K Integration with Observability Stack

The Observability Stack is a pre-packaged distribution for monitoring, logging and dashboard interfaces which can be installed into any existing Kubernetes cluster.

Introduction

The Observability Stack is a pre-packaged distribution for monitoring, logging, and dashboarding and can be installed into any existing Kubernetes cluster. It includes many of the most popular open-source observability tools with Prometheus, Grafana, Promtail**,** and Loki. The observability stack provides a straightforward, maintainable solution for analyzing server traffic and identifying potential deployment problems.

T4K Installation with Observability using Trilio Operator

To install the operator with observability enabled, run the latest helm chart with the following parameter set.

helm repo add triliovault-operator https://charts.k8strilio.net/trilio-stable/k8s-triliovault-operator
helm install tvm triliovault-operator/k8s-triliovault-operator --set observability.enabled=true

Observability Stack Configurable Parameters

The following table lists the configuration parameters of the observability stack

Parameter	Description	Default
observability.enabled	observability stack is enabled	false
observability.name	observability name for T4K integration	tvk-integration
observability.logging.loki.enabled	logging stack, loki is enabled	true
observability.logging.loki.fullnameOverride	name of the loki service	"loki"
observability.logging.loki.persistence.enabled	loki persistence storage enabled	true
observability.logging.loki.persistence.accessModes	loki persistence storage accessModes	ReadWriteOnce
observability.logging.loki.persistence.size	loki persistence storage size	10Gi
observability.logging.loki.config.limits_config.reject_old_samples_max_age	loki config, maximum accepted sample age before rejecting	168h
observability.logging.loki.config.table_manager.retention_period	loki config, how far back tables will be kept before they are deleted. 0s disables deletion.	168h
observability.logging.promtail.enabled	logging stack, promtail is enabled	true
observability.logging.promtail.fullnameOverride	name of the promtail service	"promtail"
observability.logging.promtail.config.clients.url	loki url for promtail integration	"http://loki:3100/loki/api/v1/push"
observability.monitoring.prometheus.enabled	monitoring stack, prometheus is enabled	true
observability.monitoring.prometheus.fullnameOverride	name of the prometheus service	"prom"
observability.monitoring.prometheus.server.enabled	prometheus server is enabled	true
observability.monitoring.prometheus.server.fullnameOverride	name of prometheus server service	"prom-server"
observability.monitoring.prometheus.server.persistentVolume.enabled	prometheus server with persistent volume is enabled	false
observability.monitoring.prometheus.kubeStateMetrics.enabled	prometheus kube state metrics is enabled	false
observability.monitoring.prometheus.nodeExporter.enabled	prometheus node exporter is enabled	false
observability.monitoring.prometheus.pushgateway.enabled	prometheus push gateway is enabled	false
observability.monitoring.prometheus.alertmanager.enabled	prometheus alert manager is enabled	false
observability.visualization.grafana.enabled	visualization stack, grafana is enabled	true
observability.visualization.grafana.adminPassword	grafana password for admin user	"admin123"
observability.visualization.grafana.fullnameOverride	name of grafana service	"grafana"
observability.visualization.grafana.service.type	grafana service type	"ClusterIP"

Check the observability stack configuration by running the following command:

kubectl get pods -n <install_ns>

promtail-2zpcv                                              1/1     Running            0          2m16s
grafana-554cb4f55-q4q59                                     3/3     Running            0          2m15s
prom-server-786b8cf897-nglhh                                2/2     Running            0          2m15s
k8s-triliovault-operator-85dfc877b8-5xqx9                   1/1     Running            0          2m15s
loki-0                                                      1/1     Running            0          2m15s
k8s-triliovault-admission-webhook-96db687bb-wnfh7           1/1     Running            0          62s
k8s-triliovault-control-plane-6b986c8fb9-zjbnj              2/2     Running            0          62s
k8s-triliovault-exporter-7b98cb7678-wxwvx                   1/1     Running            0          62s
k8s-triliovault-ingress-nginx-controller-57b777f45b-dnjkv   1/1     Running            0          62s
k8s-triliovault-web-85c79c9c4f-djqqz                        1/1     Running            0          62s
k8s-triliovault-web-backend-5c8c67c548-pcgvl                1/1     Running            0          62s

View Logs From T4K UI

1. Login to T4K UI with preferred authentication

2. Select "Launch Event Viewer" on any required service or application

Launch Event Viewer option

1. On click on "Launch Event Viewer" option, user will be redirected to Logs visibility page.

Logs page

Accessing Grafana Dashboards

Grafana Endpoint : http://<TVK_IP>/grafana 

Login with default grafana credentials.
username: admin
password: admin123

if a custom path is configured then:

Grafana Endpoint : http://<TVK_IP>/<custom-path>/grafana