Search…
Air-Gapped Install - Upstream Kubernetes
This page describes the instructions for deploying TVK in a network-restricted environment for Upstream or any Upstream compatible Kubernetes distribution.
If a local registry is available, then skip to step 2 directly
IMPORTANT - Please read the value of TVK_VERSION in this doc as "2.6.6" for the current release

Steps to install TVK in air-gapped upstream Kubernetes environment

There are 3 simple steps that a user needs to perform:
  1. 1.
    Step 1 - Setup Local Registry - Ref
    Deploy a registry server
  2. 2.
    Step 2 - Download TVK charts (k8s-triliovault-operator and k8s-triliovault) locally
  3. 3.
    Step 3 - Install TVK using one of the methods below
    • Directly install k8s-triliovault by referencing the helm chart downloaded locally
    • Use Helm Registry to store local charts and install the charts referring to this helm registry

Step 1 - Setup Local Registry

Setup Local Registry - Ref
Deploy a registry server
  1. 1.
    Set up local registry on Ubuntu 20.04
  2. 2.
    Install docker
  3. 3.
    Start docker registry (using one of the options below)
    • Unsecured registry
    docker run -d -p 5000:5000 --restart=always --name registry registry:2
    • A secured registry using TLS authentication (Valid certificates are required in “certs” dir)
    docker run -d --restart=always --name registry -v "$(pwd)"/certs:/certs -e REGISTRY_HTTP_ADDR=0.0.0.0:443 -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/skregistry.kulkarnisachin.ml.crt -e REGISTRY_HTTP_TLS_KEY=/certs/skregistry.kulkarnisachin.ml.key -p 443:443 registry:2
  4. 4.
    By default, the registry stores its data on the local filesystem, whether you use a bind mount or a volume. You can store the registry data in an Amazon S3 bucket, Google Cloud Platform, or on another storage back-end by using storage drivers.

Step 2 - Download TVK charts (k8s-triliovault-operator and k8s-triliovault) locally

Use one of the methods below

Using curl commands

1
curl http://charts.k8strilio.net/trilio-stable/k8s-triliovault-operator/charts/k8s-triliovault-operator-TVK_VERSION.tgz --output k8s-triliovault-operator-TVK_VERSION.tgz
2
curl http://charts.k8strilio.net/trilio-stable/k8s-triliovault/charts/k8s-triliovault-TVK_VERSION.tgz --output k8s-triliovault-TVK_VERSION.tgz
Copied!

Using Helm pull

1
helm repo add triliovault-operator http://charts.k8strilio.net/trilio-stable/k8s-triliovault-operator
2
helm repo add triliovault http://charts.k8strilio.net/trilio-stable/k8s-triliovault
3
helm repo update
4
helm pull triliovault-operator/k8s-triliovault-operator
5
helm pull triliovault/k8s-triliovault
Copied!

Step 3 - Install TVK

Upload TVK images to local registry

Use the script below to download and push images to local registry. Please edit the TVK_VERSION, list of images and tags required for given TVK version.
1
#!/bin/bash
2
3
## TVK version
4
tvk_version="<TVK_VERSION>"
5
6
## Update below details as required for the specific TVK version
7
## TVK images list for specific version TVK_VERSION
8
img_list="k8s-triliovault-operator operator-webhook-init backup-scheduler control-plane datamover datastore-attacher metamover exporter trilio-init trilio-admission-webhook backup-cleaner resource-cleaner web web-backend backup-retention target-browser hook-executor analyzer ingress-controller"
9
gcr_link="eu.gcr.io/amazing-chalice-243510"
10
## Edit the value of local_reg below to point to local registry
11
## e.g. local_reg="skregistry.kulkarnisachin.ml"
12
local_reg="<local registry url>"
13
14
for img in ${img_list}
15
do
16
echo "Adding $img"
17
docker pull ${gcr_link}/${img}:${tvk_version}
18
docker tag ${gcr_link}/${img}:${tvk_version} ${local_reg}/${img}:${tvk_version}
19
docker push ${local_reg}/${img}:${tvk_version}
20
done
21
22
## Pushing dex:v2.28.1 image as the version is different
23
docker pull ${gcr_link}/dex:v2.28.1
24
docker tag ${gcr_link}/dex:v2.28.1 ${local_reg}/dex:v2.28.1
25
docker push ${local_reg}/dex:v2.28.1
26
27
## List all the pkgs
28
docker image ls
Copied!
Individual commands to push the TVK images to local registry
  1. 1.
    Pull TVK operator image from
    Google Cloud Platform
  2. 2.
    1
    docker pull eu.gcr.io/amazing-chalice-243510/operator-webhook-init:TVK_VERSION
    Copied!
  3. 3.
    Tag the image
  4. 4.
    1
    docker tag eu.gcr.io/amazing-chalice-243510/operator-webhook-init:TVK_VERSION skregistry.kulkarnisachin.ml/operator-webhook-init:TVK_VERSION
    Copied!
  5. 5.
    Push the image
  6. 6.
    1
    docker push skregistry.kulkarnisachin.ml/operator-webhook-init:TVK_VERSION
    Copied!
  7. 7.
    Repeat the same for all the images required for TVK (listed below)
    1. 1.
      TVK operator images
      1. 1.
        1
        k8s-triliovault-operator
        2
        operator-webhook-init
        Copied!
    2. 2.
      TVK app images
      1. 1.
        1
        backup-scheduler
        2
        control-plane
        3
        datamover
        4
        datastore-attacher
        5
        metamover
        6
        exporter
        7
        dex:v2.28.1
        8
        trilio-init
        9
        trilio-admission-webhook
        10
        backup-cleaner
        11
        resource-cleaner
        12
        web
        13
        web-backend
        14
        backup-retention
        15
        target-browser
        16
        hook-executor
        17
        analyzer
        18
        ingress-controller
        Copied!

Directly install k8s-triliovault chart locally

Install TriliovaultManager by adding below values in k8s-triliovault/values.yaml or using "--set". Here the chart can be directly installed. However, future chart updates need to be managed manually.
1
## TrilioVault Manager values - Update relevant values and replace "TVK_VERSION" with actual version
2
registry: "skregistry.kulkarnisachin.ml"
3
applicationScope: "Clustered"
4
tag: "TVK_VERSION"
5
6
$ helm install local-tvk ./k8s-triliovault \
7
--set registry="skregistry.kulkarnisachin.ml" \
8
--set applicationScope="Clustered" \
9
--set tag="TVK_VERSION"
10
$ helm list
11
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
12
local-tvk default 1 2021-12-07 12:09:40.543186572 +0000 UTC deployed k8s-triliovault-TVK_VERSION TVK_VERSION
13
$
14
$ kubectl get triliovaultmanager
15
NAME TRILIOVAULT-VERSION SCOPE STATUS RESTORE-NAMESPACES
16
triliovault-manager TVK_VERSION Cluster
17
$
Copied!

Using local helm registry, it provides a way to access it on multiple systems by referring to local helm registry. Please note this is an alternative to method described in above step.

  1. 1.
    Enable Helm registry - Ref
    Helm | Registries
  2. 2.
    1
    export HELM_EXPERIMENTAL_OCI=1
    Copied!
  3. 3.
    Login to local helm registry and save the TVK charts in the local registry
    1. 1.
      1
      helm registry login --insecure http://skregistry.kulkarnisachin.ml
      2
      helm chart save k8s-triliovault-operator/ skregistry.kulkarnisachin.ml/tvk-op
      3
      helm chart save k8s-triliovault/ skregistry.kulkarnisachin.ml/tvk
      Copied!
    2. 2.
      You should see list of charts as below
    3. 3.
      1
      $ helm chart list
      2
      REF NAME VERSION DIGEST SIZE CREATED
      3
      skregistry.kulkarnisachin.ml/tvk-op:TVK_VERSION k8s-triliovault-operator TVK_VERSION 85ec8d2 8.7 KiB 38 seconds
      4
      skregistry.kulkarnisachin.ml/tvk:TVK_VERSION k8s-triliovault TVK_VERSION 85ec8d2 39.2 KiB 10 seconds
      5
      $
      Copied!
  4. 4.
    Now, these charts can be used on any cluster which has access to this local registry.
  5. 5.
    Create secret, daemonset to push certificates for all new pods/containers. This step is required when registry needs TLS certificates. These certificates will be needed on every new pod/container to authenticate with the registry to access the charts, images.
    1. 1.
      Copy the image “busybox” in the local registry.
    2. 2.
      1
      $ kubectl create secret generic registry-ca --namespace kube-system --from-file=registry-ca=./rootCA.crt
      2
      secret/registry-ca created
      3
      $
      4
      $ kubectl get secrets -A | grep registry
      5
      kube-system registry-ca Opaque 1 11s
      6
      $
      7
      $ cat registry-ca-ds.yaml
      8
      apiVersion: apps/v1
      9
      kind: DaemonSet
      10
      metadata:
      11
      name: registry-ca
      12
      namespace: kube-system
      13
      labels:
      14
      k8s-app: registry-ca
      15
      spec:
      16
      selector:
      17
      matchLabels:
      18
      name: registry-ca
      19
      template:
      20
      metadata:
      21
      labels:
      22
      name: registry-ca
      23
      spec:
      24
      containers:
      25
      - name: registry-ca
      26
      image: busybox
      27
      command: [ 'sh' ]
      28
      args: [ '-c', 'cp /home/core/registry-ca /etc/docker/certs.d/skregistry.kulkarnisachin.ml/ca.crt && exec tail -f /dev/null' ]
      29
      volumeMounts:
      30
      - name: etc-docker
      31
      mountPath: /etc/docker/certs.d/skregistry.kulkarnisachin.ml
      32
      - name: ca-cert
      33
      mountPath: /home/core
      34
      terminationGracePeriodSeconds: 30
      35
      volumes:
      36
      - name: etc-docker
      37
      hostPath:
      38
      path: /etc/docker/certs.d/skregistry.kulkarnisachin.ml
      39
      - name: ca-cert
      40
      secret:
      41
      secretName: registry-ca
      42
      $ kubectl create -f registry-ca-ds.yaml
      43
      daemonset.apps/registry-ca created
      44
      $
      45
      $ kubectl get ds -A
      46
      NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
      47
      cattle-system cattle-node-agent 6 6 6 6 6 <none> 26m
      48
      cattle-system kube-api-auth 3 3 3 3 3 <none> 26m
      49
      ingress-nginx nginx-ingress-controller 3 3 3 3 3 <none> 26m
      50
      kube-system canal 6 6 6 6 6 kubernetes.io/os=linux 27m
      51
      kube-system registry-ca 3 3 3 3 3 <none> 22s
      52
      $
      Copied!
  6. 6.
    Install TVK operator by adding local registry in the k8s-triliovault-operator/values.yaml or using "--set". Here the chart can be directly installed from local registry or downloaded from it
  7. 7.
    1
    ## TrilioVault Operator - Update relevant values and replace "TVK_VERSION" with actual version
    2
    registry: "skregistry.kulkarnisachin.ml"
    3
    tag: "TVK_VERSION"
    4
    tvkHelmRepo: skregistry.kulkarnisachin.ml
    5
    6
    $ helm install local-tvk-op ./k8s-triliovault-operator \
    7
    --set registry="skregistry.kulkarnisachin.ml" \
    8
    --set tag="TVK_VERSION" \
    9
    --set tvkHelmRepo="skregistry.kulkarnisachin.ml"
    10
    $ helm list
    11
    NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
    12
    local-tvk-op default 1 2021-12-07 12:09:40.543186572 +0000 UTC deployed k8s-triliovault-operator-TVK_VERSION TVK_VERSION
    13
    $
    Copied!
  8. 8.
    Install TriliovaultManager using the manifest for helm
  9. 9.
    1
    $ cat triliovault-manager.yaml
    2
    apiVersion: triliovault.trilio.io/v1
    3
    kind: TrilioVaultManager
    4
    metadata:
    5
    labels:
    6
    triliovault: triliovault
    7
    name: triliovault-manager
    8
    namespace: default
    9
    spec:
    10
    trilioVaultAppVersion: TVK_VERSION
    11
    helmVersion:
    12
    version: v3
    13
    applicationScope: Cluster
    14
    $
    15
    $ kubectl create -f triliovault-manager.yaml
    16
    triliovaultmanager.triliovault.trilio.io/triliovault-manager created
    17
    $ kubectl get triliovaultmanager
    18
    NAME TRILIOVAULT-VERSION SCOPE STATUS RESTORE-NAMESPACES
    19
    triliovault-manager TVK_VERSION Cluster
    20
    $
    Copied!
TVK installation is complete. Continue using it following Getting Started Guide.
Last modified 3d ago