Creating a Custom VM Image for Recovery VM

📘 Introduction

This guide walks you through the process of building a custom Virtual Machine (VM) image specifically tailored for file recovery operations using TrilioVault for Kubernetes (TVK). Creating your own image ensures compatibility, security, and operational consistency within your environment.

⚙️ Prerequisites

Before building the FileRecovery image, ensure the following prerequisites are met:

  • CentOS 8 Base Image: A CentOS 8 Stream-based disk image (.qcow2 or .img) must be available either:

    • Locally on the build system, or

    • Accessible via a public URL, such as:

      curl -L -o /disk.img https://cloud.centos.org/centos/8-stream/$(uname -m)/images/CentOS-Stream-GenericCloud-8-latest.$(uname -m).qcow2

    💡 You may use any CentOS 8-compatible image. If using a different variant, ensure that installation commands in the customize.sh script are adjusted accordingly.

  • Docker Installed: Docker is required for building and managing the custom image. Ensure that Docker and its associated tools are installed and functioning on your system.

📁 Create Required Directory Structure

You can create the working directory anywhere you prefer (e.g., ~/trilio, /home/user/trilio, etc.). Let’s say you want to create it under ~/trilio:

mkdir -p ~/trilio/docker-images/filerecovery-vm
cd ~/trilio/docker-images/filerecovery-vm

📁 Your Initial directory structure looks like:

~/trilio/docker-images/
└── filerecovery-vm/

1️⃣ Create the Dockerfile

Create a file named Dockerfile in the filerecovery-vm directory:

📌 Note: Use the sample Dockerfile below as a template. Copy its contents into your Dockerfile and adjust according to your environment.

📁 Your directory now looks like:

~/trilio/docker-images/
└── filerecovery-vm/
    └── Dockerfile

🧠 Dockerfile Explanation

This Dockerfile uses fedora:35 as the builder image to create a customized FileRecoveryVM disk image.

We start by downloading a CentOS 8 Stream base image using the following curl command:

curl -L -o /disk.img https://cloud.centos.org/centos/8-stream/$(uname -m)/images/CentOS-Stream-GenericCloud-8-latest.$(uname -m).qcow2

⚠️ Note: Trilio uses the CentOS 8 image available at the above URL for convenience, but you can use any CentOS 8-based image of your choice, as long as it is compatible with the required configurations.

To install required dependencies into the downloaded CentOS image, we use the virt-customize tool. To enable this tool, the builder stage installs necessary libguestfs packages, including:

  • libguestfs

  • guestfs-tools

Trilio provides a customize.sh script. This script is used to configure the disk image with all required components.

For FileRecoveryVM to function properly, several essential files must be copied into the CentOS disk image:

  1. datastore-attacher – A Trilio utility required for mounting trilio targets inside the VirtualMachine

  2. filerecovery-vm-ssh-server.tar – A tarball containing the SSH server image, which enables remote access to mounted VM backups

  3. cleanup.service – A systemd service that ensures clean removal of temporary resources Trilio creates during recovery operations inside VirtualMachine

📎 Scroll down to view the complete Dockerfile or jump to Dockerfile

2️⃣ Create the customize.sh Script

Next, you'll create the customize.sh script, which is responsible for installing all required dependencies and preparing the CentOS 8 disk image for FileRecovery operations.

📁 Your directory now looks like:

bashCopyEdit<your-path>/docker-images/
└── filerecovery-vm/
    ├── Dockerfile
    └── customize.sh

🧠 customize.sh Explanation

The customize.sh script plays a crucial role in preparing the CentOS 8 disk image for FileRecoveryVM. It is used to install all the required dependencies inside the downloaded CentOS 8 qcow2 image (/disk.img in our case).

This script includes a comprehensive set of package installations and configurations that are essential for Trilio’s FileRecoveryVM functionality.

To apply these changes to the image, we use the virt-customize tool. This tool copies the customize.sh script into the CentOS 8 disk image and executes it in the context of the image, ensuring all dependencies are installed directly into the filesystem.

⚠️ Note: The customize.sh script is specifically written for CentOS 8. If you choose to use a different flavor of CentOS 8 or another compatible base image, you may need to modify the package manager commands or package names accordingly. The end goal remains the same — to ensure all required FileRecoveryVM dependencies are properly installed in the image.

📎 Scroll down to view the complete customize.sh script or jump to customize.sh

3️⃣ Create the cleanup.service Script

Next, you'll create the cleanup.service script, which is responsible for cleaning up temporary resources—such as qemu-nbd devices and directory structures—created during FileRecovery operations. This service ensures proper cleanup upon VM shutdown or recovery completion.

📁 Your directory now looks like:

bashCopyEdit<your-path>/docker-images/
└── filerecovery-vm/
    ├── Dockerfile
    └── customize.sh
    └── cleanup.service

📎 Scroll down to view the complete cleanup.service script or jump to cleanup.service

4️⃣ Download the FileRecoveryVM SSH Server Image Tar

We have hosted the FileRecoveryVM SSH Server Docker image at the following location:

quay.io/triliodata/filerecovery-vm-ssh-server:5.1.0

To proceed, download the image locally using the docker pull command and save it as a tar archive named filerecovery-vm-ssh-server.tar using the docker save command:

docker pull quay.io/triliodata/filerecovery-vm-ssh-server:5.1.0
docker save quay.io/triliodata/filerecovery-vm-ssh-server:5.1.0 -o filerecovery-vm-ssh-server.tar

This will create the tarball required for building the custom FileRecoveryVM image.

📁 After completing this step, your directory structure should look like:

pgsqlCopyEdit<your-path>/docker-images/
└── filerecovery-vm/
    ├── Dockerfile
    ├── customize.sh
    └── filerecovery-vm-ssh-server.tar

5️⃣ Build and Export the Docker Image

We’ve hosted the base image required for this build—datastore-attacher—on: 📦 quay.io/triliodata/datastore-attacher:5.1.0

This build also requires a secret environment variable: 🔐 S3_FUSE_PIP_INDEX_URL — https://pypi.fury.io/t4k/

✅ Run the Docker Build

Use the following command to build and push the image for the linux/amd64 platform:

docker build --push --platform linux/amd64 \
  -t $(DOCKER_REGISTRY)/$(PROJECT)/$(IMAGE):$(TAG) \
  -f Dockerfile ../../ \
  --build-arg BASEIMAGE=quay.io/triliodata/datastore-attacher:5.1.0 \
  --secret type=env,id=S3_FUSE_PIP_INDEX_URL

📌 Please set the following environment variables based on where you prefer to host the image:

  • DOCKER_REGISTRY – e.g., quay.io/your-org or docker.io/your-username

  • PROJECT – logical group or team name (optional)

  • IMAGE – your image name, e.g., filerecovery-vm

  • TAG – image version/tag, e.g., 5.1.0

💡 Example:

export DOCKER_REGISTRY=quay.io/my-org
export PROJECT=filerecovery
export IMAGE=filerecovery-vm
export VERSION=5.1.0
export S3_FUSE_PIP_INDEX_URL=https://pypi.fury.io/t4k/

🧹 Cleanup

Once the build is complete, you may remove the downloaded image tar file:

rm -rf filerecovery-vm-ssh-server.tar

✅ You're now ready to use this image for your FileRecovery VM workflow.

📦 Using the Built Image for FileRecovery

Once the FileRecovery image is successfully built and pushed to a container registry, you can use it to create a DataVolume in your Kubernetes cluster. This DataVolume serves as the disk for the FileRecoveryVM and is created using the built image URL.

Below is a sample DataVolume manifest:

apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata:
  name: trilio-default-dv
spec:
  source:
    registry:
      url: '<image-url>'  # Replace with your actual image URL (e.g., quay.io/your-org/file-recovery-vm:latest)
  storage:
    accessModes:
      - ReadWriteOnce
    resources:
      requests:
        storage: 30Gi

📝 Note: Be sure to replace <image-url> with the actual image URL used in your Dockerfile or wherever the image was pushed (e.g., quay.io/trilio/file-recovery-vm:latest).

This DataVolume can then be referenced by the FileRecoveryVM to initiate recovery operations.

📎 Appendix

🧱 Dockerfile

ARG BASEIMAGE

FROM ${BASEIMAGE} AS datastoreattacher

# Use fedora as the builder base image
FROM fedora:35 AS builder

# Define build argument
ARG SSH_SERVER_IMAGE

# Set environment variables
ENV LIBGUESTFS_BACKEND=direct

# Download the CentOS image
RUN curl -L -o /disk.img https://cloud.centos.org/centos/8-stream/$(uname -m)/images/CentOS-Stream-GenericCloud-8-latest.$(uname -m).qcow2

# Install necessary tools
RUN dnf install -y libguestfs guestfs-tools curl passt

# Copy the local customize.sh script into the Docker image
COPY docker-images/filerecovery-vm/customize.sh /tmp/customize.sh

# Copy the local cleanup.service file into the Docker image to ensure the system is cleaned up regularly
# This service is responsible for removing NDB devices before shutdown, ensuring a clean state for the system
COPY docker-images/filerecovery-vm/cleanup.service /tmp/cleanup.service

# Copy the filerecovery vm ssh server image tar
COPY docker-images/filerecovery-vm/filerecovery-vm-ssh-server.tar /tmp/filerecovery-vm-ssh-server.tar

# Copy the datastore-attacher code
COPY --from=datastoreattacher /opt/tvk/datastore-attacher /tmp/datastore-attacher

# Make the script executable and run it using virt-customize
RUN --mount=type=secret,id=S3_FUSE_PIP_INDEX_URL,env=S3_FUSE_PIP_INDEX_URL \
    virt-customize -a /disk.img \
    --copy-in /tmp/customize.sh:/tmp \
    --copy-in /tmp/cleanup.service:/tmp \
    --run-command "export SSH_SERVER_IMAGE=${SSH_SERVER_IMAGE} \
      export S3_FUSE_PIP_INDEX_URL=${S3_FUSE_PIP_INDEX_URL} \
      && chmod +x /tmp/customize.sh \
      && /tmp/customize.sh && rm /tmp/customize.sh" \
    --copy-in /tmp/datastore-attacher:/opt/tvk \
    --copy-in /tmp/filerecovery-vm-ssh-server.tar:/opt/tvk/filerecovery-vm-ssh-server-image \
    --selinux-relabel -v -x

# Optimize the disk image
RUN virt-sparsify --in-place /disk.img

FROM registry.access.redhat.com/ubi8/ubi:8.6

LABEL name=trilio-recovery-vm-ssh-server
LABEL vendor=Trilio\ Data\ Inc.
LABEL version=VERSION
LABEL release=RELEASE
LABEL summary=TrilioVault\ Recovery\ VM
LABEL description=TrilioVault\ Recovery\ VM
LABEL maintainer=Trilio\ Data\ Inc.

# Copy the customized disk image from the builder stage
COPY --from=builder /disk.img /disk/disk.img

🛠️ Customize.sh

#!/bin/bash

set -ex

# Update repository configuration
sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-*
sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-*


# Update all packages
yum update -y

# Install necessary packages
yum install -y epel-release
yum install -y python3 python3-pip python3-setuptools tzdata librbd1 lvm2 yum-utils qemu-img ntfs-3g

# Upgrade pip and setuptools
pip3 install -U pip setuptools

# Install Python packages
pip3 install --ignore-installed PyYAML
pip3 install jsonformatter

pip3 install 's3fuse==4.0.107' --extra-index-url "${S3_FUSE_PIP_INDEX_URL}"

# Clean up
yum clean all

# removing the default `qemu-img` binary because we will be using our custom-built `qemu-img` binary
rm -rfv /bin/qemu-img

# Download qemu-img binary and set permissions
mkdir -p /bin/qemu/
cd /bin/qemu || exit

curl -LO https://storage.googleapis.com/trilio-rpms/"$(uname -m)"/qemu-img
chmod +x qemu-img
cp qemu-img /bin/

# Add Docker CE repository and install Docker
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
yum install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

# Start Docker service (Note: This might not work in a non-systemd container)
systemctl enable docker
systemctl start docker
systemctl restart docker

# We should not create `trilio` user via script as ideally, cloudInit script is supposed to create and manage the user
# permissions. Therefore, we cannot add the user to the docker group here either. This will be handled by cloudInit script too.

# creating directories for mount
mkdir -p /triliodata
mkdir -p /triliodata-temp

# creating directories for datastore-attacher
mkdir -p /opt/tvk

# creating directory for cloudInit generated data
mkdir -p /opt/tvk/cloudinit-data

# creating directories for storing filerecovery-vm-ssh-server image tar
mkdir -p /opt/tvk/filerecovery-vm-ssh-server-image

curl -Lo /opt/tvk/minio https://storage.googleapis.com/trilio-rpms/"$(uname -m)"/minio && \
    chmod +x /opt/tvk/minio


# Persist environment variables
cat <<EOT >> /etc/profile.d/custom_env.sh
export PYTHONPATH=/opt/tvk/datastore-attacher
EOT

# Ensure the script is executable
chmod +x /etc/profile.d/custom_env.sh

# Enable the cleanup service to run on boot, ensuring the system is cleaned up regularly
cp /tmp/cleanup.service /etc/systemd/system/cleanup.service
systemctl daemon-reload
systemctl enable cleanup.service

🧹 Cleanup Service

[Unit]
Description=Remove NDB devices before shutdown
DefaultDependencies=no
Before=shutdown.target reboot.target halt.target

[Service]
WorkingDirectory=/home/trilio
Environment="PYTHONPATH=/opt/tvk/datastore-attacher"
ExecStart=/usr/bin/python3 /opt/tvk/datastore-attacher/scripts/vm_qcow2_mount.py --action=unmount
Type=oneshot
RemainAfterExit=true

[Install]
WantedBy=shutdown.target reboot.target halt.target

⚠️ Note: When using a custom VM image for FileRecovery, TVK will perform a validation check on the image before proceeding. Since a user has selected a custom VM image, The first step TVK will take is to initiate the image validation process. This process checks if all necessary dependencies are present in the image.

PreviousDownload Files from the Recovery VMNextS3 as Backup Target

Last updated

Was this helpful?