Introduction

This script automates the process of updating Docker image tags in Helm values files by fetching the latest tags from DockerHub for multiple repositories. It interacts with DockerHub’s API, retrieves the latest version tags, and updates the corresponding YAML files located in a Helm chart.

The script ensures that Kubernetes deployments always use the latest image versions, which is crucial for maintaining up-to-date microservices in a Kubernetes environment.

Prerequisites

Before you can use this script, make sure you have

python3
pip install requests pyyaml
Docker and DockerHub Access Token
Create Docker-Secret:- Creating a Docker secret is necessary when you want to pull private Docker images from a Docker registry (like DockerHub or other container registries) in your Kubernetes environment. This is especially relevant if you are deploying services through Helm or directly with Kubernetes manifests and need access to private repositories.
```
  kubectl --kubeconfig=./file.yaml(kubeconfig file name) create secret docker-registry docker-secret \
    --docker-server=docker.io \
    --docker-username=your-username \
    --docker-password=your-password \
    --docker-email=your-email \
    --namespace=your-namespace name
```
Helm
Kubernetes Cluster (kubectl)

What is Helm?

Helm is an open-source package manager for Kubernetes, the popular container orchestration platform. Similar to yum but for Kubernetes. It bundles all related manifests(such as deployment, service, etc) into a chart. Helm helps you manage Kubernetes applications by providing a way to define, install, and upgrade complex Kubernetes applications.

In this context, we are using Helm to manage deployments of microservices, with each microservice’s configuration stored in a values.yaml file.

Why Use Helm in Kubernetes?

Helm simplifies Kubernetes deployments by:

Packaging all Kubernetes resources like Pods, Services, ConfigMaps, etc., into a single chart.
Allowing version control and rollback of Kubernetes applications.
Using values files to customize deployments easily.
Simplifying updates to applications by managing Kubernetes objects under a single release.

Helm allows us to maintain a standard deployment process, making it easier to modify, install, or update our applications in a Kubernetes environment.

Overview of the Script

This script performs the following tasks:

Fetch All Repositories from DockerHub: It uses the DockerHub API to fetch all repositories under a specific account.
Fetch the Highest dev-* Tag for Each Repository: For each repository, the script fetches the list of available tags and identifies the latest dev-* tag.
Update Helm Values File: The script looks for a Helm values.yaml file associated with the Docker repository and updates the image tag to the latest dev-* tag.
Success Confirmation: It prints out success messages for updated files.

Creating a Namespace for Helm and Helm Chart

A namespace in Kubernetes is a way to logically separate resources. You should create a dedicated namespace for your Helm release to isolate the application.

kubectl create namespace development
helm create project-01

Let’s cd into the generated chart directory.

We’ll edit the files one by one according to our deployment requirements.

There are multiple files in templates directory created by Helm. We can customize our helm-chart according to our projects.

Let’s remove all default files from the template directory.

rm -rf templates/*

Create a deployment.yaml file and copy the following contents.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ .Values.deployment.name }}
  namespace: {{ .Values.namespace }}
spec:
  replicas: {{ .Values.deployment.replicas }}
  selector:
    matchLabels:
      app: {{ .Values.deployment.appLabel }}
  template:
    metadata:
      labels:
        app: {{ .Values.deployment.appLabel }}
    spec:
      containers:
      - name: {{ .Values.deployment.containerName }}
        image: {{ .Values.image.repository }}:{{ .Values.image.tag }}  # This will pull the correct image and tag
        imagePullPolicy: {{ .Values.image.pullPolicy }}
        ports:
        - containerPort: {{ .Values.deployment.containerPort }}
      imagePullSecrets:
      - name: {{ .Values.imagePullSecret }}

---

apiVersion: v1
kind: Service
metadata:
  name: {{ .Values.service.name }}
  namespace: {{ .Values.namespace }}
spec:
  selector:
    app: {{ .Values.deployment.appLabel }}
  ports:
  - protocol: TCP
    port: {{ .Values.service.port }}
    targetPort: {{ .Values.service.targetPort }}
  type: ClusterIP

values

The values.yaml file contains all the values that need to be substituted in the template directives we used in the templates. For example, deployment.yaml template contains a template directive to get the image repository, tag, and pullPolicy from the values.yaml file.

Now, create a values directory and store all the values on the basis of your project.

deployment:
  appLabel: ******
  containerName: *********
  containerPort: **
  name: ********
  replicas: ***
image:
  pullPolicy: Always
  repository: *********
  tag: ****
imagePullSecret: ******
namespace: ******
service:
  name: ********
  port: **
  targetPort: **

Now we have the project-01 helm chart ready.

Validate the Helm Chart

Now to make sure that our chart is valid and, all the indentations are fine, we can run the below command. Ensure you are inside the chart directory.

helm lint .

If there is no error or issue, it will show this result

==> Linting ./project-01
[INFO] Chart.yaml: icon is recommended

1 chart(s) linted, 0 chart(s) failed

To validate if the values are getting substituted in the templates, you can render the templated YAML files with the values using the following command. It will generate and display all the manifest files with the substituted values.

helm template .

Deploy the Helm Chart

When you deploy the chart, Helm will read the chart and configuration values from the values.yaml file and generate the manifest files. Then it will send these files to the Kubernetes API server, and Kubernetes will create the requested resources in the cluster.

Execute the following command

helm install my-release  ./project-01 -f ./values -n development

Verify Deployment

kubectl get all -n development

Since we have many values files, manually updating each one every time the Docker image tag changes is very time-consuming. Therefore, we have a Python script that automatically updates the image tags in the values files whenever the Docker image is updated.

import requests
import yaml
import os
import sys

# DockerHub credentials
username = "write-your-dockerhub-username"
token = input("Enter your DockerHub Access Token: ")

# Path to your values directory
values_directory = "./values"

# Function to fetch the highest 'dev-*' tag for a repository
def fetch_highest_dev_tag(repo_name):
    url = f"https://hub.docker.com/v2/repositories/{username}/{repo_name}/tags"
    headers = {
        'Authorization': f'Bearer {token}'
    }
    response = requests.get(url, headers=headers, params={"page_size": 100})

    if response.status_code == 200:
        data = response.json()
        dev_tags = [tag['name'] for tag in data['results'] if tag['name'].startswith('dev-')]
        if dev_tags:
            highest_tag = max(dev_tags, key=lambda x: int(x.split('-')[1]))
            return highest_tag
        else:
            return None
    else:
        print(f"Failed to fetch tags for {repo_name}")
        return None

# Function to update the image tag in the values file
def update_values_file(repo_name, new_tag):
    # Construct the expected values filename based on the repository name
    values_file = os.path.join(values_directory, f"{repo_name}.yaml")  # Adjusted here

    if os.path.exists(values_file):
        with open(values_file, 'r') as file:
            values_data = yaml.safe_load(file)

        if 'image' in values_data and 'tag' in values_data['image']:
            old_tag = values_data['image']['tag']
            values_data['image']['tag'] = new_tag

            with open(values_file, 'w') as file:
                yaml.safe_dump(values_data, file)
            print(f"Successfully updated {values_file} from {old_tag} to {new_tag}.")
        else:
            print(f"No image.tag found in {values_file}")
    else:
        print(f"Values file for {repo_name} not found.")

# Function to fetch all repositories under the DockerHub account
def fetch_all_repositories():
    repositories = []
    page = 1
    while True:
        url = f"https://hub.docker.com/v2/repositories/{username}/"
        headers = {
            'Authorization': f'Bearer {token}'
        }
        response = requests.get(url, headers=headers, params={"page": page, "page_size": 100})

        if response.status_code == 200:
            data = response.json()
            repositories.extend([repo['name'] for repo in data['results']])

            if not data['next']:
                break
            page += 1
        else:
            break
    return repositories

# Main logic to update tags
def main():
    if len(sys.argv) == 1:
        # No arguments passed, update all repositories
        print("Updating all repositories...")
        repositories = fetch_all_repositories()

        for repo_name in repositories:
            highest_tag = fetch_highest_dev_tag(repo_name)
            if highest_tag:
                update_values_file(repo_name, highest_tag)

    elif len(sys.argv) == 2:
        # Single repository mode
        repo_name = sys.argv[1]
        print(f"Updating single repository: {repo_name}")
        highest_tag = fetch_highest_dev_tag(repo_name)
        if highest_tag:
            update_values_file(repo_name, highest_tag)
        else:
            print(f"No 'dev-*' tags found for {repo_name}")
    else:
        print("Usage: python3 script.py [repository_name]")
        sys.exit(1)

if __name__ == "__main__":
    main()

To run the script we should run following command

python3 script.py // for all repository
python3 script.py microservice-name // for single repository

Automating Docker Image Tag Updates in Helm Values Files with Python

Table of contents