Managing Kubernetes configurations across multiple environments (e.g., dev, uat, production) can quickly become complex. A structured approach is necessary to maintain consistency while allowing flexibility for environment-specific overrides.

In this blog, we'll explore how to efficiently manage different Kubernetes environments using Kustomize, paired with ArgoCD for GitOps-based deployments.

---

What is Kustomize?

Kustomize is a tool that allows you to manage Kubernetes configurations without modifying the original YAML files. Instead of creating multiple, slightly different YAML files for each environment, Kustomize provides a structured way to define a base configuration and apply overlays for each environment.

Key Concepts in Kustomize

• Base: Contains the common YAML configurations shared across all environments.

• Overlays: Define environment-specific customizations (e.g., different replica counts, service types, resource limits).

• Resources: List of YAML files included in the kustomization.

• Replacements: Allow variable substitution and patching of values across different YAML files.

---

Project Structure

For this setup, we use the following directory structure:

├── application.yaml  # ArgoCD Application definition
├── kustomize
│   ├── base
│   │   ├── example
│   │   │   ├── example.yaml
│   │   │   ├── kustomization.yaml
│   │   ├── example2
│   │   │   ├── example2.yaml
│   │   │   ├── kustomization.yaml
│   │   ├── namespace
│   │       ├── kustomization.yaml
│   │       ├── namespace.yaml
│   ├── overlays
│   │   ├── dev
│   │   │   ├── kustomization.yaml
│   │   ├── prod
│   │   │   ├── kustomization.yaml
│   │   ├── uat
│   │       ├── kustomization.yaml
│   ├── templates  # Reusable templates for common components

---

Base Configuration

Each service in kustomize/base has its own folder containing:

• example.yaml : Defines the Kubernetes objects (Deployment, Service, etc.).

• kustomization.yaml: References the YAML files for inclusion.

Example kustomize/base/example/kustomization.yaml:

resources:
  - example.yaml

---

Overlay Configuration

When deploying to a specific environment, the corresponding overlay in kustomize/overlays is used. To add a service to a particular environment (e.g., dev), update the kustomization.yaml file within the respective overlay directory.

resources:
  - ../../base/example
  - ../../base/example2
  - ../../base/namespace

namespace: dev

This allows services to be included or excluded dynamically from different environments.

---

Using Replacements for Environment-Specific Customization

Kustomize supports replacements to override values dynamically. For example, if the dev environment needs different hostnames, you can define replacements in overlays/dev/kustomization.yaml:

replacements:
- source:
    kind: Namespace
    fieldPath: metadata.name
  targets:
    - select:
        kind: Ingress
      fieldPaths:
        - spec.rules.0.host
      options:
        delimiter: "."
        index: 1

Here, the namespace name dynamically replaces part of the Ingress hostname.

Automating Service Onboarding with a CLI Tool

To further streamline the onboarding process, I developed a CLI tool that interacts with the `templates/` folder and generates service files dynamically by asking the user for inputs.

How the CLI Works

1. The user is prompted for service details (e.g., service name, port, replica count).

2. The tool uses predefined templates from `templates/` to create a new service definition.

3. The generated service files are placed under `kustomize/base/`.

4. The tool automatically updates the relevant `kustomization.yaml` files in the overlay directories.

Why This Matters?

Before the CLI, manually adding services required multiple steps, increasing the chances of misconfiguration. With the CLI, services can be onboarded in seconds while ensuring consistency across environments.

Using Patches for Customization

Kustomize allows you to define patches to modify existing configurations dynamically. For example, you can change replica counts, memory requests, and GPU limits without modifying the base YAML files. Unlike replacements, which copy values from one resource to another, patches are used to directly override or extend specific fields in a resource.

Example patch in kustomize/overlays/dev/kustomization.yaml:

patches:
  - patch: |-
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: example
      spec:
        replicas: 4
        template:
          spec:
            containers:
              - name: example
                resources:
                  requests:
                    memory: "6999Mi"
                  limits:
                    memory: "6888Mi"

This patch updates the following values:

- Replica count: Sets replicas: 4

- Memory requests & limits: Defines memory allocations for the container.

Deploying with ArgoCD

Once the Kustomize overlays are set up, ArgoCD manages the deployment to Kubernetes using GitOps principles.

ArgoCD Application Definition

Your application.yaml file in the root directory defines how ArgoCD syncs the configurations:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: dev
  namespace: argocd
spec:
  destination:
    server: 'https://kubernetes.default.svc'
  source:
    path: kustomize/overlays/dev
    repoURL: 'git@github.com:DomainName'
    targetRevision: v2
  project: default
  syncPolicy:
    automated:
      prune: false
      selfHeal: true

How ArgoCD Works with Kustomize

ArgoCD watches the kustomize/overlays/dev folder, which contains the environment-specific kustomization.yaml.

If a new service needs to be deployed, it must be referenced in overlays/dev/kustomization.yaml, regardless of whether it exists in base/.

Automated sync ensures that any updates in the Git repository are applied to the cluster.

Conclusion

By structuring Kubernetes configurations with Kustomize and managing deployments with ArgoCD, you gain:

• Consistency: Centralized base configurations.

• Flexibility: Overlays for environment-specific changes.

• Automation: GitOps-based continuous deployment.

This approach simplifies multi-environment deployments while maintaining clean and manageable Kubernetes configurations.

Here is the Example-Code to follow along.

Thank you :

Dear readers,

I want to express my heartfelt gratitude for your continuous support and engagement with my blog. Your readership and valuable feedback have been instrumental in shaping the content and direction of my writing. Thank you for being a part of this journey and for inspiring me to keep sharing my thoughts and ideas. Your presence means the world to me, and I am truly grateful for your unwavering support.

Warmest regards, Gaurav Ji Srivastava