Migrating a Python Django DRF Monolith to Microservices

Now that we have containerized our microservices using Docker and tested them locally with Docker Compose, the next step is to deploy them to Kubernetes (K8s). Kubernetes is a powerful container orchestration tool that automates deployment, scaling, and management of containerized applications. This guide will walk you through setting up a Kubernetes cluster and deploying your Dockerized microservices to it.

In this part, we will:

Set up a Kubernetes cluster locally and/or on a cloud provider (AWS EKS for this guide).
Create Kubernetes manifests for deploying microservices.
Configure networking with an Nginx ingress controller.
Implement scalability and health checks.
Deploy the application step by step.

By the end of this guide, your microservices will be running on Kubernetes, ready for scaling and production workloads.

Step 1: Setting Up Kubernetes

1.1 Local Kubernetes Setup (Optional)

For local testing, you can set up a Kubernetes cluster using tools like Minikube or Docker Desktop.

Install Minikube:

Install Minikube:

 curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
 chmod +x minikube-linux-amd64
 sudo mv minikube-linux-amd64 /usr/local/bin/minikube

Start the cluster:
```
 minikube start
```
Verify installation:
```
 kubectl get nodes
```

1.2 Cloud Kubernetes Setup with AWS EKS

For production, we’ll use Amazon Elastic Kubernetes Service (EKS). AWS EKS simplifies Kubernetes cluster setup and management.

Step-by-Step Setup:

Install AWS CLI:

 curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
 unzip awscliv2.zip
 sudo ./aws/install

Install eksctl:

 curl --silent --location "https://github.com/weaveworks/eksctl/releases/download/latest_release/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
 sudo mv /tmp/eksctl /usr/local/bin

Create an EKS Cluster:

 eksctl create cluster --name trading-platform --region us-west-2 --nodes 2 --node-type t3.medium

This will create a managed cluster with two worker nodes.

Verify Cluster:
```
 kubectl get nodes
```

Step 2: Writing Kubernetes Manifests

Kubernetes manifests are YAML files that define how your containers are deployed and managed. Let’s create manifests for our microservices.

2.1 Deployment Manifests

A deployment specifies how to run your application, including the number of replicas and container specifications.

Example Deployment for User Service:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: user-service
  labels:
    app: user-service
spec:
  replicas: 3
  selector:
    matchLabels:
      app: user-service
  template:
    metadata:
      labels:
        app: user-service
    spec:
      containers:
      - name: user-service
        image: my-dockerhub/user-service:latest
        ports:
        - containerPort: 8000
        env:
        - name: DATABASE_URL
          value: "postgres://user:password@db:5432/user_service_db"
        readinessProbe:
          httpGet:
            path: /health
            port: 8000
          initialDelaySeconds: 5
          periodSeconds: 10
        livenessProbe:
          httpGet:
            path: /health
            port: 8000
          initialDelaySeconds: 10
          periodSeconds: 30

2.2 Service Manifests

A Service exposes your Deployment to other services within the cluster or externally.

Example Service for User Service:

apiVersion: v1
kind: Service
metadata:
  name: user-service
spec:
  selector:
    app: user-service
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8000
  type: ClusterIP

2.3 ConfigMap and Secret

ConfigMaps and Secrets store environment variables and sensitive data.

Example ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: user-service-config
data:
  DATABASE_URL: "postgres://user:password@db:5432/user_service_db"

Example Secret:

apiVersion: v1
kind: Secret
metadata:
  name: user-service-secret
type: Opaque
data:
  DATABASE_PASSWORD: cGFzc3dvcmQ=  # Base64 encoded value

2.4 Ingress Manifest

Ingress routes external HTTP/S traffic to services inside the cluster.

Install Nginx Ingress Controller:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/aws/deploy.yaml

Ingress Resource for User Service:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: trading-platform-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: trading-platform.example.com
    http:
      paths:
      - path: /users
        pathType: Prefix
        backend:
          service:
            name: user-service
            port:
              number: 80

Step 3: Deploying to Kubernetes

Apply Configurations:

Deploy all manifests to the cluster:

  kubectl apply -f user-service-deployment.yaml
  kubectl apply -f user-service-service.yaml
  kubectl apply -f user-service-config.yaml
  kubectl apply -f user-service-secret.yaml
  kubectl apply -f ingress.yaml

Verify Pods:
- Check if the pods are running:
```
  kubectl get pods
```

Test Service:

Access the service:

  curl http://trading-platform.example.com/users

Step 4: Implementing Scalability

Kubernetes enables horizontal scaling with Horizontal Pod Autoscalers (HPA).

HPA for User Service:

apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
  name: user-service-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: user-service
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      targetAverageUtilization: 50

Deploy HPA:

kubectl apply -f user-service-hpa.yaml

Step 5: Monitoring and Observability

Prometheus and Grafana:

Use Prometheus for metrics collection and Grafana for visualization.

Install via Helm:

  helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
  helm install prometheus prometheus-community/kube-prometheus-stack

Centralized Logging with Fluentd:
- Aggregate logs to a centralized logging system.
Jaeger for Tracing:
- Implement distributed tracing for inter-service requests.

Conclusion

Your microservices are deployed on Kubernetes with scalable, production-ready configurations. You’ve learned how to:

Set up a Kubernetes cluster locally or on AWS EKS.
Write Kubernetes manifests for deployments, services, and ingress.
Implement horizontal scaling and health checks.

Next Steps: we will focus on setting up CI/CD pipelines using GitLab to automate the deployment process for your Kubernetes cluster.

Migrating a Python Django DRF Monolith to Microservices: Part 3 - Deploying Dockerized Microservices to Kubernetes