How to Set Up Dynamic Domain Mapping with Traefik on GCP

Introduction

In this guide, we will walk through the steps to set up a GCP instance, install Docker, Docker Compose, and Traefik, and then use a Flask application to dynamically create reverse proxies for domain mapping to container images. Additionally, we will cover how to map a domain using Google Cloud DNS.

Prerequisites

Before we begin, ensure you have the following:

• A Google Cloud Platform (GCP) account

• Basic knowledge of GCP, Docker, and Flask

• A domain name for mapping

Step 1: Set Up a GCP Instance

Create a GCP Project

1. Go to the GCP Console: Google Cloud Console.

2. Create a new project:

   • Click on the project dropdown at the top of the page.

   • Click on "New Project".

   • Enter a project name and select your billing account.

   • Click "Create".

Create a VM Instance

1. Navigate to the "Compute Engine" section:

   • Click on the hamburger menu (three horizontal lines) in the top-left corner.

   • Select "Compute Engine" > "VM instances".

2. Create a new instance:

   • Click "Create Instance".

   • Configure the instance with the following settings:

     • Name: flask-traefik-instance

     • Region: Select a region close to you

     • Machine type: e2-medium or higher

     • Boot disk: Ubuntu 20.04 LTS

   • Click "Create" to launch the instance.

3. Connect to the VM:

   • Once the instance is running, click "SSH" to connect to the VM.

Step 2: Install Docker

1. Update the package list:

    sudo apt update
   

2. Install Docker:

    sudo apt install -y docker.io
    sudo systemctl start docker
    sudo systemctl enable docker
   

3. Add your user to the Docker group:

    sudo usermod -aG docker ${USER}
    newgrp docker
   

Step 3: Install Docker Compose

1. Download Docker Compose:

    sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
   

2. Apply executable permissions:

    sudo chmod +x /usr/local/bin/docker-compose
   

3. Verify installation:

    docker-compose --version
   

Step 4: Set Up Traefik

1. Create a Docker Compose file for Traefik

   Create a docker-compose.yml file with the following content:

    version: '3.8'
   
    services:
      traefik:
        container_name: traefik
        image: 'traefik:v2.10'
        restart: unless-stopped
        network_mode: host
        extra_hosts:
          - 'host.docker.internal:host-gateway'
        volumes:
          - '/var/run/docker.sock:/var/run/docker.sock:ro'
          - '/data/traefik:/traefik'
        command:
          - '--ping=true'
          - '--ping.entrypoint=http'
          - '--api.dashboard=true'
          - '--api.insecure=false'
          - '--entrypoints.http.address=:80'
          - '--entrypoints.https.address=:443'
          - '--entrypoints.http.http.encodequerysemicolons=true'
          - '--entryPoints.http.http2.maxConcurrentStreams=50'
          - '--entrypoints.https.http.encodequerysemicolons=true'
          - '--entryPoints.https.http2.maxConcurrentStreams=50'
          - '--providers.docker.exposedbydefault=false'
          - '--providers.file.directory=/traefik/dynamic/'
          - '--providers.file.watch=true'
          - '--certificatesresolvers.letsencrypt.acme.httpchallenge=true'
          - '--certificatesresolvers.letsencrypt.acme.storage=/traefik/acme.json'
          - '--certificatesresolvers.letsencrypt.acme.httpchallenge.entrypoint=http'
          - '--providers.docker=true'
        labels:
          - traefik.enable=true
          - traefik.http.routers.traefik.entrypoints=http
          - traefik.http.routers.traefik.service=api@internal
          - traefik.http.services.traefik.loadbalancer.server.port=8080
   

2. Start Traefik:

    sudo docker-compose up -d
   

Step 5: Set Up Flask for Dynamic Domain Mapping

1. Create a Flask application

   Create a directory for your Flask application and navigate into it:

    mkdir flask-traefik
    cd flask-traefik
   

2. Create amain.py file with the following content:

    from flask import Flask, request, jsonify
    from pydantic import BaseModel
    from flask_pydantic import validate
    import os
    import yaml
   
    app = Flask(__name__)
   
    class TraefikConfig(BaseModel):
        domain: str
        port: int
        app_name: str
   
        class Config:
            schema_extra = {
                "example": {
                    "domain": "newdomain.yourdomain.com",
                    "port": 4000,
                    "app_name": "myapp"
                }
            }
   
    DYNAMIC_CONFIG_DIR = os.getenv("TRAEFIK_DYNAMIC_CONFIG_DIR", "/data/traefik/dynamic/")
    DOCKER_NETWORK_IP = os.getenv("DOCKER_NETWORK_IP", "host.docker.internal")
   
    def reload_traefik():
        os.system("docker restart traefik")
   
    def add_domain_to_traefik(config: TraefikConfig):
        dynamic_config_path = os.path.join(DYNAMIC_CONFIG_DIR, f"{config.domain}.yaml")
   
        dynamic_config = {
            "http": {
                "middlewares": {
                    "redirect-to-https": {
                        "redirectscheme": {
                            "scheme": "https"
                        }
                    },
                    "gzip": {
                        "compress": True
                    }
                },
                "routers": {
                    f"{config.app_name}-http": {
                        "middlewares": ["redirect-to-https"],
                        "entryPoints": ["http"],
                        "service": config.app_name,
                        "rule": f"Host(`{config.domain}`)"
                    },
                    f"{config.app_name}-https": {
                        "entryPoints": ["https"],
                        "service": config.app_name,
                        "rule": f"Host(`{config.domain}`)",
                        "tls": {
                            "certresolver": "letsencrypt"
                        }
                    }
                },
                "services": {
                    config.app_name: {
                        "loadBalancer": {
                            "servers": [
                                {"url```yaml
                                    f"http://{DOCKER_NETWORK_IP}:{config.port}"}
                            ]
                        }
                    }
                }
            }
        }
   
        with open(dynamic_config_path, 'w') as f:
            yaml.dump(dynamic_config, f, default_flow_style=False)
   
        reload_traefik()
   
    @app.route("/add-domain/", methods=["POST"])
    @validate()
    def add_domain(body: TraefikConfig):
        try:
            add_domain_to_traefik(body)
            return jsonify({"message": "Domain added successfully!"})
        except Exception as e:
            return jsonify({"error": str(e)}), 500
   
    if __name__ == "__main__":
        app.run(host="0.0.0.0", port=8000)
   

3. Create aDockerfile for your Flask app:

    FROM python:3.8-slim
   
    WORKDIR /app
   
    COPY . /app
   
    RUN pip install flask pydantic flask-pydantic pyyaml
   
    CMD ["python", "main.py"]
   

4. Build and run the Flask app:

    sudo docker build -t flask-traefik-app .
    sudo docker run -d --name flask-traefik-app -e TRAEFIK_DYNAMIC_CONFIG_DIR=/data/traefik/dynamic/ -e DOCKER_NETWORK_IP=host.docker.internal -p 8000:8000 flask-traefik-app
   

Step 6: Configure Domain Mapping with Google Cloud DNS

Set Up Cloud DNS

1. Navigate to Cloud DNS:

   • Click on the hamburger menu (three horizontal lines) in the top-left corner.

   • Select "Network Services" > "Cloud DNS".

2. Create a DNS Zone:

   • Click "Create Zone".

   • Enter a zone name and a DNS name (e.g., yourdomain.com).

   • Click "Create".

3. Add DNS Records:

   • Click on your newly created zone.

   • Add an "A" record to point your domain to your GCP instance's external IP address.

     • DNS Name: yourdomain.com

     • Resource Record Type: A

     • TTL: 300

     • IPv4 Address: <your-instance-external-ip>

Step 7: Add Domain Using Flask

1. Access the Flask App:

   Open your browser and navigate to http://<your-instance-ip>:8000.

2. Use the/add-domain/ Endpoint:

   You can use tools like Postman or CURL to interact with the Flask app's API. Here's an example using CURL:

    curl -X POST "http://<your-instance-ip>:8000/add-domain/" -H "Content-Type: application/json" -d '{
        "domain": "yourdomain.com",
        "port": 4000,
        "app_name": "myapp"
    }'
   

   Replace yourdomain.com with your actual domain, 4000 with the port your application is running on, and myapp with the name of your application.

Conclusion

By following these steps, you have successfully set up a GCP instance with Docker, Docker Compose, and Traefik. You have also created a Flask application that dynamically manages reverse proxy configurations for domain mapping to container images.

This setup provides a flexible and scalable solution for managing multiple applications with different domains on a single server. Feel free to customize the configurations and expand on this setup to suit your specific needs.