5. Building Your Own Git Powerhouse: Mirror GitHub to Local Gitea & Trigger Jenkins on Every Commit

bashyamImagine having your own private Git server, a local copy of your GitHub repositories, and automated Jenkins builds triggered on every commit. Sounds powerful, right? In this comprehensive guide, we'll walk you through setting up a local Gitea server on your MicroK8s cluster, mirroring your GitHub repositories, and configuring Jenkins to trigger builds on every commit. No prior knowledge is needed – let's get started!

Why a Local Git Server?

• Control: You have complete control over your code.

• Speed: Local operations are often faster.

• Security: Keep sensitive code behind your firewall.

• Offline Access: Access your repositories even without internet.

• Integration: Seamlessly integrate with your local Jenkins instance.

1. Setting Up the Local Git Server (Gitea)

We'll use Gitea, a lightweight and easy-to-deploy Git service, within your MicroK8s environment.

a. Deploying Gitea using Helm

Helm simplifies the deployment of applications on Kubernetes.

• Add the Gitea Helm repository:

  Bash

    helm repo add gitea-charts https://dl.gitea.com/charts/
    helm repo update
  

• Create a Gitea namespace:

  Bash

    kubectl create namespace gitea
  

• Install Gitea:

  Bash

    helm install gitea gitea-charts/gitea -n gitea
  

• Access Gitea:

  • Once deployed, access Gitea through your configured domain or IP address.

  • Complete the initial Gitea setup through the web UI.

• Exposing Gitea via NodePort:

  After installing gitea using helm, you may want to expose the gitea service using a nodeport.

  • To do this, edit the gitea-http service yaml.

Bash

    kubectl edit service gitea-http -n gitea

• Change the service type from ClusterIP to NodePort or simply replace it with the below YAML file.

YAML

    apiVersion: v1
    kind: Service
    metadata:
      name: gitea-http
      namespace: default
    spec:
      ports:
      - name: http
        port: 3000
        protocol: TCP
        targetPort: 3000
        nodePort: 31803 # add nodePort
      selector:
        app.kubernetes.io/instance: gitea
        app.kubernetes.io/name: gitea
      type: NodePort # change type from ClusterIP to NodePort

• Now you can access Gitea using 192.168.0.20:31803.

b. Mirroring GitHub Repositories

Now, let's connect Gitea to your GitHub account.

• Create a GitHub Personal Access Token (PAT):

  1. Go to your GitHub account settings.

  2. Click on "Developer settings."

  3. Select "Personal access tokens" and then "Tokens (classic)."

  4. Click "Generate new token (classic)."

  5. Give your token a descriptive name.

  6. Select the repo scope (or repo:public_repo for public repos).

  7. Click "Generate token."

  8. Important: Copy the generated token immediately and store it securely.

• Configure a Repository Mirror in Gitea:

  1. Access Gitea at 192.168.0.20:31803.

  2. Create an empty migration in Gitea that will mirror the GitHub repository.

  3. Remote URL: Enter the HTTPS URL of the GitHub repository (e.g., https://github.com/YOUR_GITHUB_USERNAME/YOUR_REPO_NAME.git).

  4. Authentication: Select "Password."

  5. Username: Enter your GitHub username.

  6. Password: Paste the GitHub PAT.

  7. Sync Interval: Set the frequency (e.g., "10m").

• Verify the Mirror:

  • Gitea will synchronize the repository.

  • Check the mirror's status on the "Mirrors" page.

  • View the files from your GitHub repository in your Gitea repository.

2. Connecting Gitea to Jenkins

Now, let's connect your local Gitea server with your Jenkins service on MicroK8s.

• Install Required Jenkins Plugins:

  • Git Plugin: Essential for Git interactions.

  • Gitea Plugin (Recommended): Provides better Gitea integration.

  • Go to Jenkins > Manage Jenkins > Manage Plugins.

  • Search for and install the plugins.

  • Restart Jenkins.

• Create Jenkins Credentials for Gitea:

  1. Go to Jenkins > Manage Jenkins > Manage Credentials.

  2. Click "(global)."

  3. Click "Add Credentials."

  4. Kind: Choose "Username with password."

  5. Username: Enter the Gitea username.

  6. Password: Enter the Gitea password or a PAT with repo scope.

  7. ID: Give the credentials a meaningful ID (e.g., gitea-credentials).

• Create a Jenkins Pipeline Job:

  1. Go to your Jenkins dashboard and click "New Item."

  2. Enter a name (e.g., my-gitea-pipeline).

  3. Select "Pipeline" and click "OK."

• Configure the Pipeline:

  • Pipeline Definition: Select "Pipeline script from SCM."

  • SCM: Choose "Git."

  • Repository URL: Enter the Gitea repository URL (e.g., http://192.168.0.20:31803/your-gitea-user/your-repo.git).

  • Credentials: Select the gitea-credentials.

  • Branch Specifier: Specify the branch (e.g., */master).

  • Script Path: Enter Jenkinsfile.

• Configure Build Triggers (Webhooks):

  • Gitea Webhook (Recommended):

    • If you have the Gitea plugin installed, select "Gitea Webhook".

    • In Gitea, go to your repository's settings and find the "Webhooks" section.

    • Add a new webhook:

      • Target URL: http://your-jenkins-url/gitea-webhook/post. replace your-jenkins-url with the correct address.

      • HTTP Method: POST.

      • Content type: application/json.

      • Secret: enter a secret.

      • Trigger on: Select the events you want to trigger (e.g., "Push events").

    • In Jenkins, in the Gitea Webhook section, enter the same secret.

  • Generic Webhook:

    • If you do not have the plugin, use the generic webhook plugin.

• Configure Build Triggers (Polling SCM):

  Instead of using webhooks, we'll use polling to regularly check your Gitea repository for changes.

  1. In your Jenkins pipeline configuration:

     • Go to the "Build Triggers" section.

     • Check the box labeled "Poll SCM."

  2. Schedule:

     • In the "Schedule" field, you'll enter a cron-like expression that defines how often Jenkins should poll the repository.

     • Specifications:

       • To poll every 5 minutes, you would use: H/5 * * * *

       • Let's break this down:

         • H/5: This means "hash divided by 5." The hash function ensures that polling is distributed evenly across Jenkins instances, preventing a "thundering herd" effect. The /5 means "every 5 minutes."

         • ``: This represents "any value." So, * * * means "any hour, any day of the month, any month, any day of the week."

  3. Example:

     • If you want Jenkins to check your Gitea repository for changes every 5 minutes, enter H/5 * * * * in the "Schedule" field.

• Jenkinsfile Example (Should be in the root folder of the local git repository):

  Groovy

    pipeline {
        agent any
        stages {
            stage('Checkout') {
                steps {
                    git branch: 'main', url: 'http://192.168.0.20:31803/LumiEtherResearch/Starter-Repository.git'
                }
            }
            stage('Display Text Files') {
                steps {
                    script {
                        def files = findFiles(glob: '*.txt')
                        if (files.length == 0) {
                            echo "No text files found."
                        } else {
                            for (file in files) {
                                echo "Found text file: ${file.name}"
                                // Add any other operations you want to perform on each text file here
                                // example:
                                // cat file.name
                            }
                        }
                    }
                }
            }
        }
    }