End-to-End CI/CD Pipeline with Azure DevOps: A Comprehensive Guide

Introduction

In today's fast-paced software development world, implementing a robust CI/CD pipeline is essential for delivering high-quality applications quickly and reliably. In this article, I'll walk you through a comprehensive Azure DevOps pipeline that covers everything from code analysis to production deployment, including security scanning and infrastructure management.

Pipeline Overview

The pipeline I've created is designed for a Java Spring Boot application and incorporates modern DevOps practices including:

Infrastructure as Code (Terraform)
Static Application Security Testing (SAST)
Dynamic Application Security Testing (DAST)
Containerization and container security scanning
Multi-cloud deployment (Azure and AWS)
Environment-specific deployments

Pipeline Structure Breakdown

1. Triggers and Agent Configuration

trigger:
  - development
  - uat
  - production

pool:
  name: LinuxAgentPool
  demands:
    - JDK -equals Yes
    - Terraform -equals Yes
    - Agent.Name -equals ProdADO

The pipeline triggers on changes to three branches: development, uat, and production. It requires a Linux agent with specific capabilities (JDK, Terraform) to ensure all pipeline tasks can execute properly.

2. Variables Section

variables:
  global_version: "1.0.0"
  global_email: "sheakdev@gmail.com"
  isDev: $[eq(variables['Build.SourceBranch'], 'refs/heads/development')]
  isProd: $[eq(variables['Build.SourceBranch'], 'refs/heads/production')]

Variables are defined at different scopes (global, stage, job) allowing for flexible configuration. The conditional variables (isDev, isProd) enable branch-specific behavior in the pipeline.

Key Stages Explained

Stage 1: CheckingTheAgent

This initial validation stage ensures our agent has all required tools installed:

- stage: CheckingTheAgent
  condition: and(succeeded(), eq(variables.isDev, true))
  jobs:
    - job: CheckingTerraformAndPacker
      steps:
        - script: terraform version && packer version
        - script: docker version && docker ps && docker images && docker ps -a
        - script: pwd && ls -al

It verifies versions of critical tools (Terraform, Packer, Docker) and checks the working directory structure.

Stage 2: SAST With SonarQube

- stage: SASTWithSonarQube
  jobs:
    - job: RunningSASTWithSonarqube
      steps:
        - task: SonarQubePrepare@7
        - task: Maven@4
          inputs:
            sonarQubeRunAnalysis: true
        - task: sonar-buildbreaker@8

This stage performs static code analysis using SonarQube to identify code quality issues and security vulnerabilities early in the development cycle.

Stage 3: Building Java Code with Maven

- stage: BuildingJavaCodeWithMavenCopyToJFrog
  jobs:
    - job: BuildingJavaCodeJob
      steps:
        - script: mvn versions:set -DnewVersion=Dev-2.0.$(Build.BuildId)
        - script: mvn clean package install
        - script: mvn deploy
        - task: PublishBuildArtifacts@1

The build stage compiles the Java code, runs tests, and publishes artifacts. Key features:

Dynamic versioning based on build ID
Artifact publishing for downstream stages
JFrog artifact repository integration

Stage 4: Multi-Cloud Artifact Distribution

- stage: CopyingArtifactsToAzureAndAws
  jobs:
    - job: CopyFilesToAzureBlob
      steps:
        - task: AzureCLI@2
          inputs:
            inlineScript: az storage blob upload-batch...
    - job: CopyFilesToAWSS3Bucket
      steps:
        - task: S3Upload@1

This stage demonstrates hybrid cloud deployment by copying artifacts to both Azure Blob Storage and AWS S3, ensuring availability across platforms.

Stage 5: Container Security with Trivy

- stage: DockerBuildAndTrivyScan
  jobs:
    - job: BuildingContainerImageAndSecurityScanning
      steps:
        - script: docker build -t sheakimran21/myapp:$(Build.BuildId) .
        - script: trivy image --exit-code 0 --severity LOW,MEDIUM...
        - script: trivy image --exit-code 0 --severity HIGH,CRITICAL...
        - task: PublishTestResults@2

Container security scanning with Trivy checks for vulnerabilities at different severity levels, publishing results in JUnit format for visibility.

Stage 6: Container Registry Push

- stage: BuildDockerImagePushToAzureACRAndDockerHub
  jobs:
    - job: PushToAzureACR
      steps:
        - script: docker login...
        - script: docker tag...
        - script: docker push...
    - job: PushToDockerHub
      steps:
        - task: Docker@2

The built container image is pushed to both Azure Container Registry and Docker Hub, demonstrating multi-registry support.

Stage 7: Deployment to Staging

- stage: DeployingToStagingEnvironment
  jobs:
    - deployment: DeployJARtoStagingServer
      environment: STAGING
      strategy:
        runOnce:
          deploy:
            steps:
              - script: sudo kill -9 $PROC
              - script: sudo java -jar /home/ubuntu/azagent/_work/1/ROOT$(Build.BuildId).jar/ROOT$(Build.BuildId).jar &

The staging deployment ensures the application works in a production-like environment before promoting to production.

Stage 8: DAST Testing with OWASP ZAP

- stage: ZAPOWASPTestingStagingEnvironment
  jobs:
    - job: ZapTestingStaging
      steps:
        - script: docker run ghcr.io/zaproxy/zaproxy:stable zap-baseline.py...
        - task: PublishTestResults@2

Dynamic Application Security Testing (DAST) with OWASP ZAP identifies runtime vulnerabilities in the staging environment.

Stage 9: Production Deployment

- stage: DeployingToProdEnvironment
  condition: or(eq(variables.isDev, true), eq(variables.isProd, true))
  jobs:
    - deployment: DeployJARtoProdServer
      environment: PROD
      strategy:
        runOnce:
          deploy:
            steps:
              - task: DownloadBuildArtifacts@0
              - script: sudo kill -9 $PROC
              - script: sudo java -jar /home/ubuntu/azagent/_work/1/ROOT$(Build.BuildId).jar/ROOT$(Build.BuildId).jar &

The production deployment is gated by branch conditions and follows the same pattern as staging but targets the production environment.

Key DevOps Practices Demonstrated

Infrastructure as Code: Terraform is used for infrastructure management
Shift-Left Security: SAST and DAST testing early in the pipeline
Immutable Artifacts: Versioned builds ensure traceability
Multi-Cloud Support: Deployment to both Azure and AWS
Container Security: Trivy scanning for container vulnerabilities
Environment Promotion: Clear staging → production progression
Conditional Execution: Branch-specific behavior

project github URL : https://github.com/saikiranpi/Mastering-DevSecOps/tree/Master/Day%2028%20SAST-AzureDevOps-Part-1

Conclusion

This Azure DevOps pipeline showcases a comprehensive approach to modern application delivery, incorporating security scanning, multi-cloud deployment, and environment promotion. By implementing such a pipeline, teams can achieve:

Faster feedback cycles with early testing
Improved security with SAST/DAST scanning
Consistent deployments across environments
Flexibility to deploy to multiple cloud providers
Traceability through versioned artifacts

The pipeline can be further enhanced with additional steps like performance testing, blue-green deployments, or canary releases based on specific organizational needs.

credit of this project : Pinapathruni Saikiran