🚀 Building Playwright Framework Step By Step - Implementing CI/CD

🎯 Introduction

Integrating testing frameworks within CI/CD pipelines embodies a transformative approach to software delivery, enhancing both speed and quality! 🌟 Continuous Integration (CI) and Continuous Delivery (CD) practices facilitate the automatic execution of tests at various stages of the development lifecycle, from initial code commit through to production deployment. By embedding a testing framework into CI/CD workflows, teams can automatically run unit, integration, UI, and API tests. This not only accelerates feedback loops but also significantly reduces the risk of defects making their way into production! 🛡️

🎯 Why CI/CD Integration?: Automated testing in CI/CD pipelines ensures consistent quality, faster releases, and early bug detection - it's the foundation of modern DevOps!

🌟 Key Benefits:

• 🔍 Early Bug Detection: Automated tests within CI/CD identify issues early, when they are easier and less costly to fix

• ✅ Quality Assurance: Consistent testing ensures that code changes meet quality standards before they are merged or released

• ⚡ Release Velocity: Automating the testing process eliminates bottlenecks in the deployment pipeline, enabling faster release cycles

• 🛡️ Reliability and Confidence: Automated, repeatable testing ensures that software releases are more reliable, increasing confidence in the development and deployment process

💡 Modern Development: Incorporating a testing framework into CI/CD pipelines is not just a best practice; it's a cornerstone of modern, efficient, and reliable software development processes!

🔧 Implementing GitHub Actions Workflow

📋 Workflow Details

Our comprehensive CI/CD pipeline includes multiple stages and advanced features! 🎯

• 🔐 Environment Variables: All workflows use environment variables defined in GitHub Secrets

• 🧪 Test Stages: The Pipeline includes stages for setup and smoke test, and testing stage (sanity tests, API tests, and regression tests)

• 📊 Reports: Test reports are uploaded as artifacts for review

• 🔗 Merge Report (Optional): All uploaded reports are downloaded, merged in one, and merged report is uploaded as artifact

• 🏗️ Build Report (Optional): Merged report is downloaded and github-pages are generated and uploaded as artifact

• 🌐 Deploy Report (Optional): Uploaded github-pages is deployed in GitHub Pages for the workflow

⚠️ Important Note: Due to GitHub consideration of url to consist secrets, the workaround was to hardcode the url for the GitHub Pages, so it appears just below the job name and it is easily accessible for everyone.

📝 TO DO: If the job is implemented, the URL should be updated.

🧪 Add failingTest.spec.ts file in the tests/clientSite folder of the project for demonstration purposes

import { test, expect } from '../../fixtures/pom/test-options';

test('Failing Sanity Test', { tag: '@Sanity' }, async ({ homePage }) => {
    await homePage.navigateToHomePageUser();
    expect(2).toEqual(3);
});

test('Failing API Test', { tag: '@Api' }, async ({ homePage }) => {
    await homePage.navigateToHomePageUser();
    expect(2).toEqual(3);
});

test('Failing Regression Test', { tag: '@Regressions' }, async ({ homePage }) => {
    await homePage.navigateToHomePageUser();
    expect(2).toEqual(3);
});

🔧 Create .github/actions folder in the root directory of the project and add playwright-report/action.yml file to it and playwright-setup/action.yml file to it

Playwright Report Action:

name: 'Playwright Report'
description: 'Upload Playwright Report'
inputs:
    test-step-outcome:
        description: 'Outcome of the test step (success or failure)'
        required: true
    blob-report-upload:
        description: 'Wheather to always upload blob report as artifact or only on failure'
        required: false
        default: 'false'
    blob-report-retention-days:
        description: 'Retention days for blob report'
        required: false
        default: '3'
    blob-report-path:
        description: 'Path for blob report'
        required: false
        default: 'blob-report'
runs:
    using: 'composite'
    steps:
        - name: Upload BLOB Report
          uses: actions/upload-artifact@v4
          if: ${{ inputs.blob-report-upload == 'true' || inputs.test-step-outcome == 'failure' }}
          with:
              name: blob-report-${{ github.job }}
              path: ${{ inputs.blob-report-path }}
              retention-days: ${{ inputs.blob-report-retention-days }}

Playwright Setup on Runner Action:

name: 'Playwright Setup on Runner'
description: 'Setup Playwright on Runner by getting the code, setup Node, getting and caching dependencies and getting and caching Playwright Browsers'
inputs:
    caching-dependencies:
        description: 'Wheather to cache dependencies or not'
        required: false
        default: 'true'
    caching-browsers:
        description: 'Wheather to cache browsers or not'
        required: false
        default: 'true'
runs:
    using: 'composite'
    steps:
        - name: Setup Node
          uses: actions/setup-node@v4
          with:
              node-version: lts/*

        - name: Cache Dependencies
          if: inputs.caching-dependencies == 'true'
          id: cache-dependencies
          uses: actions/cache@v4
          with:
              path: node_modules
              key: deps-node-modules-${{ hashFiles('**/package-lock.json') }}

        - name: Install dependencies
          if: steps.cache-dependencies.outputs.cache-hit != 'true' || inputs.caching-dependencies != 'true'
          run: npm ci
          shell: bash

        - name: Cache Playwright Browsers
          if: inputs.caching-browsers == 'true'
          id: cache-browsers
          uses: actions/cache@v4
          with:
              path: ~/.cache/ms-playwright
              key: playwright-browsers-${{ runner.os }}

        - name: Install Playwright Browsers
          if: steps.cache-browsers.outputs.cache-hit != 'true' || inputs.caching-browsers != 'true'
          run: npx playwright install --with-deps chromium
          shell: bash

🔧 Create 'playwright-custom-runner.yml' file in the .github/workflows folder of the project

name: Playwright Tests on Custom Runner Extended

on:
    push:
        branches: [main, master]
    pull_request:
        branches: [main, master]
    workflow_dispatch:

env:
    URL: ${{ secrets.URL }}
    API_URL: ${{ secrets.API_URL }}
    USER_NAME: ${{ secrets.USER_NAME }}
    EMAIL: ${{ secrets.EMAIL }}
    PASSWORD: ${{ secrets.PASSWORD }}

permissions:
    contents: read
    pages: write
    id-token: write

concurrency:
    group: 'pages'
    cancel-in-progress: false

jobs:
    setup-and-smoke-test:
        name: Setup and Smoke Test
        timeout-minutes: 60
        runs-on: ubuntu-latest
        outputs:
            smoke_outcome: ${{ steps.smoke.outcome }}
        steps:
            - name: Checkout code
              uses: actions/checkout@v4

            - name: Playwright Setup on Runner
              uses: ./.github/actions/playwright-setup

            - name: Run Smoke tests
              id: smoke
              run: npm run smoke

            - name: Upload Playwright Report
              if: always()
              uses: ./.github/actions/playwright-report
              with:
                  test-step-outcome: ${{ steps.smoke.outcome }}

    sanity-test:
        name: Sanity Test
        needs: setup-and-smoke-test
        timeout-minutes: 60
        runs-on: ubuntu-latest
        outputs:
            sanity_outcome: ${{ steps.sanity.outcome }}
        steps:
            - name: Checkout code
              uses: actions/checkout@v4

            - name: Playwright Setup on Runner
              uses: ./.github/actions/playwright-setup

            - name: Run Sanity tests
              id: sanity
              run: npm run sanity

            - name: Upload Playwright Report
              if: always()
              uses: ./.github/actions/playwright-report
              with:
                  test-step-outcome: ${{ steps.sanity.outcome }}

    api-test:
        name: API Test
        needs: setup-and-smoke-test
        timeout-minutes: 60
        runs-on: ubuntu-latest
        outputs:
            api_outcome: ${{ steps.api.outcome }}
        steps:
            - name: Checkout code
              uses: actions/checkout@v4

            - name: Playwright Setup on Runner
              uses: ./.github/actions/playwright-setup

            - name: Run API tests
              id: api
              run: npm run api

            - name: Upload Playwright Report
              if: always()
              uses: ./.github/actions/playwright-report
              with:
                  test-step-outcome: ${{ steps.api.outcome }}

    regression-test:
        name: Regression Test
        needs: setup-and-smoke-test
        timeout-minutes: 60
        runs-on: ubuntu-latest
        outputs:
            regression_outcome: ${{ steps.regression.outcome }}
        steps:
            - name: Checkout code
              uses: actions/checkout@v4

            - name: Playwright Setup on Runner
              uses: ./.github/actions/playwright-setup

            - name: Run Regression tests
              id: regression
              run: npm run regression

            - name: Upload Playwright Report
              if: always()
              uses: ./.github/actions/playwright-report
              with:
                  test-step-outcome: ${{ steps.regression.outcome }}

    merge-report:
        name: Merge Report
        if: always()
        needs: [setup-and-smoke-test, sanity-test, api-test, regression-test]
        timeout-minutes: 60
        runs-on: ubuntu-latest
        steps:
            - name: Checkout code
              uses: actions/checkout@v4

            - name: Setup Node.js
              uses: actions/setup-node@v4
              with:
                  node-version: lts/*

            - name: Install Dependencies
              run: npm ci

            - name: Download blob reports from GitHub Actions Artifacts
              uses: actions/download-artifact@v4
              with:
                  path: all-blob-reports
                  pattern: blob-report-*
                  merge-multiple: true

            - name: Merge into HTML Report
              run: npx playwright merge-reports --reporter html ./all-blob-reports

            - name: Upload HTML report
              uses: actions/upload-artifact@v4
              with:
                  name: Merged HTML Reports
                  path: playwright-report
                  retention-days: 3

    build-report:
        name: Build Report
        if: always()
        needs: merge-report
        runs-on: ubuntu-latest
        steps:
            - name: Checkout
              uses: actions/checkout@v4

            - name: Download Merged HTML Reports
              uses: actions/download-artifact@v4
              with:
                  name: Merged HTML Reports
                  path: ./_site

            - name: Upload artifact
              uses: actions/upload-pages-artifact@v3
              with:
                  path: ./_site

    depoloy-report:
        name: Deploy Report
        if: always()
        needs: build-report
        environment:
            name: github-pages
            url: 'https://idavidov13.github.io/Playwright-Framework/'
        runs-on: ubuntu-latest
        steps:
            - name: Deploy to GitHub Pages
              id: deployment
              uses: actions/deploy-pages@v4

🔐 Set Up GitHub Actions Secrets

Secure configuration is essential for protecting sensitive information! 🛡️

🔒 Security Best Practice: Never hardcode sensitive information in your workflows - always use GitHub Secrets for credentials and URLs.

Steps to configure secrets:

• 📂 Navigate to repository setting in GitHub

• 🔐 Go to Secrets and variables

• ⚙️ Go to Actions

• ➕ Click on New repository secret

• 📝 Add the following secrets:

  • URL - Your application URL

  • API_URL - Your API base URL

  • USER_NAME - Test user name

  • EMAIL - Test user email

  • PASSWORD - Test user password

🌐 Enable GitHub Pages

GitHub Pages deployment provides easy access to your test reports! 📊

🎯 Public Accessibility: GitHub Pages makes your test reports accessible to the entire team without requiring repository access.

Configuration steps:

• 📂 Navigate to repository setting in GitHub

• 📄 Go to Pages

• ⚙️ Select Build and deployment source GitHub Actions

🎯 What's Next?

With the implementation of CI/CD, we've accomplished a major part of the work! 🎉 From this point forward, you're equipped with all the essential tools needed to develop an automation framework for various applications. Continue to experiment and expand your knowledge! 🚀

💬 Community: Please feel free to initiate discussions on this topic, as every contribution has the potential to drive further refinement.

---

✨ Congratulations! You've built a comprehensive Playwright testing framework with full CI/CD integration. Keep exploring and building amazing test automation solutions! 🌟