How to Run Selenium Tests Using Maven and Jenkins (Part 6)

Samiksha KuteSamiksha Kute
14 min read

Table of contents

Welcome to Part 6 of our Selenium Framework Design! In this guide, we’ll learn how to run our Selenium tests using Maven commands, pass runtime parameters, integrate with Jenkins, run tests in headless mode, and schedule Jenkins jobs. By the end, you’ll have a robust framework ready for Continuous Integration (CI). Let’s dive in!

Why Maven and Jenkins?

Until now, we’ve been running our TestNG tests by right-clicking files in Eclipse and selecting “Run As > TestNG Test.” This works fine in development, but in real-world projects, tests are executed in a Continuous Integration (CI) environment like Jenkins, where you can’t rely on a UI. Instead, you use commands to trigger tests. Maven, our build tool, simplifies this by allowing us to run tests from the terminal with simple commands. Jenkins, a CI tool, automates test execution, schedules runs, and integrates with version control systems like GitHub. In this part, we’ll:

  1. Run tests using Maven commands from the terminal.

  2. Pass runtime parameters (e.g., browser type) via Maven.

  3. Set up Jenkins to run tests automatically.

  4. Add headless mode for faster test execution.

  5. Schedule Jenkins jobs for automated nightly runs.

This guide assumes your Selenium framework (from Parts 1–5) is set up as a Maven project, with test suites like testng.xml, Purchase.xml, and ErrorValidation.xml. Check out my previous blogs for reference if needed.

Part 1: Running Tests with Maven Commands

Why Use Maven Commands?

Maven allows us to execute tests from the terminal without relying on Eclipse’s UI. This is crucial for CI environments like Jenkins, where tests are triggered via commands. Maven comes pre-installed as a plugin in Eclipse, but to run commands from the terminal, we need to install Maven on our system.

Step 1: Installing Maven on Your System

  1. Download Maven:

    • Search for “Maven download” on Google and visit maven.apache.org.

    • Download the appropriate file based on your operating system:

      • Windows: Download the .zip file.

      • Mac: Download the .tar.gz file.

    • Unzip the file to a folder (e.g., C:\Program Files\apache-maven).

  2. Set MAVEN_HOME:

    • Create a system environment variable called MAVEN_HOME with the path to the unzipped Maven folder (e.g., C:\Program Files\apache-maven).

    • Optionally, add the Maven bin folder to the system’s Path variable:

      • Append %MAVEN_HOME%\bin to the Path variable.

    • Follow a reliable online guide for detailed steps (search “set Maven home environment variable”).

  3. Verify Installation:

    • Open a terminal (Command Prompt on Windows, Terminal on Mac).

    • Run mvn -version.

    • If Maven is installed correctly, you’ll see the Maven version (e.g., Apache Maven 3.x.x).

    • Restart the terminal after setting environment variables to apply changes.

Step 2: Adding Maven-TestNG Integration

To run TestNG suites via Maven, we use the Maven Surefire Plugin. This plugin allows Maven to execute TestNG XML files.

  1. Update pom.xml:

    • Open your project’s pom.xml file in Eclipse.

    • Add the Surefire Plugin inside the <plugins> section under <build>.

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.0.0-M5</version>
                <configuration>
                    <suiteXmlFiles>
                        <suiteXmlFile>testSuites/testng.xml</suiteXmlFile>
                    </suiteXmlFiles>
                </configuration>
            </plugin>
        </plugins>
    </build>

  • Explanation:

    • <groupId> and <artifactId>: Specify the Surefire Plugin.

    • <version>: Use a stable version (e.g., 3.0.0-M5; check Maven Repository for the latest).

    • <suiteXmlFiles>: Points to the TestNG XML file (e.g., testSuites/testng.xml, which runs all tests).

    • The path is relative to the project root where pom.xml resides.

  1. Problem with Hard-coding:

    • Hard coding testng.xml in pom.xml limits flexibility. Our project has multiple XML files (testng.xml, Purchase.xml, ErrorValidation.xml) for different test groups (e.g., Regression, Purchase, ErrorValidation).

    • Hard-coding requires updating pom.xml each time we want to run a different XML file, which isn’t practical for projects with many suites.

  2. Solution: Using Maven Profiles:

    • Maven profiles allow us to define multiple configurations in pom.xml and choose one at runtime.

    • Add a <profiles> section in pom.xml to support all TestNG XML files.

    <profiles>
        <profile>
            <id>Regression</id>
            <build>
                <pluginManagement>
                    <plugins>
                        <plugin>
                            <groupId>org.apache.maven.plugins</groupId>
                            <artifactId>maven-surefire-plugin</artifactId>
                            <version>3.0.0-M5</version>
                            <configuration>
                                <suiteXmlFiles>
                                    <suiteXmlFile>testSuites/testng.xml</suiteXmlFile>
                                </suiteXmlFiles>
                            </configuration>
                        </plugin>
                    </plugins>
                </pluginManagement>
            </build>
        </profile>
        <profile>
            <id>Purchase</id>
            <build>
                <pluginManagement>
                    <plugins>
                        <plugin>
                            <groupId>org.apache.maven.plugins</groupId>
                            <artifactId>maven-surefire-plugin</artifactId>
                            <version>3.0.0-M5</version>
                            <configuration>
                                <suiteXmlFiles>
                                    <suiteXmlFile>testSuites/Purchase.xml</suiteXmlFile>
                                </suiteXmlFiles>
                            </configuration>
                        </plugin>
                    </plugins>
                </pluginManagement>
            </build>
        </profile>
        <profile>
            <id>ErrorValidation</id>
            <build>
                <pluginManagement>
                    <plugins>
                        <plugin>
                            <groupId>org.apache.maven.plugins</groupId>
                            <artifactId>maven-surefire-plugin</artifactId>
                            <version>3.0.0-M5</version>
                            <configuration>
                                <suiteXmlFiles>
                                    <suiteXmlFile>testSuites/ErrorValidation.xml</suiteXmlFile>
                                </suiteXmlFiles>
                            </configuration>
                        </plugin>
                    </plugins>
                </pluginManagement>
            </build>
        </profile>
    </profiles>

  • Explanation:

    • Each <profile> has an <id> (e.g., Regression, Purchase, ErrorValidation) mapping to a TestNG XML file.

    • The <suiteXmlFile> specifies the path to the respective XML file.

    • <pluginManagement> ensures proper plugin configuration.

    • Use the same <groupId>, <artifactId>, and <version> across profiles for consistency.

  1. Set Java Version:

    • Ensure your pom.xml specifies Java 1.8 (or higher) for compatibility, as some string functions require it.

    • Add this to the <properties> section:

    <properties>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
    </properties>
  • If not set, Maven may default to Java 1.7, causing compilation errors.

Step 3: Running Tests via Maven

  1. Navigate to Project Directory:

    • Open a terminal and navigate to the project root (where pom.xml is located) using cd:

        cd path/to/your/project

  2. Run a Profile:

    • To run the Regression profile (which triggers testng.xml):

        mvn test -PRegression

    • Explanation:

      • mvn test: Compiles the code and runs tests.

      • -PRegression: Activates the Regression profile, executing testng.xml.

      • This triggers all tests in parallel (as configured in testng.xml), opening two browsers simultaneously, just like running from Eclipse.

  3. Verify Results:

    • After execution, check the terminal for output (e.g., “5 ran, 1 failure” due to the intentional failure in ErrorValidation.xml).

    • Refresh the project in Eclipse to view the Extent Report (from Part 5) in the project folder.

    • The report confirms the same results as running via Eclipse, with screenshots for failures (e.g., LoginErrorValidation).

  4. Run Other Profiles:

    • To run the Purchase profile:

        mvn test -PPurchase

    • To run the ErrorValidation profile:

        mvn test -PErrorValidation

    • Each command triggers the respective XML file, running only the specified test group.

Why Profiles?

Profiles make the framework flexible. Instead of hard-coding one XML file, you can switch between test suites (e.g., Regression, Purchase, ErrorValidation) by changing the profile name in the command. This is ideal for projects with multiple test suites (e.g., 10–15 XML files in real-world scenarios).

Part 2: Passing Runtime Parameters with Maven

Why Runtime Parameters?

Our framework uses a GlobalData.properties file (from Part 4) to set variables like browser=chrome. However, changing this file requires updating the code, committing to GitHub, and redeploying time-consuming steps. Instead, we can pass parameters (e.g., browser type) at runtime via Maven commands, avoiding code changes.

Step 1: Modifying BaseTest for Runtime Parameters

We’ll update BaseTest.java to prioritize Maven parameters over GlobalData.properties using a Java ternary operator.

  1. Original Code (in BaseTest.java):

    • The browser is set by reading GlobalData.properties:

        String browserName = prop.getProperty("browser");

  2. Updated Code:

    • Use System.getProperty to check for Maven parameters, falling back to GlobalData.properties if none are provided:

        String browserName = System.getProperty("browser") != null ? System.getProperty("browser") : prop.getProperty("browser");

    • Explanation:

      • Ternary Operator: condition ? valueIfTrue : valueIfFalse

        • Condition: System.getProperty("browser") != null checks if a browser parameter is passed via Maven.

        • ValueIfTrue: System.getProperty("browser") uses the Maven parameter (e.g., Firefox).

        • ValueIfFalse: prop.getProperty("browser") uses GlobalData.properties (e.g., Chrome).

      • System.getProperty: Reads system-level variables, including Maven parameters

        (e.g., -Dbrowser=Firefox).

  3. Add Firefox Support:

    • Ensure BaseTest.java supports Firefox by adding logic for geckodriver:

        if (browserName.equalsIgnoreCase("firefox")) {
      System.setProperty("webdriver.gecko.driver", "path/to/geckodriver");
      driver = new FirefoxDriver();
  }

    • Without this, running with -Dbrowser=Firefox causes a NullPointerException because the driver isn’t initialized.

Step 2: Running Tests with Runtime Parameters

  1. Run with Firefox:

    • In the terminal, run:

        mvn test -PRegression -Dbrowser=Firefox

    • Explanation:

      • -PRegression: Runs the Regression profile (testng.xml).

      • -Dbrowser=Firefox: Passes browser=Firefox as a Maven parameter.

      • The ternary operator in BaseTest.java detects Firefox, overriding GlobalData.properties.

    • Result: Two Firefox browsers open, running tests in parallel. The report shows results (e.g., 5 ran, 1 failure).

  2. Run Without Browser Parameter:

    • Run:

        mvn test -PRegression

    • Explanation:

      • No browser parameter is provided, so System.getProperty("browser") returns null.

      • The ternary operator falls back to GlobalData.properties, using browser=Chrome.

    • Result: Tests run in Chrome, with the same output as before.

Why Ternary Operator?

The ternary operator condenses an if-else condition into one line, making the code concise and impressing interviewers. It checks for Maven parameters first, ensuring flexibility without modifying GlobalData.properties.

Part 3: Setting Up Jenkins for Test Execution

Why Jenkins?

Jenkins automates test execution, integrates with GitHub, and schedules runs (e.g., nightly tests). It’s a server-based tool that runs on your local machine or a dedicated server, triggered via commands.

Step 1: Installing Jenkins

  1. Download Jenkins:

    • Visit jenkins.io and download the Generic Java Package (.war) file.

    • Save it as jenkins.war in a folder (e.g., C:\work).

  2. Start Jenkins:

    • Open a terminal and navigate to the folder containing jenkins.war:

        cd C:\work

    • Run:

        java -jar jenkins.war --httpPort=9090

    • Explanation:

      • java -jar: Executes the .war file.

      • jenkins.war: The Jenkins file.

      • --httpPort=9090: Runs Jenkins on port 9090 (use 8080 if preferred, but avoid conflicts with other software).

    • Jenkins starts a local server, showing logs like “started initialization.”

  3. Access Jenkins:

    • Open a browser and go to http://localhost:9090.

    • On first run, Jenkins prompts for a username and password (set these up).

    • Log in to access the Jenkins homepage.

Step 2: Creating a Jenkins Job

  1. Create a New Job:

    • On the Jenkins homepage, click New Item.

    • Name the job (e.g., Selenium Framework).

    • Select Freestyle project (suitable for Maven-based projects) and click OK.

  2. Configure the Job:

    • Source Code Management:

      • If your code is on GitHub, select Git and provide the repository URL.

      • For local code, click Advanced and select Use custom workspace. Paste the project path (e.g., C:\path\to\your\project), ensuring pom.xml is in this directory.

    • Build Step:

      • Click Add build step > Invoke top-level Maven targets.

      • Enter the Maven command (without mvn):

          test -PRegression -Dbrowser=chrome

      • Explanation:

        • test: Runs tests.

        • -PRegression: Uses the Regression profile.

        • -Dbrowser=chrome: Sets the browser.

        • Jenkins automatically recognizes mvn since it’s a Maven job.

    • Save the configuration.

  3. Run the Job:

    • Click Build Now.

    • Jenkins executes the command, opening two Chrome browsers in parallel (as per testng.xml).

    • Check Console Output for logs, confirming “5 ran, 1 failure” with the Extent Report generated.

Step 3: Parameterizing the Jenkins Job

Hardcoding browser=Chrome in the Maven command requires editing the job configuration for changes (e.g., to Firefox). Instead, we’ll parameterize the job for flexibility.

  1. Add Parameters:

    • Go to Configure > Check This project is parameterized.

    • Click Add Parameter > Choice Parameter.

    • Name: browser.

    • Choices: Enter options (one per line):

        Chrome
  Firefox
  Edge

    • Add another Choice Parameter for profiles:

      • Name: profile.

      • Choices:

          Regression
  Purchase
  ErrorValidation

  2. Update the Maven Command:

    • In the Invoke top-level Maven targets field, replace the hardcoded command:

        test -P"$profile" -Dbrowser="$browser"

    • Explanation:

      • "$profile": Uses the selected profile (e.g., Regression) from the dropdown.

      • "$browser": Uses the selected browser (e.g., Firefox) from the dropdown.

      • Jenkins replaces $variable with the chosen value at runtime.

  3. Run with Parameters:

    • Click Build with Parameters.

    • Select browser=Firefox and profile=ErrorValidation as shown in the image below:

    • Click Build.

    • Result: Jenkins runs mvn test -PErrorValidation -Dbrowser=Firefox, executing ErrorValidation.xml in Firefox. The console output confirms the command and results.

  4. Fixing Errors:

    • If a profile fails (e.g., due to a typo like Errorvalidation instead of ErrorValidation), check the console output for errors and correct the profile ID in pom.xml.

Why Parameterization?

Parameterizing the job allows non-technical users (e.g., business teams) to select browsers or profiles via dropdowns, avoiding the need to edit configurations. This makes the framework user-friendly and scalable.

Part 4: Running Tests in Headless Mode

What is Headless Mode?

Headless mode runs tests without opening a visible browser window, executing in the background using the browser’s engine (e.g., Chrome’s engine). This speeds up execution, as no UI rendering is involved, making it ideal for CI environments.

Step 1: Adding Headless Support to BaseTest

  1. Modify the initializeDriver() method BaseTest.java:

    • Add ChromeOptions to configure Chrome for headless mode:

        import org.openqa.selenium.chrome.ChromeOptions;

  public class BaseTest {
      public WebDriver initializeDriver() throws IOException {
          Properties prop = new Properties();
          FileInputStream fis = new FileInputStream(
                  System.getProperty("user.dir") + "/src/main/java/resources/GlobalData.properties");
          prop.load(fis);
          String browserName = System.getProperty("browser") != null ? System.getProperty("browser") : prop.getProperty("browser");
          if (browserName.contains("chrome")) {
              ChromeOptions options = new ChromeOptions(); // adding ChromeOptions
              if(browserName.contains("headless")) {
                  options.addArguments("headless");
              }
              driver = new ChromeDriver(options);
              driver.manage().window().setSize(new Dimension(1440, 900));
          } else if (browserName.equalsIgnoreCase("firefox")) {
              driver = new FirefoxDriver();
          } else if (browserName.equalsIgnoreCase("edge")) {
              driver = new EdgeDriver();
          }
          driver.manage().window().maximize();
          driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(10));
          return driver;
      }
  }

    • Explanation:

      • ChromeOptions: Configures Chrome settings.

      • options.addArguments("headless"): Enables headless mode if browserName contains “headless” (e.g., Chrome headless).

      • driver = new ChromeDriver(options): Passes the options to the driver.

      • setSize(new Dimension(1440, 900)): Sets a custom window size (optional, prevents element visibility issues in headless mode).

      • contains("chrome"): Allows both Chrome and Chrome headless to enter the Chrome block.

  2. Why Set Window Size?:

    • In headless mode, the browser may open in a small viewport, causing elements (e.g., “Place Order” button) to be invisible, leading to test failures.

    • Setting 1440x900 ensures the application fits the screen, mimicking a maximized browser.

Step 2: Running in Headless Mode

  1. Update Jenkins Job:

    • In Configure > Choice Parameter for browser, add:

        Chrome
  Firefox
  Edge
  Chrome headless

  2. Run the Job:

    • Click Build with Parameters > Select browser=Chrome headless and profile=Regression.

    • Click Build.

    • Result:

      • No browser windows open (execution happens in the background).

      • The console output shows mvn test -PRegression -Dbrowser=Chrome headless.

      • Tests run faster, with results in the Extent Report (e.g., all pass if bugs are fixed).

  3. Fixing Failures:

    • If tests fail (e.g., due to incorrect email/password in ErrorValidation), fix the test data and rerun.

    • Headless mode may reveal flakiness (e.g., element not visible). The setSize step helps mitigate this.

Why Headless Mode?

  • Faster Execution: No UI rendering, ideal for CI.

  • Warning: Some prefer headed mode to catch UI-specific issues, as it mimics user behavior. Use headless for speed, headed for debugging.

Part 5: Scheduling Jenkins Jobs

Why Schedule Jobs?

Scheduling allows Jenkins to run tests automatically (e.g., nightly at 6:00 AM) when application traffic is low, ensuring reliable execution. This is a core feature of CI.

Step 1: Configuring a Schedule

  1. Go to Configure:

    • Open the Jenkins job > Configure > Triggers.

    • Check Build periodically.

  2. Set the Schedule:

    • Use a cron-like syntax with five parameters: MINUTE HOUR DAY MONTH DAYOFWEEK.

    • Examples:

      • 15 7 *: Runs daily at 7:15 AM.

      • 45 7 *: Runs daily at 7:45 AM.

      • H 5 *: Runs daily at 5:00–5:59 AM (random minute to avoid conflicts).

      • H 6 0: Runs every Sunday at 6:00–6:59 AM.

      • H 5 12: Runs daily in December at 5:00–5:59 AM.

    • Explanation:

      • MINUTE: Minute of the hour (0–59 or H for random).

      • HOUR: Hour of the day (0–23).

      • DAY: Day of the month (1–31 or * for any).

      • MONTH: Month (1–12 or * for any).

      • DAYOFWEEK: Day of the week (0–6, 0=Sunday, or * for any).

      • Use * for daily runs, specific numbers for targeted schedules (e.g., 12 for December).

    • Recommended: H 6 * (runs daily at 6:00–6:59 AM).

  3. Avoiding Pitfalls:

    • Using * for MINUTE may trigger the job every minute in the hour (e.g., 6:00–6:59). Use H or a specific minute (e.g., 40 for 6:40 AM).

    • Save the configuration.

  4. Test the Schedule:

    • Keep the Jenkins server running (don’t close the terminal or shut down the system).

    • The next day at 6:00 AM, check the Jenkins job for a new build in the Build History.

    • Verify the console output and Extent Report for results.

Why Scheduling?

Scheduling automates testing, running tests at low-traffic times (e.g., 5:00 AM) to ensure stability. In real-world projects, nightly builds are common for regression testing.

Key Takeaways

  1. Maven Commands:

    • Installed Maven and set MAVEN_HOME for terminal execution.

    • Used the Surefire Plugin to run TestNG XML files.

    • Added profiles in pom.xml for flexible suite execution (Regression, Purchase, ErrorValidation).

    • Ran tests with mvn test -P<profile>.

  2. Runtime Parameters:

    • Used System.getProperty and the ternary operator to prioritize Maven parameters (e.g., -Dbrowser=Firefox) over GlobalData.properties.

    • Enabled dynamic browser selection without code changes.

  3. Jenkins Integration:

    • Installed Jenkins with java -jar jenkins.war --httpPort=9090.

    • Created a Freestyle project with a custom workspace and Maven command.

    • Parameterized the job for browser and profile, using $variable in commands.

  4. Headless Mode:

    • Added ChromeOptions with headless for faster, UI-less execution.

    • Set window size (1440x900) to avoid visibility issues.

    • Updated Jenkins to support Chrome headless.

  5. Scheduling:

    • Configured Build periodically with cron-like syntax (e.g., H 6 *).

    • Ensured jobs run automatically at specified times (e.g., 6:00 AM daily).

What’s Next?

In Part 7, we’ll integrate Cucumber to create a hybrid TestNG-Cucumber framework, writing tests in Gherkin for business-readable scenarios. We’ll reuse our POM classes and add feature files, step definitions, and tags, enhancing our framework’s flexibility.

Happy testing!

0
Subscribe to my newsletter

Read articles from Samiksha Kute directly inside your inbox. Subscribe to the newsletter, and don't miss out.

selenium-webdriverJenkinsmavenframework-design

Written by

Samiksha Kute
Samiksha Kute

Passionate Learner!