Retry Failed Test in TestNG with IRetryAnalyzer

Introduction

Automated testing is a crucial part of the software development lifecycle, providing rapid feedback on the health of an application. However, dealing with failed test cases and understanding their root causes can be challenging. In this article, we'll explore how to retry failed tests in TestNG using the IRetryAnalyzer interface. Additionally, we'll learn how to rerun all tests from scratch in a test class.

How to Retry Failed Test in TestNG with IRetryAnalyzer

To implement the retry mechanism for failed tests, we'll create a separate class that implements the IRetryAnalyzer interface. Here's an example:

package com.mystore.utility;

import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;

public class Retry implements IRetryAnalyzer {
    private static final int MAX_RETRY_COUNT = 3;
    private int retryCount = 0;

    @Override
    public boolean retry(ITestResult result) {
        if (retryCount < MAX_RETRY_COUNT) {
            retryCount++;
            return true;
        }
        return false;
    }
}

In this example, the failed test cases will be retried up to three times. Adjust the maxTry value based on your requirements.

To associate your test cases with IRetryAnalyzer, use the retryAnalyzer attribute in the @Test annotation:

package com.mystore.testcases;

import org.testng.Assert;
import org.testng.annotations.Test;

public class RetryTestCase {

  @Test(retryAnalyzer = Retry.class)
    public void Demo1() {
    System.out.println("Demo1");
    Assert.assertEquals(1, 10);
  }

  @Test(retryAnalyzer = Retry.class)
    public void Demo2() {
    System.out.println("Demo2");
    Assert.assertEquals(1, 10);
  }

  @Test(retryAnalyzer = Retry.class)
    public void Demo3() {
    System.out.println("Demo3");
    Assert.assertEquals(1, 1);
  }

  @Test(retryAnalyzer = Retry.class)
    public void Demo4() {
    System.out.println("Demo4");
    Assert.assertEquals(1, 1);
  }
}

Now, the Demo1 and Demo2 methods will be retried three times, and if they continue to fail, the test execution will stop.

Using Retry Class with IAnnotationTransformer Interface

The IAnnotationTransformer listener allows us to modify TestNG annotations during runtime. This approach provides flexibility without requiring recompilation. Here's an example:

package com.mystore.utility;

import java.lang.reflect.Constructor;
import java.lang.reflect.Method;
import org.testng.IAnnotationTransformer;
import org.testng.annotations.ITestAnnotation;

public class AnnotationTransformer implements IAnnotationTransformer {
    @Override
    public void transform(ITestAnnotation annotation, Class testClass, Constructor testConstructor, Method testMethod) {
        annotation.setRetryAnalyzer(Retry.class);
    }
}

Add this listener to your TestNG XML file:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="RetrySuite">
  <listeners>
    <listener class-name="com.mystore.utility.AnnotationTransformer"></listener>
  </listeners>
  <test name="RetryTest">
    <classes>
      <class name="com.mystore.testcases.RetryTestCase"></class>
    </classes>
  </test>
</suite>

Now, you can use the @Test annotation without the retryAnalyzer attribute in your test class:

package com.mystore.testcases;

import org.testng.Assert;
import org.testng.annotations.Test;

public class RetryTestCase {

  @Test
  public void Demo1() {
    System.out.println("Demo1");
    Assert.assertEquals(1, 10);
  }

  @Test
  public void Demo2() {
    System.out.println("Demo2");
    Assert.assertEquals(1, 10);
  }

  @Test
  public void Demo3() {
    System.out.println("Demo3");
    Assert.assertEquals(1, 1);
  }

  @Test
  public void Demo4() {
    System.out.println("Demo4");
    Assert.assertEquals(1, 1);
  }
}

Rerun All Tests in a Class When One Fails

To rerun all tests in a class when one of them fails, we'll utilize the onTestFailure method in a custom TestListener. Here's the relevant code snippet:

package com.mystore.utility;

import org.testng.ITestListener;
import org.testng.ITestResult;
import org.testng.TestNG;

public class TestListener implements ITestListener{
  private static int count = 0;
  private final static int maxTry = 2;

  public void onTestFailure(ITestResult iTestResult) {
      System.out.println("I am in onTestFailure method " + iTestResult.getName() + " failed");
      if (count < maxTry) {
          count++;
          TestNG tng = new TestNG();
          tng.setDefaultTestName("RETRY TEST");
          Class[] classes1 = {iTestResult.getTestClass().getRealClass()};
          tng.setTestClasses(classes1);
          tng.addListener(new TestListener());
          tng.run();
      }
  }

}

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="RetryAllSuite">
  <listeners>
    <listener class-name="com.mystore.utility.TestListener"></listener>
  </listeners>
  <test name="RetryTest">
    <classes>
      <class name="com.mystore.testcases.RetryTestCase"></class>
    </classes>
  </test>
</suite>

Now, when one of the tests fails, all tests in the class will be rerun.

This comprehensive approach to retrying failed tests in TestNG provides flexibility and control over your test automation suite. Choose the method that best suits your project's needs, whether it's retrying individual tests or rerunning the entire suite when needed.

Conclusion

In conclusion, leveraging the IRetryAnalyzer in TestNG provides a powerful mechanism to handle test failures gracefully. By incorporating retry functionality, you can improve the stability and reliability of your test suite. This blog has walked you through the essentials of implementing IRetryAnalyzer for retrying failed tests in TestNG, empowering you to build more resilient automated testing scenarios.