Skip to content

TestNG Comprehensive Cheatsheet

Installation

Platform/Tool Installation Method
Maven Add to pom.xml:
<dependency>
  <groupId>org.testng</groupId>
  <artifactId>testng</artifactId>
  <version>7.8.0</version>
  <scope>test</scope>
</dependency>
Gradle Add to build.gradle:
testImplementation 'org.testng:testng:7.8.0'
Eclipse IDE Help → Eclipse Marketplace → Search "TestNG" → Install
IntelliJ IDEA Bundled by default (or File → Settings → Plugins → Marketplace → "TestNG")
VS Code code --install-extension vscjava.vscode-java-pack
code --install-extension testng.testng
Manual (Linux/macOS) wget https://repo1.maven.org/maven2/org/testng/testng/7.8.0/testng-7.8.0.jar
export CLASSPATH=$CLASSPATH:/path/to/testng-7.8.0.jar
Manual (Windows) Download JAR from Maven Central
set CLASSPATH=%CLASSPATH%;C:\path\to\testng-7.8.0.jar

Core Annotations

Annotation Description
@Test Marks a method as a test method
@Test(priority = 1) Sets execution order (lower numbers run first)
@Test(description = "...") Adds descriptive text to test for reporting
@Test(timeOut = 5000) Fails test if execution exceeds timeout (milliseconds)
@Test(expectedExceptions = Exception.class) Expects specific exception to be thrown
@Test(enabled = false) Disables/skips the test
@Test(groups = {"smoke", "regression"}) Assigns test to one or more groups
@Test(dependsOnMethods = {"testMethod"}) Runs after specified method(s) complete
@Test(dependsOnGroups = {"smoke"}) Runs after all tests in specified group(s)
@Test(alwaysRun = true) Runs test even if dependencies fail
@Test(invocationCount = 3) Runs test multiple times
@Test(threadPoolSize = 5) Runs multiple invocations in parallel threads
@BeforeMethod Executes before each @Test method
@AfterMethod Executes after each @Test method
@BeforeClass Executes once before any test method in the class
@AfterClass Executes once after all test methods in the class
@BeforeTest Executes before any test method in <test> tag
@AfterTest Executes after all test methods in <test> tag
@BeforeSuite Executes once before all tests in the suite
@AfterSuite Executes once after all tests in the suite
@BeforeGroups Executes before first test method of specified group(s)
@AfterGroups Executes after last test method of specified group(s)
@DataProvider Supplies data to test methods for parameterization
@Parameters Injects parameters from testng.xml into test methods
@Factory Creates test instances dynamically
@Listeners Attaches custom listeners to test class

Command Line Execution

Command Description
java -cp "classes:lib/*" org.testng.TestNG testng.xml Run tests using XML suite file
java -cp "classes:lib/*" org.testng.TestNG -testclass com.example.MyTest Run specific test class
java -cp "classes:lib/*" org.testng.TestNG -testclass Test1,Test2 Run multiple test classes (comma-separated)
java -cp "classes:lib/*" org.testng.TestNG -groups smoke testng.xml Run tests from specific group(s)
java -cp "classes:lib/*" org.testng.TestNG -excludegroups slow testng.xml Exclude specific group(s) from execution
java -cp "classes:lib/*" org.testng.TestNG -d test-output testng.xml Specify output directory for reports
java -cp "classes:lib/*" org.testng.TestNG -parallel methods -threadcount 5 Run tests in parallel with thread count
java -cp "classes:lib/*" org.testng.TestNG -verbose 10 testng.xml Set verbosity level (0-10, higher = more detail)
java -cp "classes:lib/*" org.testng.TestNG -methods MyTest.test1,MyTest.test2 Run specific test methods
java -cp "classes:lib/*" org.testng.TestNG -suitename "MySuite" -testname "MyTest" Override suite and test names
java -cp "classes:lib/*" org.testng.TestNG -reporter org.testng.reporters.EmailableReporter Use specific reporter
java -cp "classes:lib/*" org.testng.TestNG -listener com.example.MyListener Add custom listener

Maven Commands

Command Description
mvn test Run all tests
mvn test -Dtest=MyTestClass Run specific test class
mvn test -Dtest=MyTestClass#testMethod Run specific test method
mvn test -Dtest=MyTestClass#test* Run test methods matching pattern
mvn test -DsuiteXmlFile=smoke-tests.xml Run specific suite XML file
mvn test -Dgroups=smoke,regression Run specific test groups
mvn test -DexcludedGroups=slow Exclude specific test groups
mvn test -Denvironment=staging -Dbrowser=chrome Pass system properties to tests
mvn test -DskipTests Skip test execution
mvn test -Dmaven.test.failure.ignore=true Continue build even if tests fail
mvn test -Dparallel=methods -DthreadCount=4 Run tests in parallel
mvn clean test Clean previous builds and run tests
mvn test -X Run tests with debug output
mvn surefire-report:report Generate HTML test report

Gradle Commands

Command Description
gradle test Run all tests
gradle test --tests MyTestClass Run specific test class
gradle test --tests MyTestClass.testMethod Run specific test method
gradle test --tests *IntegrationTest Run tests matching pattern
gradle test --tests MyTestClass --tests OtherTest Run multiple test classes
gradle test -Denvironment=staging Pass system properties
gradle clean test Clean and run tests
gradle test --info Run with detailed logging
gradle test --debug Run with debug-level logging
gradle test --rerun-tasks Force re-run even if up-to-date
gradle test --continue Continue execution after test failures
gradle test --fail-fast Stop execution on first test failure

Assertion Methods

Method Description
Assert.assertEquals(actual, expected) Verify two values are equal
Assert.assertEquals(actual, expected, "message") Assert with custom failure message
Assert.assertNotEquals(actual, expected) Verify two values are not equal
Assert.assertTrue(condition) Verify condition is true
Assert.assertFalse(condition) Verify condition is false
Assert.assertNull(object) Verify object is null
Assert.assertNotNull(object) Verify object is not null
Assert.assertSame(actual, expected) Verify same object reference
Assert.assertNotSame(actual, expected) Verify different object references
Assert.fail("message") Explicitly fail a test
Assert.assertThrows(Exception.class, () -> {...}) Verify exception is thrown
Assert.expectThrows(Exception.class, () -> {...}) Same as assertThrows (alias)

Data Providers

Pattern Description
@DataProvider(name = "testData") Define a data provider method
@Test(dataProvider = "testData") Use data provider in test
@DataProvider(parallel = true) Run data provider iterations in parallel
Object[][] dataProvider() Return 2D array of test data
Iterator<Object[]> dataProvider() Return iterator for large datasets
@DataProvider(indices = {0, 2, 4}) Run only specific data set indices

Data Provider Example

@DataProvider(name = "loginData")
public Object[][] getLoginData() {
    return new Object[][] {
        {"user1", "pass1", true},
        {"user2", "pass2", false},
        {"user3", "pass3", true}
    };
}

@Test(dataProvider = "loginData")
public void testLogin(String username, String password, boolean expected) {
    boolean result = login(username, password);
    Assert.assertEquals(result, expected);
}

TestNG XML Configuration

Basic Suite Configuration

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Test Suite" parallel="methods" thread-count="5" verbose="1">

    <!-- Suite-level parameters -->
    <parameter name="browser" value="chrome"/>
    <parameter name="environment" value="staging"/>

    <!-- Define test groups -->
    <test name="Smoke Tests">
        <groups>
            <run>
                <include name="smoke"/>
                <exclude name="slow"/>
            </run>
        </groups>

        <!-- Specify test classes -->
        <classes>
            <class name="com.example.LoginTest"/>
            <class name="com.example.SearchTest">
                <!-- Include specific methods -->
                <methods>
                    <include name="testBasicSearch"/>
                    <include name="testAdvancedSearch"/>
                </methods>
            </class>
        </classes>
    </test>

    <!-- Another test configuration -->
    <test name="Regression Tests">
        <packages>
            <package name="com.example.regression.*"/>
        </packages>
    </test>

    <!-- Listeners -->
    <listeners>
        <listener class-name="com.example.CustomListener"/>
    </listeners>
</suite>

Parallel Execution Configuration

<!-- Parallel at suite level -->
<suite name="Parallel Suite" parallel="tests" thread-count="3">
    <test name="Test1">...</test>
    <test name="Test2">...</test>
</suite>

<!-- Parallel options: methods, tests, classes, instances -->

Maven Surefire Plugin Configuration

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0</version>
            <configuration>
                <!-- Specify suite files -->
                <suiteXmlFiles>
                    <suiteXmlFile>testng.xml</suiteXmlFile>
                    <suiteXmlFile>smoke-tests.xml</suiteXmlFile>
                </suiteXmlFiles>

                <!-- Run specific groups -->
                <groups>smoke,regression</groups>
                <excludedGroups>slow,manual</excludedGroups>

                <!-- Parallel execution -->
                <parallel>methods</parallel>
                <threadCount>5</threadCount>

                <!-- System properties -->
                <systemPropertyVariables>
                    <browser>chrome</browser>
                    <environment>staging</environment>
                </systemPropertyVariables>

                <!-- Continue on failures -->
                <testFailureIgnore>false</testFailureIgnore>
            </configuration>
        </plugin>
    </plugins>
</build>

Gradle Test Configuration

test {
    useTestNG() {
        // Suite files
        suites 'src/test/resources/testng.xml'

        // Include/exclude groups
        includeGroups 'smoke', 'regression'
        excludeGroups 'slow'

        // Parallel execution
        parallel = 'methods'
        threadCount = 5

        // Preserve order
        preserveOrder = true

        // Group by instances
        groupByInstances = true
    }

    // System properties
    systemProperty 'browser', 'chrome'
    systemProperty 'environment', 'staging'

    // Test output
    testLogging {
        events "passed", "skipped", "failed"
        exceptionFormat "full"
    }
}

Common Use Cases

Use Case 1: Basic Test Class with Setup and Teardown

import org.testng.annotations.*;
import org.testng.Assert;

public class UserManagementTest {

    private DatabaseConnection db;
    private UserService userService;

    @BeforeClass
    public void setupClass() {
        // Initialize database connection once for all tests
        db = new DatabaseConnection("jdbc:mysql://localhost:3306/testdb");
        db.connect();
    }

    @BeforeMethod
    public void setupMethod() {
        // Create fresh service instance before each test
        userService = new UserService(db);
    }

    @Test(priority = 1, groups = {"smoke"})
    public void testCreateUser() {
        User user = userService.createUser("john@example.com", "John Doe");
        Assert.assertNotNull(user.getId());
        Assert.assertEquals(user.getEmail(), "john@example.com");
    }

    @Test(priority = 2, dependsOnMethods = {"testCreateUser"})
    public void testFindUser() {
        User user = userService.findByEmail("john@example.com");
        Assert.assertNotNull(user);
        Assert.assertEquals(user.getName(), "John Doe");
    }

    @AfterMethod
    public void cleanupMethod() {
        // Clean up test data after each test
        userService.deleteAllUsers();
    }

    @AfterClass
    public void cleanupClass() {
        // Close database connection after all tests
        db.disconnect();
    }
}

Use Case 2: Data-Driven Testing with DataProvider

import org.testng.annotations.*;
import org.testng.Assert;

public class LoginTest {

    private LoginPage loginPage;

    @BeforeMethod
    public void setup() {
        loginPage = new LoginPage();
        loginPage.open();
    }

    @DataProvider(name = "loginCredentials")
    public Object[][] getLoginData() {
        return new Object[][] {
            {"valid@user.com", "ValidPass123", true, "Dashboard"},
            {"invalid@user.com", "WrongPass", false, "Invalid credentials"},
            {"", "password", false, "Email is required"},
            {"user@test.com", "", false, "Password is required"},
            {"admin@test.com", "AdminPass!", true, "Admin Panel"}
        };
    }

    @Test(dataProvider = "loginCredentials")
    public void testLogin(String email, String password, 
                         boolean shouldSucceed, String expectedMessage) {
        loginPage.enterEmail(email);
        loginPage.enterPassword(password);
        loginPage.clickLogin();

        if (shouldSucceed) {
            Assert.assertTrue(loginPage.isLoggedIn());
            Assert.assertEquals(loginPage.getPageTitle(), expectedMessage);
        } else {
            Assert.assertTrue(loginPage.hasError());
            Assert.assertTrue(loginPage.getErrorMessage().contains(expectedMessage));
        }
    }

    @AfterMethod
    public void teardown() {
        loginPage.close();
    }
}

Use Case 3: Parallel Test Execution with Groups

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Parallel Test Suite" parallel="tests" thread-count="3">

    <test name="Chrome Tests" parallel="methods" thread-count="2">
        <parameter name="browser" value="chrome"/>
        <groups>
            <run>
                <include name="smoke"/>
            </run>
        </groups>
        <classes>
            <class name="com.example.tests.HomePageTest"/>
            <class name="com.example.tests.SearchTest"/>
        </classes>
    </test>

    <test name="Firefox Tests" parallel="methods" thread-count="2">
        <parameter name="browser" value="firefox"/>
        <groups>
            <run>
                <include name="smoke"/>
            </run>
        </groups>
        <classes>
            <class name="com.example.tests.HomePageTest"/>
            <class name="com.example.tests.SearchTest"/>
        </classes>
    </test>

    <test name="API Tests" parallel="classes" thread-count="3">
        <groups>
            <run>
                <include name="api"/>
            </run>
        </groups>
        <packages>
            <package name="com.example.api.*"/>
        </packages>
    </test>
</suite>
// Test class using parameters
public class CrossBrowserTest {

    private WebDriver driver;

    @Parameters({"browser"})
    @BeforeMethod
    public void setup(String browser) {
        if (browser.equalsIgnoreCase("chrome")) {
            driver = new ChromeDriver();
        } else if (browser.equalsIgnoreCase("firefox")) {
            driver = new FirefoxDriver();
        }
        driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(10));
    }

    @Test(groups = {"smoke"})
    public void testHomePage() {
        driver.get("https://example.com");
        Assert.assertEquals(driver.getTitle(), "Example Domain");
    }

    @AfterMethod
    public void teardown() {
        if (driver != null) {
            driver.quit();
        }
    }
}

Use Case 4: API Testing with Retry Logic

import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;

// Retry analyzer for flaky tests
public class RetryAnalyzer implements IRetryAnalyzer {
    private int retryCount = 0;
    private static final int MAX_RETRY = 3;

    @Override
    public boolean retry(ITestResult result) {
        if (retryCount < MAX_RETRY) {
            retryCount++;
            return true;
        }
        return false;
    }
}

// API Test class
public class APITest {

    private RestClient client;

    @BeforeClass
    public void setup() {
        client = new RestClient("https://api.example.com");
    }

    @Test(groups = {"api"}, retryAnalyzer = RetryAnalyzer.class)
    public void testGetUser() {
        Response response = client.get("/users/1");
        Assert.assertEquals(response.getStatusCode(), 200);
        Assert.assertNotNull(response.jsonPath().getString("name"));
    }

    @Test(groups = {"api"}, dependsOnMethods = {"testGetUser"})
    public void testCreateUser() {
        String payload = "{\"name\":\"John\",\"email\":\"john@test.com\"}";
        Response response = client.post("/users", payload);
        Assert.assertEquals(response.getStatusCode(), 201);
    }

    @Test(groups = {"api"}, timeOut = 5000)
    public void testPerformance() {
        long startTime = System.currentTimeMillis();
        Response response = client.get("/users");
        long endTime = System.currentTimeMillis();

        Assert.assertEquals(response.getStatusCode(), 200);
        Assert.assertTrue((endTime - startTime) < 3000, 
            "API response time exceeded 3 seconds");
    }
}

Use Case 5: Custom Listeners and Reporting

import org.testng.ITestListener;
import org.testng.ITestResult;
import org.testng.ITestContext;

public class CustomTestListener implements ITestListener {

    @Override
    public void onTestStart(ITestResult result) {
        System.out.println("Starting test: " + result.getName());
    }

    @Override
    public void onTestSuccess(ITestResult result) {
        System.out.println("Test passed: " + result.getName());
    }

    @Override
    public void onTestFailure(ITestResult result) {
        System.out.println("Test failed: " + result.getName());
        // Take screenshot, log error, etc.
        captureScreenshot(result.getName());
    }

    @Override
    public void onTestSkipped(ITestResult result) {
        System.out.println("Test skipped: " + result.getName());
    }

    @Override
    public void onFinish(ITestContext context) {
        System.out.println("Total tests run: " + context.getAllTestMethods().length);
        System.out.println("Passed: " + context.getPassedTests().size());
        System.out.println("Failed: " + context.getFailedTests().size());
        System.out.println("Skipped: " + context.getSkippedTests().size());
    }

    private void captureScreenshot(String testName) {
        // Screenshot logic here
    }
}

// Using the listener
@Listeners(CustomTestListener.class)
public class MyTest {

    @Test
    public void testExample() {
        Assert.assertTrue(true);
    }
}

Best Practices

  • Use meaningful test names: Name tests clearly to describe what they verify (e.g., testUserCanLoginWithValidCredentials instead of test1)

  • Leverage groups effectively: Organize tests into logical groups (smoke, regression, api, ui) to run subsets of tests based on context and save execution time

  • Implement proper setup and teardown: Use @BeforeMethod/@AfterMethod for test-level setup and @BeforeClass/@AfterClass for expensive operations like database connections

  • Make tests independent: Each test should be self-contained and not rely on execution order or shared state. Use dependsOnMethods sparingly and only when logical dependency exists

  • Use DataProviders for test data: Separate test data from