Selenium Java - Extract and Print Web Table Data

Introduction

In this blog, we'll dive into using Selenium WebDriver with Java to grab and print all the data from a web table, even if it spans multiple pages. This is super handy for automating tasks like data scraping or checking table content in web apps. Let's get started!

Extracting and Printing Web Table Data

Below is a comprehensive Selenium code example that demonstrates how to navigate through a paginated web table, retrieve data, and store it in custom objects for easy processing.

java

package sa.Demo.SeleniumPractice;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.annotations.Test;

import java.time.Duration;
import java.util.ArrayList;
import java.util.List;

public class Selenium_083 {
    /****
     *  Sample Code Definition :- Selenium - WebTable - Print All Table Data
     *  Developed By :- Pankaj Kumar
     *  Date of Creation :- 15-June-2024
     *  Project :- SAJVSAUTO - Java [Web / Mobile / API]
     *  SA_BLOG_CODE :- SAJVSAUTO_083
     */
    @Test
    public void Selenium_083() throws InterruptedException {
        System.out.println("Selenium - WebTable - Print All Table Data");
        // Open Chrome Browser
        WebDriver driver = new ChromeDriver();
        // Maximize the Browser Window
        driver.manage().window().maximize();
        // Setup the implicit Wait
        driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(30));
        // Open the URL
        driver.get("https://practice.solveautomation.in");
        // Wait for 5 Seconds
        Thread.sleep(5000);
        // Click on WebTable
        driver.findElement(By.xpath("//a[normalize-space()='WebTable']")).click();
        // Wait for 5 Seconds to observe the action (optional)
        Thread.sleep(5000);
        // List to store CountryInfo objects
        List<CountryInfo> countries = new ArrayList<>();
        // Loop to navigate through all pages
        int i = 1;
        while (i < 19) {
            // Find the table
            WebElement table = driver.findElement(By.xpath("//table//tbody"));
            // Get all rows of the table
            List<WebElement> rows = table.findElements(By.tagName("tr"));
            // Loop through all rows and columns to get data
            for (WebElement row : rows) {
                List<WebElement> columns = row.findElements(By.tagName("td"));
                String country = columns.get(1).getText();
                String capital = columns.get(2).getText();
                String currency = columns.get(3).getText();
                String language = columns.get(4).getText();
                CountryInfo objData = new CountryInfo(country, capital, currency, language);
                countries.add(objData);
            }
            // Click on Next Page
            driver.findElement(By.xpath("//button[normalize-space()='>']")).click();
            i++;
        }
        // Print All Data
        System.out.println("Total Country Found in List ==> " + countries.size());
        for (CountryInfo country : countries) {
            System.out.println(country.toString());
        }
        // Quit the Driver
        driver.quit();
    }

    public class CountryInfo {
        private String country;
        private String capital;
        private String currency;
        private String language;

        // Constructor
        public CountryInfo(String country, String capital, String currency, String language) {
            this.country = country;
            this.capital = capital;
            this.currency = currency;
            this.language = language;
        }

        // toString method for easy printing
        @Override
        public String toString() {
            return "Country: " + country + ", Capital: " + capital + ", Currency: " + currency + ", Language: " + language;
        }
    }
}

Explanation

• Browser Setup: Configures ChromeDriver, maximizes the browser window, and applies an implicit wait for smooth operation.

• Web Table Navigation: Opens the web application, navigates to the WebTable page, and handles pagination to process data from multiple pages.

• Data Extraction:

  • Fetches table rows and columns using XPath locators.

  • Extracts relevant information (country, capital, currency, and language) and stores it in a CountryInfo custom object.

• Data Storage and Printing:

  • Stores all extracted data in a list of CountryInfo objects.

  • Prints the size of the list and iterates through it to display each entry.

• Driver Cleanup: Ensures proper closure of the WebDriver session to release resources.

Conclusion

By using Selenium WebDriver with Java, you can easily automate the task of extracting and printing data from web tables, even if they have multiple pages. This approach is great for testing and data scraping, making your automation tasks much smoother and more efficient.