Introduction

When working with Playwright in a multi-threaded Java application, ensuring efficient resource management and thread safety is crucial. Playwright is not thread-safe, so each instance must be isolated to avoid concurrency issues. This article introduces a thread-safe Playwright pool that optimizes resource utilization while maintaining thread safety.

By the end of this guide, you will:

Understand the need for a Playwright pool.
Learn how to initialize and manage Playwright instances efficiently.
Use the pool to execute browser automation tasks in a thread-safe manner.

Why Use a Playwright Pool?

In multi-threaded applications, launching a new Playwright instance for each task can be inefficient. A pool enables:

Better resource management: Playwright instances are pre-initialized and reused.
Improved performance: Reduces overhead from frequent browser launches.
Thread safety: Ensures that only one thread interacts with an instance at a time.

Implementing the Playwright Pool

1. Creating the Playwright Pool

The PlaywrightPool class initializes a fixed number of Playwright instances and stores them in a thread-safe queue. Each instance runs in a dedicated thread, ensuring isolation.

public class PlaywrightPool {

    private ExecutorService dedicatedThreads;
    private BlockingQueue<PlaywrightInstance> instances;

    public PlaywrightPool(int poolSize) {
        dedicatedThreads = Executors.newFixedThreadPool(poolSize);
        instances = new LinkedBlockingQueue<>(poolSize);

        for (int i = 0; i < poolSize; i++) {
            dedicatedThreads.submit(() -> {
                Playwright playwright = Playwright.create();
                Browser browser = playwright.chromium().launch(new BrowserType.LaunchOptions()
                        .setHeadless(true)
                        .setArgs(List.of("--disable-gpu", "--no-sandbox")));
                try {
                    instances.put(new PlaywrightInstance(playwright, browser));
                } catch (InterruptedException e) {
                    throw new RuntimeException(e);
                }
            });
        }
    }

Breakdown:

A fixed thread pool is created to match the number of Playwright instances.
Playwright instances are pre-initialized and stored in a blocking queue.
Each instance is launched with headless mode enabled for performance.

2. Executing Playwright Actions Safely

To interact with a browser instance, the usePlaywright method retrieves an available instance, executes an action, and returns the instance to the pool.

public void usePlaywright(PlaywrightAction action) throws Exception {
    PlaywrightInstance instance = instances.take();
    try {
        instance.execute(action);
    } finally {
        instances.put(instance);
    }
}

3. Defining the Playwright Instance Wrapper

Each Playwright instance is wrapped inside PlaywrightInstance, ensuring exclusive access using a reentrant lock.

private static class PlaywrightInstance {
    private final Playwright playwright;
    private final Browser browser;
    private final ReentrantLock lock = new ReentrantLock();

    PlaywrightInstance(Playwright playwright, Browser browser) {
        this.playwright = playwright;
        this.browser = browser;
    }

    void execute(PlaywrightAction action) throws Exception {
        lock.lock();
        var options = generateDefaultContextOptions();

        try (BrowserContext context = browser.newContext(options);
             Page page = context.newPage()) {
            action.run(context, page);
        } finally {
            lock.unlock();
        }
    }
}

Key Features:

Uses a lock mechanism to prevent concurrent access.
Creates a new browser context per execution to ensure session isolation.
Closes the context and page after execution to prevent memory leaks.

4. Customizing Browser Context Options

A default configuration for browser contexts is applied to ensure consistent behavior.

private static Browser.NewContextOptions generateDefaultContextOptions() {
    return new Browser.NewContextOptions()
            .setUserAgent("Mozilla/5.0 ...")
            .setExtraHTTPHeaders(Map.of(
                    "accept", "text/html,application/xhtml+xml,...",
                    "sec-fetch-site", "none"
            ));
}

5. Closing the Pool

To release resources, ensure the pool is closed properly when no longer needed.

public void close() {
    dedicatedThreads.shutdownNow();
    instances.forEach(instance -> {
        instance.browser.close();
        instance.playwright.close();
    });
}

Using the Playwright Pool

To use the pool in your application:

PlaywrightPool pool = new PlaywrightPool(Runtime.getRuntime().availableProcessors());
try {
    pool.usePlaywright((context, page) -> {
        page.navigate("https://example.com");
        System.out.println("Page title: " + page.title());
    });
} finally {
    pool.close();
}

The Full Code

import com.microsoft.playwright.*;

import java.util.List;
import java.util.Map;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.locks.ReentrantLock;

/**
 * A thread-safe pool for managing Playwright instances, designed to optimize resource utilization
 * and performance for CPU-bound Playwright tasks. Each Playwright instance in the pool is confined
 * to its own platform thread, ensuring thread safety as required by Playwright's design.
 *
 * <p>
 * <strong>Usage Example:</strong>
 * <pre>
 * {@code
 * PlaywrightPool pool = new PlaywrightPool(Runtime.getRuntime().availableProcessors());
 * try {
 *     pool.usePlaywright((context, page) -> {
 *         page.navigate("https://example.com");
 *         // Perform Playwright operations
 *     });
 * } finally {
 *     pool.close();
 * }
 * }
 * </pre>
 * </p>
 *
 * <p>
 * <strong>Thread Safety:</strong>
 * The pool is thread-safe and can be shared across multiple virtual threads. Each Playwright instance
 * is locked during use to ensure exclusive access by a single thread at a time.
 * </p>
 *
 * <p>
 * <strong>Resource Management:</strong>
 * The pool automatically initializes and manages Playwright instances, including their cleanup when
 * the {@link #close()} method is called. Pages and browser contexts are created and closed per task
 * to avoid memory leaks.
 * </p>
 */
public class PlaywrightPool {

    private ExecutorService dedicatedThreads;
    private BlockingQueue<PlaywrightInstance> instances;

    public PlaywrightPool(int poolSize) {
        dedicatedThreads = Executors.newFixedThreadPool(poolSize);
        instances = new LinkedBlockingQueue<>(poolSize);

        for (int i = 0; i < poolSize; i++) {
            dedicatedThreads.submit(() -> {
                Playwright playwright = Playwright.create();
                Browser browser = playwright.chromium().launch(new BrowserType.LaunchOptions()
                        .setHeadless(true)
                        .setArgs(List.of("--disable-gpu", "--no-sandbox")));
                try {
                    instances.put(new PlaywrightInstance(playwright, browser));
                } catch (InterruptedException e) {
                    throw new RuntimeException(e);
                }
            });
        }
    }

    public void usePlaywright(PlaywrightAction action) throws Exception {
        PlaywrightInstance instance = instances.take();
        try {
            instance.execute(action);
        } finally {
            instances.put(instance);
        }
    }

    public void close() {
        dedicatedThreads.shutdownNow();
        instances.forEach(instance -> {
            instance.browser.close();
            instance.playwright.close();
        });
    }

    @FunctionalInterface
    public interface PlaywrightAction {
        void run(BrowserContext context, Page page) throws Exception;
    }

    private static class PlaywrightInstance {

        private static Browser.NewContextOptions generateDefaultContextOptions() {
            return new Browser.NewContextOptions()
                    .setUserAgent("Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36")
                    .setExtraHTTPHeaders(Map.of(
                            "accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
                            "accept-encoding", "gzip, deflate, br, zstd",
                            "accept-language", "en-US,en;q=0.9",
                            "sec-ch-ua", "\"Not A(Brand\";v=\"8\", \"Chromium\";v=\"132\", \"Google Chrome\";v=\"132\"",
                            "sec-ch-ua-mobile", "?0",
                            "sec-ch-ua-platform", "\"macOS\"",
                            "sec-fetch-dest", "document",
                            "sec-fetch-mode", "navigate",
                            "sec-fetch-site", "none"
                    ));
        }

        private final Playwright playwright;
        private final Browser browser;
        private final ReentrantLock lock = new ReentrantLock();

        PlaywrightInstance(Playwright playwright, Browser browser) {
            this.playwright = playwright;
            this.browser = browser;
        }

        void execute(PlaywrightAction action) throws Exception {
            lock.lock();
            var options = generateDefaultContextOptions();

            try (BrowserContext context = browser.newContext(options);
                 Page page = context.newPage()) {
                action.run(context, page);
            } finally {
                lock.unlock();
            }
        }

    }

}

Conclusion

The PlaywrightPool class provides a high-performance, thread-safe solution for executing Playwright tasks in a multi-threaded environment. By pooling instances, it reduces overhead while maintaining session isolation and resource efficiency.

With this approach, you can execute concurrent browser automation tasks efficiently, making it ideal for web scraping, testing, and automation in large-scale applications.

Optimizing Playwright Performance with a Thread-Safe Pool in Java

Table of contents