Thread Pool
Abdullah KhairyThis is an explanation of C++ thread pools — what they're used for, why they're needed, and how they work, in a way that teaches the topic step-by-step:
🔧 What Is a Thread Pool?
A thread pool is a group of worker threads (threads that always running and, waiting for tasks) that are created once and then reused to execute multiple tasks.
Think of it like a team of barber shop workers (threads) waiting in a room (the pool). When a new Customer (task) arrives, you assign it to one of the available workers. When the job is done, the worker goes back to the rest room, ready for the next Customer.
🤔 Why Do We Need a Thread Pool?
1. Avoid the Cost of Creating Threads Repeatedly
Creating and destroying threads is expensive (in time and system resources). If your program spawns a new thread for every small task, performance will suffer.
A thread pool solves this by:
Creating a fixed number of threads once.
Reusing them to do multiple jobs.
2. Efficient Task Management
When many short-lived tasks need to run (e.g., image processing, network handling, or parallel computations), thread pools let you:
Manage concurrency easily.
Prevent too many threads from overwhelming the system.
3. Better Control of System Resources
If you allow unlimited threads:
You may overload the CPU or memory.
Your app can crash or slow down.
A thread pool limits the number of threads (e.g., based on CPU cores), giving predictable performance.
🔨 How Does a Thread Pool Work?
Here's the basic idea:
Initialize the Pool
Create a fixed number of threads (e.g., 4 or 8).
Each thread waits for a task in a queue.
Add Tasks to the Queue usually named enqueue() or submit()
- When your app has a job (e.g., process a file, handle a request), it puts the job into the task queue.
Worker Threads Pull from the Queue
Each thread waits for tasks.
When a task appears, a thread grabs it, does the work, then waits for the next one.
Shutdown (Optional)
- Once all work is done, you can signal threads to stop and clean up.
🧠Real-World Example Use Cases
Web servers handling many requests (each request is a task).
Multimedia apps processing frames or audio.
Games performing AI updates or physics simulations in parallel.
Data processing pipelines (e.g., encoding, filtering).
</> Code Example
- to the point code thread return void
#include <iostream>
#include <vector>
#include <queue>
#include <thread>
#include <mutex>
#include <condition_variable>
#include <functional>
#include <atomic>
class ThreadPool {
public:
// Constructor: initializes the pool with a given number of threads
ThreadPool(size_t numThreads) : stop(false) {
for (size_t i = 0; i < numThreads; ++i) {
// Each worker runs this lambda in its own thread
workers.emplace_back([this]() {
while (true) {
std::function<void()> task;
// Scoped locking to safely access shared queue
{
std::unique_lock<std::mutex> lock(queueMutex);
// Wait until a task is available or pool is stopping
condition.wait(lock, [this]() {
return stop || !tasks.empty();
});
// If we're stopping and no tasks remain, exit thread
if (stop && tasks.empty())
return;
// Get the next task from the queue
task = std::move(tasks.front());
tasks.pop();
}
// Execute the task
task();
}
});
}
}
// Add a new task to the pool
void enqueue(std::function<void()> task) {
{
std::unique_lock<std::mutex> lock(queueMutex);
if (stop)
throw std::runtime_error("Enqueue on stopped ThreadPool");
tasks.push(std::move(task));
}
// Notify one worker that a task is ready
condition.notify_one();
}
// Destructor: stops the pool and joins all threads
~ThreadPool() {
{
std::unique_lock<std::mutex> lock(queueMutex);
stop = true;
}
// Wake up all threads so they can exit
condition.notify_all();
// Join all threads (wait for them to finish)
for (std::thread &worker : workers)
worker.join();
}
private:
std::vector<std::thread> workers; // Thread pool
std::queue<std::function<void()>> tasks; // Task queue
std::mutex queueMutex; // Protects task queue
std::condition_variable condition; // Signals available work
std::atomic<bool> stop; // Stop flag
};
- advanced code
#include <iostream>
#include <vector>
#include <thread>
#include <mutex>
#include <condition_variable>
#include <queue>
#include <functional>
#include <future>
#include <atomic>
class ThreadPool {
private:
std::vector<std::thread> workers; // The worker threads
std::queue<std::function<void()>> tasks; // Task queue (FIFO)
std::mutex queue_mutex; // Protects access to the queue
std::condition_variable cv; // Notifies worker threads of new tasks
std::atomic<bool> stop{false}; // Flag to indicate shutdown
public:
// Constructor: create worker threads
ThreadPool(size_t num_threads) {
for (size_t i = 0; i < num_threads; ++i) {
workers.emplace_back([this] {
while (true) {
std::function<void()> task;
// Acquire task from the queue
{
std::unique_lock<std::mutex> lock(queue_mutex);
// Wait until either:
// - there's a task available
// - OR the pool is stopping
cv.wait(lock, [this] {
return !tasks.empty() || stop;
});
// Exit if pool is stopping AND no tasks left
if (stop && tasks.empty())
return;
// Take the next task from the queue
task = std::move(tasks.front());
tasks.pop();
}
// Execute the task (outside the lock)
task();
}
});
}
}
// Adds a new task to the thread pool and returns a future result
// 'enqueue' takes a callable 'f' and any number of arguments 'args...',
// deduces the return type of invoking 'f(args...)',
// and returns a std::future that will eventually hold the result.
template <typename F, typename... Args>
auto enqueue(F&& f, Args&&... args) -> std::future<decltype(f(args...))> {
// Deduce the return type of the callable 'f' when invoked with arguments 'args...'
// Example: If f is a function returning int, return_type becomes int
using return_type = decltype(f(args...));
// Wrap the task and arguments into a packaged_task
// Create a packaged_task to wrap the callable and its arguments:
// 1. std::bind + perfect forwarding preserves argument value categories (lvalue/rvalue)
// 2. Converts f(args...) to a nullary function (no-argument function)
// 3. Wrapped in shared_ptr for lifetime management across threads
auto task = std::make_shared<std::packaged_task<return_type()>>(
std::bind(std::forward<F>(f), std::forward<Args>(args)...)
);
// Obtain a future linked to the packaged_task's result:
// - The future/promise pair shares a state
// - Task stores result/exception in shared state when executed
// - res.get() will block until result is available
std::future<return_type> res = task->get_future(); // Get the associated future
{
std::unique_lock<std::mutex> lock(queue_mutex);
// If pool is stopped, throw an exception
if (stop)
throw std::runtime_error("Enqueue on stopped ThreadPool");
// Add the task to the queue (as a void() function)
tasks.emplace([task]() { (*task)(); });
}
// Notify one worker that there's a new task
cv.notify_one();
return res; // Return the future so the caller can get the result later
}
// Destructor: clean up the threads and shutdown the pool
~ThreadPool() {
{
std::unique_lock<std::mutex> lock(queue_mutex);
stop = true; // Signal to stop
}
cv.notify_all(); // Wake up all threads
// Join all worker threads
for (std::thread& worker : workers) {
if (worker.joinable())
worker.join();
}
}
};
// Example usage
int main() {
ThreadPool pool(4); // Create a pool with 4 worker threads
std::vector<std::future<int>> results;
// Submit 8 tasks to the pool
for (int i = 0; i < 8; ++i) {
results.emplace_back(pool.enqueue([i] {
std::cout << "Task " << i << " executed by thread "
<< std::this_thread::get_id() << std::endl;
std::this_thread::sleep_for(std::chrono::milliseconds(100)); // Simulate work
return i * i; // Return square of i
}));
}
// Retrieve results from futures (this blocks until each task is done)
for (auto&& result : results) {
std::cout << "Result: " << result.get() << std::endl;
}
return 0;
}
Subscribe to my newsletter
Read articles from Abdullah Khairy directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by