Thread Pool Pattern

The Thread Pool Pattern is a design pattern that reduces the overhead associated with thread creation and management in concurrent programming. It maintains a pool of worker threads that can efficiently execute tasks without the need to create a new thread for each task, which can be resource-intensive.

• **Thread Pool**: A collection of pre-initialized threads that are ready to execute tasks.
• **Worker Thread**: A thread that executes tasks from the thread pool.
• **Task Queue**: A queue that holds tasks waiting to be executed by worker threads.
• **Concurrency**: The ability to run multiple tasks simultaneously.

Here's a simple implementation of a Thread Pool in Python:

import threading
import queue
import time

class ThreadPool:
    def __init__(self, num_threads):
        self.tasks = queue.Queue()
        self.threads = [
            threading.Thread(target=self.worker) for _ in range(num_threads)
        ]
        for thread in self.threads:
            thread.start()

    def worker(self):
        while True:
            task, args = self.tasks.get()
            if task is None:
                break
            task(*args)
            self.tasks.task_done()

    def add_task(self, task, *args):
        self.tasks.put((task, args))

    def wait_completion(self):
        self.tasks.join()

    def shutdown(self):
        for _ in self.threads:
            self.add_task(None)
        for thread in self.threads:
            thread.join()

def sample_task(duration):
    print(f"Task started for {duration} seconds")
    time.sleep(duration)
    print("Task completed")

if __name__ == "__main__":
    pool = ThreadPool(3)
    for i in range(5):
        pool.add_task(sample_task, i + 1)
    pool.wait_completion()
    pool.shutdown()
            

• **Reuse Threads**: Avoid creating new threads frequently; use a pool of threads instead.
• **Limit Pool Size**: Set a maximum number of threads to prevent resource exhaustion.
• **Handle Exceptions**: Implement error handling in tasks to prevent thread termination.
• **Graceful Shutdown**: Ensure the pool can shut down gracefully, allowing pending tasks to complete.