Concurrent.futures Module in Python

The concurrent.futures module in Python provides a high-level interface for asynchronously executing callables using threads or processes. It simplifies the task of concurrent programming.

This module is especially useful for I/O-bound and CPU-bound tasks, allowing developers to leverage multi-core processors and improve performance.

• Executor: An abstract class that provides methods to execute calls asynchronously.
• ThreadPoolExecutor: An implementation of Executor that uses a pool of threads.
• ProcessPoolExecutor: An implementation of Executor that uses a pool of processes.
• Future: A placeholder for a result that hasn’t been computed yet.

3.1. Using ThreadPoolExecutor

The ThreadPoolExecutor is suitable for I/O-bound tasks. Below is a simple example:

from concurrent.futures import ThreadPoolExecutor
import time

def fetch_data(seconds):
    time.sleep(seconds)
    return f"Fetched data after {seconds} seconds"

with ThreadPoolExecutor() as executor:
    futures = [executor.submit(fetch_data, i) for i in range(1, 4)]
    for future in futures:
        print(future.result())

3.2. Using ProcessPoolExecutor

The ProcessPoolExecutor is suitable for CPU-bound tasks. Below is an example:

from concurrent.futures import ProcessPoolExecutor
import math

def compute_factorial(n):
    return math.factorial(n)

with ProcessPoolExecutor() as executor:
    results = executor.map(compute_factorial, range(1, 6))
    for result in results:
        print(result)

• Always use context managers (with statement) when creating executors to ensure proper clean-up.
• Limit the number of workers to the number of available CPU cores for CPU-bound tasks.
• Handle exceptions from futures to avoid crashing the program.
• Avoid sharing mutable state between threads or processes.