Returning Values from Threads

In multithreaded programming, it is often necessary to retrieve values returned by threads after they have completed their execution. However, threads do not directly support returning values like functions do. In this tutorial, we will explore how to overcome this limitation and effectively retrieve return values from threads in Python.

Understanding the Problem

When you start a thread using the threading module in Python, it runs concurrently with your main program flow. The start() method initiates the execution of the thread’s target function, but it does not wait for the thread to finish. Instead, you use the join() method to pause the main program until the thread completes. However, join() itself returns None, meaning you cannot directly retrieve the return value of the thread’s target function through this method.

Using Shared State

One approach to getting around this limitation is by using shared state between threads. You can pass a mutable object (like a list or dictionary) as an argument to the thread’s target function, and have it modify that object with its result. Here’s an example:

import threading

def worker(result_list):
    # Simulate some work
    result = "Worker result"
    print("Worker finished")
    result_list.append(result)

result_list = []
thread = threading.Thread(target=worker, args=(result_list,))
thread.start()
thread.join()

print(result_list[0])  # Prints: Worker result

This method works but requires careful synchronization if multiple threads are accessing the shared state.

Extending Thread Class

Another approach is to subclass Thread and override its methods to store the return value. However, due to Python’s name mangling for private variables, this can get complex:

import threading

class ThreadWithReturnValue(threading.Thread):
    def __init__(self, group=None, target=None, name=None,
                 args=(), kwargs={}, daemon=None):
        threading.Thread.__init__(self, group, target, name, args, kwargs, daemon=daemon)
        self._return = None

    def run(self):
        if self._target is not None:
            self._return = self._target(*self._args,
                                        **self._kwargs)

    def join(self, *args):
        threading.Thread.join(self, *args)
        return self._return

def worker(bar):
    print(f"Hello {bar}")
    return "Worker result"

thread = ThreadWithReturnValue(target=worker, args=("World",))
thread.start()
result = thread.join()
print(result)  # Prints: Worker result

Using concurrent.futures

The most straightforward and modern way to handle threads and retrieve their return values in Python is by using the concurrent.futures module. This module provides a high-level interface for asynchronously executing callables.

import concurrent.futures

def worker(bar):
    print(f"Hello {bar}")
    return "Worker result"

with concurrent.futures.ThreadPoolExecutor() as executor:
    future = executor.submit(worker, "World")
    result = future.result()
    print(result)  # Prints: Worker result

The submit() method schedules the execution of a callable and returns a Future object representing the result. You can then call the result() method on this Future to wait for the computation to complete and retrieve its result.

Conclusion

Returning values from threads in Python requires some creativity due to the nature of threading. By using shared state, extending the Thread class, or leveraging the concurrent.futures module, you can effectively retrieve return values from threads. The choice of method depends on your specific requirements and the complexity of your application.