rex.utilities.execution.SpawnProcessPool

class SpawnProcessPool(*args, loggers=None, **kwargs)

Bases: ProcessPoolExecutor

An adaptation of concurrent futures ProcessPoolExecutor with spawn processes instead of fork or forkserver.

Parameters:

loggers (str | list, optional) – logger(s) to initialize on workers, by default None

Methods

kill_workers()	Attempts to kill the executor's workers.
map(fn, *iterables[, timeout, chunksize, ...])	Returns an iterator equivalent to map(fn, iter).
shutdown([wait, cancel_futures])	Clean-up the resources associated with the Executor.
submit(fn, /, *args, **kwargs)	Submits a callable to be executed with the given arguments.
terminate_workers()	Attempts to terminate the executor's workers.

kill_workers()

Attempts to kill the executor’s workers. Iterates through all of the current worker processes and kills each one that is still alive.

After killing workers, the pool will be in a broken state and no longer usable (for instance, new tasks should not be submitted).

map(fn, *iterables, timeout=None, chunksize=1, buffersize=None)

Returns an iterator equivalent to map(fn, iter).

Args:

fn: A callable that will take as many arguments as there are

passed iterables.

timeout: The maximum number of seconds to wait. If None, then

there is no limit on the wait time.

chunksize: If greater than one, the iterables will be chopped

into chunks of size chunksize and submitted to the process pool. If set to one, the items in the list will be sent one at a time.

buffersize: The number of submitted tasks whose results have not

yet been yielded. If the buffer is full, iteration over the iterables pauses until a result is yielded from the buffer. If None, all input elements are eagerly collected, and a task is submitted for each.

Returns:

An iterator equivalent to: map(func, *iterables) but the calls may be evaluated out-of-order.

Raises:

TimeoutError: If the entire result iterator could not be

generated before the given timeout.

Exception: If fn(*args) raises for any values.

shutdown(wait=True, *, cancel_futures=False)

Clean-up the resources associated with the Executor.

It is safe to call this method several times. Otherwise, no other methods can be called after this one.

Args:

wait: If True then shutdown will not return until all running

futures have finished executing and the resources used by the executor have been reclaimed.

cancel_futures: If True then shutdown will cancel all pending

futures. Futures that are completed or running will not be cancelled.

submit(fn, /, *args, **kwargs)

Submits a callable to be executed with the given arguments.

Schedules the callable to be executed as fn(*args, **kwargs) and returns a Future instance representing the execution of the callable.

Returns:

A Future representing the given call.

terminate_workers()

Attempts to terminate the executor’s workers. Iterates through all of the current worker processes and terminates each one that is still alive.

After terminating workers, the pool will be in a broken state and no longer usable (for instance, new tasks should not be submitted).