pyufunc.run_parallel#

pyufunc.run_parallel(func, iterable, num_processes=None, chunksize=0)#

Run a function in parallel with multiple processors.

Parameters:

  • func (callable) – The function to run in parallel.

  • iterable (Iterable) – The input iterable to the function.

  • num_processes (int, optional) – The number of processors to use. Defaults to os.cpu_count() - 1.

  • chunksize (int, optional) – The chunksize for the parallel processing. Defaults to 0.

Raises:

  • TypeError – If the input function is not callable, - or if chunksize or num_processes are not integers, - or if iterable is not an Iterable.

  • TypeError – if the input iterable is not an Iterable - The input iterable should be an Iterable.

  • TypeError – if the input number of processors is not an integer

  • TypeError – if the input chunksize should be an integer

  • ValueError – if the input number of processors is not greater than 0

  • ValueError – if the input chunksize is not greater than 0

Examples

>>> import numpy as np
>>> from pyufunc import run_parallel
>>> def my_func(x):
>>>     print(f"running {x}")
>>>    return x**2
>>> results = run_parallel(my_func, list(range(10)), num_processes=7)
>>> for res in results:
>>>    print(res)
>>> 0, 1, 4, 9, 16, 25, 36, 49, 64, 81
Returns:

The results of the function.

Return type:

list