Module src.qollib.processing.execution
provides housekeeping / setup methods to reduce the programming overhead of spawning threads or processes.
Global variables
var CPU_LIM-
70% of CPUs from the current machine (max usage limit)
Functions
def simultaneous(func, args, workers: int = None, raise_exception: bool = True)-
Calls the given function in multiple processes for the set of given arguments Note that this does spawn processes, not threads. Use this for task that depend heavily on CPU and can be done in parallel. Method returns once all calls are done.
Params
- func: [Function] the function to call
- args: [Iterable] the 'list' of arguments for each call
- workers: [Integer] the number of concurrent threads to use (Default: NUM_CPUs)
- raise_exception: [Bool] Flag if an exception in a thread shall be raised or just logged
Returns
Results from all
Threadsas list def threaded(func, args, workers=10, raise_exception=True)-
Calls the given function in multiple threads for the set of given arguments Note that this does not spawn processes, but threads. Use this for non CPU CPU dependent tasks, i.e. I/O Method returns once all calls are done.
Params
- func: [Function] the function to call
- args: [Iterable] the 'list' of arguments for each call
- workers: [Integer] the number of concurrent threads to use
- raise_exception: [Bool] Flag if an exception in a thread shall be raised or just logged
Returns
Results from all
Threadsas list