Bug report
Bug description:
Below code demonstrates two problems that occur when
(a) worker process throws an exception, and
(b) shutdown(wait=False) is used
- When max_workers < number of submitted tasks, raises
exception TypeError: object of type 'NoneType' has no len()
- When max_workers > number of submitted tasks, raises
exception TypeError: object of type 'NoneType' has no len() AND then program hangs
There is no problem when shutdown(wait=True) is used or if the worker processes exit cleanly.
import concurrent.futures
import multiprocessing as mp
import time
def task(n: int) -> int:
if n == 2:
raise Exception("Not gonna do it")
else:
time.sleep(n)
return n
def main() -> None:
# max_workers=2 results in raised exception and program exits
# max_workers=4 results in raised exception and program hangs
#
executor = concurrent.futures.ProcessPoolExecutor(
max_workers=2, mp_context=mp.get_context("forkserver"), max_tasks_per_child=1
)
f1 = executor.submit(task, 1)
f2 = executor.submit(task, 2)
f3 = executor.submit(task, 3)
result = 0
try:
result += f1.result()
result += f2.result()
result += f3.result()
print(f"Result = {result}")
except Exception as e:
print(f"Exception while getting result : {e}")
executor.shutdown(wait=False)
if __name__ == "__main__":
main()CPython versions tested on:
3.12
Operating systems tested on:
Linux, macOS
Linked PRs
Bug report
Bug description:
Below code demonstrates two problems that occur when
(a) worker process throws an exception, and
(b)
shutdown(wait=False)is usedexception TypeError: object of type 'NoneType' has no len()exception TypeError: object of type 'NoneType' has no len()AND then program hangsThere is no problem when
shutdown(wait=True)is used or if the worker processes exit cleanly.CPython versions tested on:
3.12
Operating systems tested on:
Linux, macOS
Linked PRs