Inconsistent KeyboardInterrupt Handling in multiprocessing due to Context Manager Order

[hcording]

Bug report

Bug description:

When using multiprocessing.Manager with concurrent.futures.ProcessPoolExecutor, there is a particular ordering of the context managers that results in multiprocessing.managers.py hanging in the method serve_forever, forever. This can be triggered by raising a KeyboardInterrupt while the child process(es), here something, are busy. The ordering that leads to the bug is: Manager first, then ProcessPoolExecutor inside.

import multiprocessing
from concurrent.futures import ProcessPoolExecutor
from time import sleep


def something():
    sleep(10)

# Uncomment one of the following blocks

### Works correctly:

# # Interrupt with CTRL+C while `something` is busy
# # Takes just one KeyboardInterrupts to terminate fully
# with ProcessPoolExecutor() as executor:
#     futures = []
#     with multiprocessing.Manager() as manager:
#         futures.append(executor.submit(something))
#
#         for f in futures:
#             f.result()

### Doesn't work correctly, will hang often, try it a few times:

# # Interrupt with CTRL+C while `something` is busy
# # Takes one KeyboardInterrupts to get stuck, and another to terminate fully
# with multiprocessing.Manager() as manager:
#     with ProcessPoolExecutor() as executor:
#         futures = [executor.submit(something)]
# 
#         for f in futures:
#             f.result()

I am not too familiar with the exact inner workings of these two context managers, but as a user, there was at least nothing to make me aware that the 2nd example is bad. If it's not a bug, and just incorrect ordering, perhaps ProcessPoolExecutor could raise an exception or print a warning that it shouldn't be used inside a Manager context in such a way.

CPython versions tested on:

3.9, 3.10, 3.11, 3.12

Operating systems tested on:

Linux