Hi guys, I want to have a version of `concurrent.futures.ProcessPoolExecutor` in which every worker process shuts down after each task. I want this because I believe I have a memory leak in my program that I wasn't able to fix. (Possibly it's in C extensions that I use.)
Since I'm going to use a process pool anyway, I'm hoping to sidestep this memory leak by launching every task on a worker process that shuts down after it's done. A few years back when I was working at Dell, they did something similar. They had a Python server that had a memory leak, and they worked around it by having a process that can be easily restarted, and then they restarted it often. The plan is that each new work item will run on a fresh process that has no memory leaks from previous work items. I looked at the `futures/process.py` file to see whether I could subclass `ProcessPoolExecutor` and add this functionality, and boy oh boy this is not easy. Do you think it's worthwhile to add this feature to `ProcessPoolExecutor`? Thanks, Ram.
_______________________________________________ Python-ideas mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/[email protected]/message/UNDY7YKBJ4DOB6H45N7T6QK7PVJBIKZV/ Code of Conduct: http://python.org/psf/codeofconduct/
