ashb opened a new pull request, #23944: URL: https://github.com/apache/airflow/pull/23944
Something changed between Python 3.7 and 3.10 meaning that a limited parallelism LocalExecutor scheduler now doesn't shutdown cleanly on receiving a signal. On closer investigation it appears that much of the code we are doing to manage this is a poor imitation of the built in concurrent.futures.ProcessPoolExecutor, so re-write it to use just that. I have also removed the Unlimited/Limited versions and simplified the code massively by having just a single version and "unlimited" is now implemented as having a limit of 2 billion concurrent processes. Should be enough for anyone -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
