New submission from Julian Taylor: multiprocessing.cpu_count and os.cpu_count which are often used to determine how many processes one can run in parallel do not respect the cpuset which may limit the process to only a subset of online cpus leading to heavy oversubscription in e.g. containerized environments:
$ taskset -c 0 python3.4 -c 'import multiprocessing; print(multiprocessing.cpu_count())' 32 $ taskset -c 0 python3.4 -c 'import os; print(os.cpu_count())' 32 While the correct result here should be 1. This requires programs to have to use less portable methods like forking to gnu nproc or having to read the /proc filesystem. Having a keyword argument to switch between online and available cpus would be fine too. ---------- components: Library (Lib) messages: 236671 nosy: jtaylor priority: normal severity: normal status: open versions: Python 3.4 _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue23530> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
