[issue26692] cgroups support in multiprocessing

2018-01-16 Thread David Chin
David Chin added the comment: I would like to state strong support if is.get_usable_cpu_count() I administer a typical HPC cluster which may have multiple jobs scheduled on the same physical server. The fact that multiprocessing ignores cgroups leads to bad oversubscription. -- nosy:

[issue26692] cgroups support in multiprocessing

2017-11-07 Thread Mihai Capotă
Change by Mihai Capotă : -- nosy: +mihaic ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.

[issue26692] cgroups support in multiprocessing

2017-09-04 Thread Giampaolo Rodola'
Changes by Giampaolo Rodola' : -- nosy: +giampaolo.rodola ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https

[issue26692] cgroups support in multiprocessing

2017-07-23 Thread Antoine Pitrou
Antoine Pitrou added the comment: Agreed that it is not possible for multiprocessing to choose an optimal default in all settings. However, making the default adequate for more use cases sounds like a reasonable goal. Currently, we are using `os.cpu_count()`. Ideally, we would have a second A

[issue26692] cgroups support in multiprocessing

2017-07-23 Thread Charles-François Natali
Charles-François Natali added the comment: I'm not convinced. The reason is that using the number of CPU cores is just a heuristic for a *default value*: the API allows the user to specify the number of workers to use, so it's not really a limitation. The problem is that if you try to think abou

[issue26692] cgroups support in multiprocessing

2017-07-22 Thread Antoine Pitrou
Changes by Antoine Pitrou : -- nosy: +neologix, pitrou ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://

[issue26692] cgroups support in multiprocessing

2017-07-22 Thread Antoine Pitrou
Changes by Antoine Pitrou : -- stage: -> needs patch type: behavior -> enhancement versions: +Python 3.7 -Python 3.6 ___ Python tracker ___ _

[issue26692] cgroups support in multiprocessing

2016-04-04 Thread SilentGhost
Changes by SilentGhost : -- nosy: +jnoller, sbt versions: +Python 3.6 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubsc

[issue26692] cgroups support in multiprocessing

2016-04-04 Thread Satrajit Ghosh
New submission from Satrajit Ghosh: multiprocessing cpucount returns the number of cpus on the system as returned by /proc/cpuinfo. this is true even on machines where linux kernel cgroups is being used to restrict cpu usage for a given process. this results in significant thread swithcing o