On Tue, Sep 5, 2017 at 12:59 PM, Nathaniel Smith <[email protected]> wrote:
> On Mon, Sep 4, 2017 at 8:59 PM, Giampaolo Rodola' <[email protected]> > wrote: > > Recently os.cpu_count() on Windows has been fixed in order to take > process > > groups into account and return the number of all available CPUs: > > http://bugs.python.org/issue30581 > > This made me realize that os.cpu_count() does not return the number of > > *usable* CPUs, which could possibly represent a better default value for > > things like multiprocessing and process pools. > > It is currently possible to retrieve this info on UNIX with > > len(os.sched_getaffinity(0)) which takes CPU affinity and (I think) Linux > > cgroups into account, but it's not possible to do the same on Windows > which > > provides this value via GetActiveProcessorCount() API. > > As such I am planning to implement this in psutil but would like to know > how > > python-ideas feels about adding a new os.usable_cpu_count() function (or > > having os.cpu_count(usable=True)). > > This was discussed in https://bugs.python.org/issue23530 > > It looks like the resolution at that time was: > > - os.cpu_count() should *not* report the number of CPUs accessible to > the current process, but rather continue to report the number of CPUs > that exist in the system (whatever that means in these days of > virtualization... e.g. if you use KVM to set up a virtual machine with > limited CPUs, then that changes os.cpu_count, but if you do it with > docker then that doesn't). > > - multiprocessing and similar should continue to treat os.cpu_count() > as if it returned the number of CPUs accessible to the current > process. > > Possibly some lines got crossed there... > > -n > > -- > Nathaniel J. Smith -- https://vorpus.org > I agree current os.cpu_count() behavior should remain unchanged. Point is multiprocessing & similar are currently not taking CPU affinity and Linux cgroups into account, spawning more processes than necessary. -- Giampaolo - http://grodola.blogspot.com
_______________________________________________ Python-ideas mailing list [email protected] https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
