[issue17914] add os.cpu_count()

Mark Summerfield added the comment: Since this is closed I've created a new issue as requested: http://bugs.python.org/issue23037 -- ___ Python tracker ___ __

[issue17914] add os.cpu_count()

STINNER Victor added the comment: > From my reading, and based on feedback from one of my customers, I believe he > is correct and that GetSystemInfo() ought to be used on Windows. (It is > available in pywin32 win32api.) Please open a new issue to suggest this enhancement, this issue is close

[issue17914] add os.cpu_count()

Mark Summerfield added the comment: In message http://bugs.python.org/issue17914#msg188626 Victor Stenner says "On Windows, GetSystemInfo() is called instead of reading an environment variable. I suppose that this function is more reliable." >From my reading, and based on feedback from one of

[issue17914] add os.cpu_count()

Changes by Charles-François Natali : -- resolution: -> fixed stage: patch review -> committed/rejected status: open -> closed ___ Python tracker ___

[issue17914] add os.cpu_count()

Roundup Robot added the comment: New changeset 6a0437adafbd by Charles-François Natali in branch 'default': Issue #17914: Use os.cpu_count() instead of multiprocessing.cpu_count() where http://hg.python.org/cpython/rev/6a0437adafbd -- ___ Python track

[issue17914] add os.cpu_count()

Changes by Charles-François Natali : -- stage: needs patch -> patch review ___ Python tracker ___ ___ Python-bugs-list mailing list Un

[issue17914] add os.cpu_count()

Charles-François Natali added the comment: > * rewrite Mac OS X implementation: code in 5e0c56557390 looks wrong. It > gets a MIB but then don't use it when calling _bsd_cpu_count(). But I didn't > check my patch nor the commit version on Mac OS X. Indeed. I just removed the OS-X special case a

[issue17914] add os.cpu_count()

Roundup Robot added the comment: New changeset f9d815522cdb by Charles-Francois Natali in branch 'default': Issue #17914: We can now inline _bsd_cpu_count(). http://hg.python.org/cpython/rev/f9d815522cdb -- ___ Python tracker

[issue17914] add os.cpu_count()

Roundup Robot added the comment: New changeset a85ac58e9eaf by Charles-Francois Natali in branch 'default': Issue #17914: Remove OS-X special-case, and use the correct int type. http://hg.python.org/cpython/rev/a85ac58e9eaf -- ___ Python tracker

[issue17914] add os.cpu_count()

STINNER Victor added the comment: In my patch cpu_count.patch, I changed posix_cpu_count(): * rewrite Mac OS X implementation: code in 5e0c56557390 looks wrong. It gets a MIB but then don't use it when calling _bsd_cpu_count(). But I didn't check my patch nor the commit version on Mac OS X.

[issue17914] add os.cpu_count()

Charles-François Natali added the comment: Alright, committed. Yogesh, thanks for the patch! I'm attaching a patch to replace several occurrences of multiprocessing.cpu_count() by os.cpu_count() in the stdlib/test suite. -- Added file: http://bugs.python.org/file30319/use_cpu_count.diff

[issue17914] add os.cpu_count()

Roundup Robot added the comment: New changeset 5e0c56557390 by Charles-Francois Natali in branch 'default': Issue #17914: Add os.cpu_count(). Patch by Yogesh Chaudhari, based on an http://hg.python.org/cpython/rev/5e0c56557390 -- nosy: +python-dev ___

[issue17914] add os.cpu_count()

Giampaolo Rodola' added the comment: +1 for returning None. I haven't looked into patches but if needed feel free to borrow some code from psutil: Linux: https://code.google.com/p/psutil/source/browse/psutil/_pslinux.py?spec=svn30f3c67322f99ab30ed87205245dc8394f89f0ac&r=c970f35bc9640ac32eb9f09d

[issue17914] add os.cpu_count()

Yogesh Chaudhari added the comment: Typo fix -- Added file: http://bugs.python.org/file30284/issue17914-7.patch ___ Python tracker ___ ___

[issue17914] add os.cpu_count()

Yogesh Chaudhari added the comment: Minor modifications based on review comments. 1. Change mib array size to 2, 2. return value set to 0 consistently (in C code), and 3. removed IRIX #defines -- Added file: http://bugs.python.org/file30282/issue17914-6.patch _

[issue17914] add os.cpu_count()

Charles-François Natali added the comment: > Well, they can be wrong sometimes, too :-) Indeed, as can I ;-) > The patch doesn't seem to rely on the glibc, so we are fine here. > Or do the other libs work likewise? sysconf(_SC_NPROCESSORS_CONF) is implemented with the above function in the gli

[issue17914] add os.cpu_count()

Yogesh Chaudhari added the comment: Based on the last 3 messages by Ned, Charles and Antoine, I keep thinking that arguments made by Charles are very valid ones and that it would be better to return 1. I say this (partly from the 'type' argument, but), mainly, *if* its known that the underlyin

[issue17914] add os.cpu_count()

Antoine Pitrou added the comment: > > Python's goal is not to emulate the suboptimal parts of other languages. > > Well, I'm sure they could have returned -1 or 0, which are valid C > long distinct from any valid integer representing a number of CPUs. If > the libc guys (and many other APIs out

[issue17914] add os.cpu_count()

Charles-François Natali added the comment: > Python's goal is not to emulate the suboptimal parts of other languages. Well, I'm sure they could have returned -1 or 0, which are valid C long distinct from any valid integer representing a number of CPUs. If the libc guys (and many other APIs out t

[issue17914] add os.cpu_count()

Ned Batchelder added the comment: Python's goal is not to emulate the suboptimal parts of other languages. We have dynamic typing, and so can return None from the same function that returns 1. And we have compact expressions like `cpu_count() or 1`, so we don't have to make unfortunate compr

[issue17914] add os.cpu_count()

Antoine Pitrou added the comment: > And on Linux, 1 is returned as a fallback when you don't have the > right /sys or /proc entry: > http://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/unix/sysv/linux/getsysstats.c > > (The enum discussion enlighted me, endless discussions are so fun!) Do y

[issue17914] add os.cpu_count()

Charles-François Natali added the comment: Just for giggles, here's the glibc default implementation on non Linux platforms: http://sourceware.org/git/?p=glibc.git;a=blob;f=misc/getsysstats.c;hb=HEAD """ int __get_nprocs () { /* We don't know how to determine the number. Simply return always 1

[issue17914] add os.cpu_count()

Yogesh Chaudhari added the comment: Modified patch based on further comments and review. 1. Removed *everything* from os.py 2. removed typecasting for ncpu where not required. 3. removed redundant comments -- Added file: http://bugs.python.org/file30246/issue17914-5.patch _

[issue17914] add os.cpu_count()

Yogesh Chaudhari added the comment: @Stinner: 1. While I agree with your idea of what you have done in test_os, (particularly, for determining if platform is supported or not) there seems to be no reason(AFAIK) to have a shutil for cpu_count. I agree with neologox there. 2. Also I am not comf

[issue17914] add os.cpu_count()

Antoine Pitrou added the comment: > * add a os.cpu_count() which returns the raw value (may be zero or > negative if the OS returns a dummy value) and raise an OSError on > error > * os.cpu_count() is not available on all platforms > * shutil.cpu_count() is the high level API using 1 as a fall

[issue17914] add os.cpu_count()

Charles-François Natali added the comment: > Here is a new patch (cpu_count.patch) with a different approach: > > * add a os.cpu_count() which returns the raw value (may be zero or negative > if the OS returns a dummy value) and raise an OSError on error > * os.cpu_count() is not available on

[issue17914] add os.cpu_count()

STINNER Victor added the comment: The return type of the C function sysconf() is long, but Python uses int: I opened the issue #17964 to track this bug. (os.cpu_count() uses sysconf()). -- ___ Python tracker _

[issue17914] add os.cpu_count()

STINNER Victor added the comment: Here is a new patch (cpu_count.patch) with a different approach: * add a os.cpu_count() which returns the raw value (may be zero or negative if the OS returns a dummy value) and raise an OSError on error * os.cpu_count() is not available on all platforms * s

[issue17914] add os.cpu_count()

Yogesh Chaudhari added the comment: Modified patch based on review by neologix -- Added file: http://bugs.python.org/file30236/issue17914-4.patch ___ Python tracker ___ _

[issue17914] add os.cpu_count()

Yogesh Chaudhari added the comment: @STINNER: I don't understand. Where exactly should the patch handle this? -- ___ Python tracker ___ __

[issue17914] add os.cpu_count()

Changes by Stefan Drees : -- nosy: +dilettant ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pytho

[issue17914] add os.cpu_count()

STINNER Victor added the comment: Not being able to decide for the default value is not a problem: just an optional "default" argument, which is 1 by default (most convinient value), return default on error. os.get_terminal_size() has a similar API for example. -- __

[issue17914] add os.cpu_count()

Yogesh Chaudhari added the comment: @Ned: if cpus is None, then this will raise an exception in Python 3: `cpus >= 1 or cpus == None` I understand that cpus >= INTEGER will raise an exception and have already modified the condition to remove that kind of check. I was merely stating that equa

[issue17914] add os.cpu_count()

Ned Batchelder added the comment: @Yogesh: if cpus is None, then this will raise an exception in Python 3: `cpus >= 1 or cpus == None` Perhaps you don't have enough test cases yet. -- ___ Python tracker

[issue17914] add os.cpu_count()

Yogesh Chaudhari added the comment: Modified patch to return None from C code -- Added file: http://bugs.python.org/file30226/issue17914-3.patch ___ Python tracker ___ __

[issue17914] add os.cpu_count()

Charles-François Natali added the comment: > Returning None from C code sounds reasonable to me. Anyone else wants to > pitch in with suggestions for/against this? Go for it ;-) -- ___ Python tracker

[issue17914] add os.cpu_count()

Yogesh Chaudhari added the comment: Returning None from C code sounds reasonable to me. Anyone else wants to pitch in with suggestions for/against this? -- ___ Python tracker __

[issue17914] add os.cpu_count()

Serhiy Storchaka added the comment: Now we have three cpu_count() functions: multiprocessing.cpu_count() raises an exception on failure, posix.cpu_count() returns -1, and os.cpu_count() returns None. It will be easy to get rid of Python wrapper in the os module and return None directly from C

[issue17914] add os.cpu_count()

Yogesh Chaudhari added the comment: @Serhiy Sorry, I missed your comments in the thread. I have made 2 changes and ignored the cpu_count() returning 0, because it returns -1 on failure, else give the number of CPUs. Also the test_os, checks for 0 return if that was to e the case. -- Ad

[issue17914] add os.cpu_count()

Serhiy Storchaka added the comment: Yogesh, didn't you notice comments on Rietveld? -- ___ Python tracker ___ ___ Python-bugs-list mai

[issue17914] add os.cpu_count()

Yogesh Chaudhari added the comment: Appreciate everyone's feedback. I have modified the patch based on further messages in the thread. @Neologix modified posixmodule according to one in Trent's branch and used that for cpu_count(). Kindly suggest improvements/changes if any. @Ned: Thanks for

[issue17914] add os.cpu_count()

Serhiy Storchaka added the comment: > Since the user can't do anything except falling back to 1, > os.cpu_count() should always return a positive number (1 by default). In general I agree with you. Actually the os module should contains two functions: cpu_count() which fallbacks to 1 and is_cpu

[issue17914] add os.cpu_count()

Antoine Pitrou added the comment: > And I maintain it's an ugly idiom ;-) > Since the user can't do anything except falling back to 1, > os.cpu_count() should always return a positive number (1 by default). The user can also raise an error. For example, if I'm writing a benchmark to measure per-

[issue17914] add os.cpu_count()

Charles-François Natali added the comment: > I think the idiom `os.cpu_count() or 1` should be mentioned in the > documentation an officially recommended. Otherwise people will produce a > non-portable code which works on their developer's computers but not on > exotic platforms. And I mainta

[issue17914] add os.cpu_count()

Antoine Pitrou added the comment: I agree with Charles-François. An approach using C library functions is far superior to launching external commands. -- ___ Python tracker ___

[issue17914] add os.cpu_count()

Serhiy Storchaka added the comment: I think the idiom `os.cpu_count() or 1` should be mentioned in the documentation an officially recommended. Otherwise people will produce a non-portable code which works on their developer's computers but not on exotic platforms. I have added some other com

[issue17914] add os.cpu_count()

Ned Batchelder added the comment: A few small points: Use `num is None` instead of `num == None`. Use `isinstance(cpus, int)` rather than `type(cpus) is int`. And this I think will throw an exception in Python 3: `cpus >= 1 or cpus == None`, because you can't compare None to 1. -- _

[issue17914] add os.cpu_count()

Charles-François Natali added the comment: > Based on the conversation and the particular inputs to the thread form > neologix and ezio, I would like to submit this patch. > > It probably needs modification(s) as I am not sure what to do with the > implementation that is already present in mult

[issue17914] add os.cpu_count()

Yogesh Chaudhari added the comment: Based on the conversation and the particular inputs to the thread form neologix and ezio, I would like to submit this patch. It probably needs modification(s) as I am not sure what to do with the implementation that is already present in multiprocessing. Th

[issue17914] add os.cpu_count()

R. David Murray added the comment: As for why to not return 1, I can imagine code that checks cpu_count, and only if it returns the "don't know" result would it invoke some more expensive method of determining the CPU count on platforms that cpu_count doesn't support. Since the os module is t

[issue17914] add os.cpu_count()

Ezio Melotti added the comment: Returning None sounds reasonable to me. Raising an exception pretty much means that the function should always be called in a try/except (unless you are sure that the code is running on an OS that knows the number of CPUs). Returning -1 is not very Pythonic, and

[issue17914] add os.cpu_count()

Charles-François Natali added the comment: Fair enough, I guess we should use it then. We just have to agree on the value to return when the number of CPUs can't be determined ;-) -- ___ Python tracker ___

[issue17914] add os.cpu_count()

STINNER Victor added the comment: > I don't see exactly what this C implementation brings over the one > in multiprocessing (which is written in Python)? multiprocessing.cpu_count() creates a subprocess on BSD and Darwin to get the number of CPU. Calling sysctl() or sysctlnametomib() should be

[issue17914] add os.cpu_count()

Charles-François Natali added the comment: > I also vote +1 for returning None when the information is unknown. I still don't like it. If a function returns a number of CPU, it should either return an integer >= 1, or raise an exception. None is *not* an integer. And returning an exception is I

[issue17914] add os.cpu_count()

STINNER Victor added the comment: See also #17444, Trent Nelson wrote an implementation of os.cpu_count(). -- nosy: +trent ___ Python tracker ___

[issue17914] add os.cpu_count()

STINNER Victor added the comment: I also vote +1 for returning None when the information is unknown. Just write "os.cpu_count() or 1" if you need 1 when the count is unknown ;-) -- nosy: +haypo ___ Python tracker

[issue17914] add os.cpu_count()

Antoine Pitrou added the comment: Returning 0 or None sounds better to me than 1 or -1. (I have a preference for None) -- nosy: +pitrou ___ Python tracker ___ ___

[issue17914] add os.cpu_count()

Ned Batchelder added the comment: Seriously, return zero, and I can use it as: cpu_count = os.cpu_count() or 1 Why throw away information? -- ___ Python tracker ___

[issue17914] add os.cpu_count()

Charles-François Natali added the comment: > I am interested to submit a patch on this. Should I move the implementation > to os module and made the multiprocessing one as an alias ? or keep it in > both places ? Yes, you should move it, add a corresponding documentation to Doc/modules/os.rst

[issue17914] add os.cpu_count()

Kushal Das added the comment: I am interested to submit a patch on this. Should I move the implementation to os module and made the multiprocessing one as an alias ? or keep it in both places ? I prefer the idea of returning -1 instead of the current way of raising NotImplementedError in case

[issue17914] add os.cpu_count()

Ned Batchelder added the comment: If you can't determine the number of CPUs, return a clear "can't determine" value, such as 0 or -1. Returning 1 will hide information, and it's an easy default for the caller to apply if they want to. -- nosy: +nedbat

[issue17914] add os.cpu_count()

Charles-François Natali added the comment: Note that I think it might be interesting to return 1 if the actual value cannot be determined, since that's a sensible default, and likely what the caller will do anyway. This contrasts with the current multiprocessing's implementation which raised N

[issue17914] add os.cpu_count()

New submission from Charles-François Natali: multiprocessing.cpu_count() implementation should be made available in the os module, where it belongs. -- keywords: easy messages: 188506 nosy: neologix priority: normal severity: normal stage: needs patch status: open type: enhancement vers