[issue40280] Consider supporting emscripten/webassembly as a build target

[Christian Heimes]

Change by Christian Heimes :


--
pull_requests: +28025
pull_request: https://github.com/python/cpython/pull/29789

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Ruben Vorderman]

New submission from Ruben Vorderman :

Python now uses the excellent timsort for most (all?) of its sorting. But this 
is not the fastest sort available for one particular use case.

If the number of possible values in the array is limited, it is possible to 
perform a counting sort: https://en.wikipedia.org/wiki/Counting_sort. In a 
counting sort each value maps to an integer corresponding to its relative 
value. The values are then counted by using key = map_to_key(value); 
count_array[key]. Then from this count_array a new array of sorted values can 
be constructed. (See the wikipedia article for a more detailed explanation).

For the python bytes and bytesarray types this is extremely simple to 
implement. All 256 possible values are can be directly used as keys. 

Rough implementation:
- Use buffer protocol to get pointer to bytes/bytesarray internal c-string
- Declare a count_array: Py_ssize_t[256] count_array . (use memset to set it to 
0)
- Iterate over the c-string and add each value to the countarray. 
count_array[buffer[i]] += 1
- Allocate a new bytes(array) object, or in the case of bytesarray the sorting 
can be performed inplace when bytesarray.sort() is used. 
- Iterate over the count_array. Get the number of values for each key and use 
memset to insert the sequences of keys into position.


The most obvious way to implement this algorithm will be as bytesarray.sort() 
where it is sorted inplace and as bytes.sort() which returns a new sorted bytes 
object. This is much much faster than using bytes(sorted(bytes)).

I made a quick cython implementation for speed testing here: 
https://github.com/rhpvorderman/bytes_sort/blob/main/bytes_sort.pyx

Currently to get a sorted bytestring one has to do bytes(sorted(my_bytes)). 

Test results:

# First make sure there is no regression when sorting an empty string
$ python -m timeit -c "from bytes_sort import bytes_sort" "bytes(sorted(b''))"
50 loops, best of 5: 560 nsec per loop
$ python -m timeit -c "from bytes_sort import bytes_sort" "bytes_sort(b'')"
50 loops, best of 5: 565 nsec per loop

# Test result for very small strings
$ python -m timeit -c "from bytes_sort import bytes_sort" 
"bytes(sorted(b'abc'))"
50 loops, best of 5: 628 nsec per loop
$ python -m timeit -c "from bytes_sort import bytes_sort" "bytes_sort(b'abc')"
50 loops, best of 5: 578 nsec per loop

# Even on a very small already sorted string, a counting sort is faster.

# Test with a proper string
$ python -m timeit -c "from bytes_sort import bytes_sort" 
"bytes(sorted(b'Let\'s test a proper string now. One that has some value to be 
sorted.'))"
10 loops, best of 5: 2.32 usec per loop
$ python -m timeit -c "from bytes_sort import bytes_sort" "bytes_sort(b'Let\'s 
test a proper string now. One that has some value to be sorted.')"
50 loops, best of 5: 674 nsec per loop

More than three times faster!

--
components: C API
messages: 407032
nosy: rhpvorderman
priority: normal
severity: normal
status: open
title: Bytes and bytesarrays can be sorted with a much faster count sort.
type: performance
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Ruben Vorderman]

Ruben Vorderman  added the comment:

Also I didn't know if this should be in Component C-API or Interpreter Core. 
But I guess this will be implemented as C-API calls PyBytes_Sort and 
PyByteArray_SortInplace so I figured C-API is the correct component here.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40280] Consider supporting emscripten/webassembly as a build target

[Christian Heimes]

Change by Christian Heimes :


--
dependencies: +Cross compiling on Linux is untested, undocumented, and broken, 
Fix Program/_freeze_module for cross compiling Python, Undefinied 
_Py_Sigset_Converter function when HAVE_SIGSET_T not set, ctypes cfield.c 
defines duplicate ffi_type_* symbols, update config.guess and config.sub

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45898] ctypes cfield.c defines duplicate ffi_type_* symbols

[Christian Heimes]

Change by Christian Heimes :


--
keywords: +patch
pull_requests: +28026
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/29791

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Alex Waygood]

Change by Alex Waygood :


--
nosy: +Mark.Shannon, brett.cannon, rhettinger, serhiy.storchaka, tim.peters, 
vstinner, yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Ruben Vorderman]

Ruben Vorderman  added the comment:

I changed the cython script a bit to use a more naive implementation without 
memset.
Now it is always significantly faster than bytes(sorted(my_bytes)).

$ python -m timeit -c "from bytes_sort import bytes_sort" "bytes_sort(b'')"
50 loops, best of 5: 495 nsec per loop
$ python -m timeit -c "from bytes_sort import bytes_sort" "bytes_sort(b'abc')"
50 loops, best of 5: 519 nsec per loop
$ python -m timeit -c "from bytes_sort import bytes_sort" "bytes_sort(b'Let\'s 
test a proper string now. One that has some value to be sorted.')"
50 loops, best of 5: 594 nsec per loop

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39026] Include/cpython/pystate.h contains non-relative of initconfig.h include causing macOS Framework include failure

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 92631a4144fba041a5fb6578620db784821dfdc5 by Victor Stinner in 
branch '3.9':
bpo-39026: Fix Python.h when building with Xcode (GH-29488) (GH-29776)
https://github.com/python/cpython/commit/92631a4144fba041a5fb6578620db784821dfdc5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39026] Include/cpython/pystate.h contains non-relative of initconfig.h include causing macOS Framework include failure

[STINNER Victor]

STINNER Victor  added the comment:

Thanks Gaige Paulsen for the bug report, it's now fixed.

> I think GH-28612 broke Windows builds:

It was related to C++ build, it's now fixed by bpo-45893.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.10, Python 3.11, Python 3.9 -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45876] Improve accuracy of stdev functions in statistics

[Mark Dickinson]

Mark Dickinson  added the comment:

Related: 
https://stackoverflow.com/questions/32150888/should-ldexp-round-correctly

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45898] ctypes cfield.c defines duplicate ffi_type_* symbols

[Christian Heimes]

Christian Heimes  added the comment:

Windows tests are failing without ffi_type_ symbols:

cfield.obj : error LNK2001: unresolved external symbol _ffi_type_double 
[D:\a\cpython\cpython\PCbuild\_ctypes.vcxproj]
cfield.obj : error LNK2001: unresolved external symbol _ffi_type_float 
[D:\a\cpython\cpython\PCbuild\_ctypes.vcxproj]
cfield.obj : error LNK2001: unresolved external symbol _ffi_type_uint32 
[D:\a\cpython\cpython\PCbuild\_ctypes.vcxproj]
cfield.obj : error LNK2001: unresolved external symbol _ffi_type_sint8 
[D:\a\cpython\cpython\PCbuild\_ctypes.vcxproj]
cfield.obj : error LNK2001: unresolved external symbol _ffi_type_uint8 
[D:\a\cpython\cpython\PCbuild\_ctypes.vcxproj]
cfield.obj : error LNK2001: unresolved external symbol _ffi_type_uint64 
[D:\a\cpython\cpython\PCbuild\_ctypes.vcxproj]
cfield.obj : error LNK2001: unresolved external symbol _ffi_type_uint16 
[D:\a\cpython\cpython\PCbuild\_ctypes.vcxproj]
cfield.obj : error LNK2001: unresolved external symbol _ffi_type_sint16 
[D:\a\cpython\cpython\PCbuild\_ctypes.vcxproj]

--
nosy: +paul.moore, steve.dower, tim.golden, zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45884] datetime.strptime incorrectly handling hours and minutes with bad format string

[Christian Heimes]

Change by Christian Heimes :


--
superseder:  -> strptime %d handling of single digit day of month

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45866] Out of tree build of Python 3.11.0a2+ breaks regen-frozen

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 253b7a0a9fef1d72a4cb87b837885576e68e917c by Victor Stinner in 
branch 'main':
bpo-45866: pegen strips directory of "generated from" header (GH-29777)
https://github.com/python/cpython/commit/253b7a0a9fef1d72a4cb87b837885576e68e917c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Ruben Vorderman]

Ruben Vorderman  added the comment:

Sorry for the spam. I see I made a typo in the timeit script. Next time I will 
be more dilligent when making these kinds of reports and triple checking it 
before hand, and sending it once. I used -c instead of -s and now all the setup 
time is also included. This confounds the results.

The proper test commands should be:

python -m timeit -s "from bytes_sort import bytes_sort, bytearray_sort_inplace" 
"bytes_sort(b'My string here')"

python -m timeit "bytes(sorted(b'My string here'))"

Using just sorted, to purely compare the sorting algorithms without the 
overhead of creating a new bytes object.
python -m timeit "sorted(b'My string here')"

Correct comparison results
# String = b''
using bytes(sorted: 188 nsec
using sorted:   108 nsec
using byte_sort:125 nsec  # Some overhead here, setting up the countarray
# String = b'abc'
using bytes(sorted: 252 nsec
using sorted:   145 nsec
using byte_sort:136 nsec  # Overhead compared to sorted already negated 
when sorting 3 items(!)
# String = b'Let\'s test a proper string now. One that has some value to be 
sorted.'
using bytes(sorted: 1830 nsec (reported as 1.83 usec)
using sorted:   1550 nsec (reported as 1.55 usec)
using byte_sort: 220 nsec

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45866] Out of tree build of Python 3.11.0a2+ breaks regen-frozen

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +28027
pull_request: https://github.com/python/cpython/pull/29792

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Mark Dickinson]

Mark Dickinson  added the comment:

Can you give example use-cases for sorting a bytes or bytearray object? I see 
value in the intermediate object - the frequency table, but the reconstructed 
sorted bytes object just seems like an inefficient representation of the 
frequency table, and I'm not sure how it would be useful.

As the wikipedia page for counting sort says, the real value is in sorting 
items by keys that are small integers, and the special case where the item is 
identical to the key isn't all that useful:

> In some descriptions of counting sort, the input to be sorted is assumed to 
> be more simply a sequence of integers itself, but this simplification does 
> not accommodate many applications of counting sort.

--
nosy: +mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Mark Dickinson]

Mark Dickinson  added the comment:

(Changing the issue type: as I understand it, this is a proposal for a new 
feature, namely new methods bytes.sort and bytearray.sort, rather than a 
performance improvement to an existing feature.)

--
type: performance -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Ruben Vorderman]

Ruben Vorderman  added the comment:

I used it for the median calculation of FASTQ quality scores 
(https://en.wikipedia.org/wiki/FASTQ_format). But in the end I used the 
frequency table to calculate the median more quickly. So as you say, the 
frequency table turned out to be more useful.

Having said that the usefulness depends on how many times 8-bit data is passed 
into sorted. (bytes,bytearrays, most python strings are 8-bit I believe). I 
raised this issue not because I want a .sort() method on bytes or 
bytearrays, but mostly because I think python's sorted function can be improved 
with regards to 8-bit data. I think it is an interesting thing to consider, 
depending on how often this occurs.

For example:
sorted(b'Let\'s test a proper string now. One that has some value to be 
sorted.')
and 
list(bytes_sort(b'Let\'s test a proper string now. One that has some value to 
be sorted.'))

This returns the same result (a list of integers). But the byte_sort 
implementation is 2.5 times faster. So sorted is not optimally implemented here.

Since sorted is now basically throwing everything into list.sort an alternative 
codepath using bytes.sort can be considered. (If there are enough use cases for 
it).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40280] Consider supporting emscripten/webassembly as a build target

[Roman Yurchak]

Change by Roman Yurchak :


--
nosy: +Roman Yurchak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27580] CSV Null Byte Error

[Petr Viktorin]

Petr Viktorin  added the comment:

The NUL check was around for a long time, available to be used (XKCD-1172 
style) as a simple check against reading binary files. The change did break 
tests of distutils.
Maybe it deserves a What's New entry?

--
nosy: +petr.viktorin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27580] CSV Null Byte Error

[Petr Viktorin]

Petr Viktorin  added the comment:

*docutils, not distutils

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40280] Consider supporting emscripten/webassembly as a build target

[Roman Yurchak]

Roman Yurchak  added the comment:

Thanks a lot for working on this!

> _sys_shutdown is the syscall for shutdown(2) used by the socket module.

Yes, the issue with Emscripten is that a number of system calls are either not 
implemented or implemented but not tested. See a list we are using in 
https://github.com/pyodide/pyodide/blob/main/cpython/pyconfig.undefs.h (though 
things might have improved since it was created).

FYI, with Emscripten, the list of CPython unit tests that are currently skipped 
(as of Python 3.9.5) is in 
https://github.com/pyodide/pyodide/blob/main/src/tests/python_tests.txt some of 
those are due to browser VM limitations (e.g. virtual filestem by Emscripten 
that's not fully POSIX compliant, no processes, no sockets, async only via the 
browser event loop etc), others because we are not yet using threading since 
not all browsers support it, and some failures probably need more 
investigation. 

Also opened https://github.com/pyodide/pyodide/issues/2000 . Let us know if 
there is anything we can do help with this effort.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40280] Consider supporting emscripten/webassembly as a build target

[Christian Heimes]

Change by Christian Heimes :


--
pull_requests: +28029
pull_request: https://github.com/python/cpython/pull/29794

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45886] Fix Program/_freeze_module for cross compiling Python

[Christian Heimes]

Change by Christian Heimes :


--
pull_requests: +28028
pull_request: https://github.com/python/cpython/pull/29793

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40280] Consider supporting emscripten/webassembly as a build target

[Christian Heimes]

Change by Christian Heimes :


--
pull_requests: +28030
pull_request: https://github.com/python/cpython/pull/29795

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45795] urllib http client vulnerable to DOS attack

[Muhammad Farhan]

Muhammad Farhan  added the comment:

Hi,
Hope all of you are doing good. Looks like you guys are not interested in this 
issue. Can you please provide me the source code for yhe urllib, I will fix it 
myself

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40280] Consider supporting emscripten/webassembly as a build target

[Christian Heimes]

Christian Heimes  added the comment:


New changeset d224e769b83a6a027bec8e21ee62db2b96d5af8e by Christian Heimes in 
branch 'main':
bpo-40280: clean and ignore .wasm files (GH-29794)
https://github.com/python/cpython/commit/d224e769b83a6a027bec8e21ee62db2b96d5af8e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45898] ctypes cfield.c defines duplicate ffi_type_* symbols

[Roman Yurchak]

Change by Roman Yurchak :


--
nosy: +Roman Yurchak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40280] Consider supporting emscripten/webassembly as a build target

[Christian Heimes]

Christian Heimes  added the comment:

Thanks Roman, I replied on the pyodide issue tracker.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45760] Remove "PyNumber_InMatrixMultiply"

[Dong-hee Na]

Dong-hee Na  added the comment:


New changeset db671b010cb4662011c6e6567a1f4fcd509102b8 by Dong-hee Na in branch 
'main':
bpo-45760: Remove PyNumber_InMatrixMultiply (GH-29751)
https://github.com/python/cpython/commit/db671b010cb4662011c6e6567a1f4fcd509102b8


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45760] Remove "PyNumber_InMatrixMultiply"

[Dong-hee Na]

Change by Dong-hee Na :


--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45884] datetime.strptime incorrectly handling hours and minutes with bad format string

[Jon Oxtoby]

Jon Oxtoby  added the comment:

I was indeed overlooking the note in the documentation that the leading zero is 
optional for some formatters when using strptime. Closing.

--
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40280] Consider supporting emscripten/webassembly as a build target

[Dong-hee Na]

Change by Dong-hee Na :


--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40280] Consider supporting emscripten/webassembly as a build target

[Christian Heimes]

Christian Heimes  added the comment:


New changeset 8caceb7a474bf32cddfd25fba25b531ff65f4365 by Christian Heimes in 
branch 'main':
bpo-40280: Add configure check for socket shutdown (GH-29795)
https://github.com/python/cpython/commit/8caceb7a474bf32cddfd25fba25b531ff65f4365


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45558] shutil.copytree: Give the option to disable copystat

[Jason R. Coombs]

New submission from Jason R. Coombs :

Can you provide a description of what motivated you to change the behavior or 
what benefits this change would have for you or others? Do you know why others 
haven’t reported this need previously?

--
nosy: +jaraco

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13475] Add '--mainpath'/'--nomainpath' command line options to override sys.path[0] initialisation

[Neil Isaac]

Change by Neil Isaac :


--
nosy: +nisaac

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[miss-islington]

Change by miss-islington :


--
pull_requests: +28031
pull_request: https://github.com/python/cpython/pull/29796

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45903] What’s New In Python 3.11: wrong reference to Signature.from_callable

[Jakub Wilk]

New submission from Jakub Wilk :

 says:

"Removed from the inspect module: […] the undocumented Signature.from_callable 
and Signature.from_function functions, deprecated since Python 3.5; use the 
Signature.from_callable() method instead."

But Signature.from_callable can't possibly be removed and the suggested 
replacement at the same time.

I think it should say: "… the undocumented Signature.from_builtin and …"

--
assignee: docs@python
components: Documentation
messages: 407054
nosy: docs@python, hugovk, jwilk
priority: normal
severity: normal
status: open
title: What’s New In Python 3.11: wrong reference to Signature.from_callable
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44353] PEP 604 NewType

[miss-islington]

miss-islington  added the comment:


New changeset 3f024e27c29a57dd4f805aa2431d713ed0fe57b2 by Miss Islington (bot) 
in branch '3.10':
bpo-44353: Correct docstring for `NewType` (GH-29785)
https://github.com/python/cpython/commit/3f024e27c29a57dd4f805aa2431d713ed0fe57b2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43137] webbrowser to support "gio open "

[Christian Heimes]

Christian Heimes  added the comment:

gio is not available on older distros. For example I could not find it in 
Xenial with libglib2.0-bin 2.48.2-0ubuntu4.8. I suggest that you keep the 
fallbacks or seek agreement that we no longer support old distros without gio.

--
keywords: +3.11regression -patch
nosy: +christian.heimes, pablogsal
priority: normal -> release blocker
type:  -> enhancement
versions: +Python 3.11 -Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45866] Out of tree build of Python 3.11.0a2+ breaks regen-frozen

[STINNER Victor]

STINNER Victor  added the comment:


New changeset b6defde2afe656db830d6fedf74ca5f6225f5928 by Victor Stinner in 
branch '3.10':
bpo-45866: pegen strips directory of "generated from" header (GH-29777) 
(GH-29792)
https://github.com/python/cpython/commit/b6defde2afe656db830d6fedf74ca5f6225f5928


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue9436] test_sysconfig failure: build a 32-bit Python a 64-bit OS

[Irit Katriel]

Irit Katriel  added the comment:

This test is disabled when platform == "win32". Does that cover this case or 
does it need to be disabled for platform == "win64" as well?


commit db902ac0b4df295bd90109852c1abd05da831b81
Author: Brian Curtin 
Date:   Thu Jul 22 15:38:28 2010 +

Skip this test as it doesn't apply to Windows. It was added for
#9189 for some GCC flags.

diff --git a/Lib/test/test_sysconfig.py b/Lib/test/test_sysconfig.py
index 428379c3c3..053859238d 100644
--- a/Lib/test/test_sysconfig.py
+++ b/Lib/test/test_sysconfig.py
@@ -277,6 +277,7 @@ def test_main(self):
 _main()
 self.assertTrue(len(output.getvalue().split('\n')) > 0)
 
+@unittest.skipIf(sys.platform == "win32", "Does not apply to Windows")
 def test_ldshared_value(self):
 ldflags = sysconfig.get_config_var('LDFLAGS')
 ldshared = sysconfig.get_config_var('LDSHARED')

--
nosy: +iritkatriel
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Mark Dickinson]

Mark Dickinson  added the comment:

> If there are enough use cases for it.

Well, that's the question. :-) *I* can't think of any good use cases, but maybe 
others can. But if we can't come up with some use cases, then this feels like a 
solution looking for a problem, and that makes it hard to justify both the 
short-term effort and the longer-term maintenance costs of adding the 
complexity.

FWIW, given a need to efficiently compute frequency tables for reasonably long 
byte data, I'd probably reach first for NumPy (and numpy.bincount in 
particular):

Python 3.10.0 (default, Nov 12 2021, 12:32:57) [Clang 12.0.5 
(clang-1205.0.22.11)]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.28.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import collections, numpy as np

In [2]: t = 
b'MDIAIHHPWIRRPFFPFHSPSRLFDQFFGEHLLESDLFSTATSLSPFYLRPPSFLRAPSWIDTGLSEMRLEKDRFSVNLDVKHFSPEELKVKVLGDVIEVHGKHEERQDEHGFISREFHRKYRI
   ...: PADVDPLAITSSLSSDGVLTVNGPRKQVSGPERTIPITREEKPAVAAAPKK';  t *= 100

In [3]: %timeit np.bincount(np.frombuffer(t, np.uint8))
32.7 µs ± 3.15 µs per loop (mean ± std. dev. of 7 runs, 1 loops each)

In [4]: %timeit collections.Counter(t)
702 µs ± 25.8 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

In [5]: %timeit sorted(t)
896 µs ± 64.2 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45876] Improve accuracy of stdev functions in statistics

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Instead of calling float(), perhaps do an int/int division to match the other 
code path so that the function depends on only one mechanism for building the 
float result.

-return float(_isqrt_frac_rto(n, m << 2 * q) << q)
+(_isqrt_frac_rto(n, m << 2 * q) << q) / 1

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45866] Out of tree build of Python 3.11.0a2+ breaks regen-frozen

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +28032
pull_request: https://github.com/python/cpython/pull/29797

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45795] urllib http client vulnerable to DOS attack

[Senthil Kumaran]

Senthil Kumaran  added the comment:

Hi Muhammad, 

I haven't gotten to this. urllib doesn't maintain a client state during 
multiple request / response.

The code is available here 
https://github.com/python/cpython/tree/main/Lib/urllib

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45886] Fix Program/_freeze_module for cross compiling Python

[Christian Heimes]

Christian Heimes  added the comment:


New changeset 765b2a3ad2e8abf9a06d5e9b3802b575ec115d76 by Christian Heimes in 
branch 'main':
bpo-45886: Fix OOT build when srcdir has frozen module headers (GH-29793)
https://github.com/python/cpython/commit/765b2a3ad2e8abf9a06d5e9b3802b575ec115d76


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45876] Improve accuracy of stdev functions in statistics

[Mark Dickinson]

Mark Dickinson  added the comment:

[Raymond]
> [...] perhaps do an int/int division to match the other code path [...]

Sure, works for me.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45903] What’s New In Python 3.11: wrong reference to Signature.from_callable

[Alex Waygood]

Change by Alex Waygood :


--
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45866] Out of tree build of Python 3.11.0a2+ breaks regen-frozen

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 93a540d74c654819ad04d72fcdcf827d0e25 by Victor Stinner in 
branch '3.9':
bpo-45866: pegen strips directory of "generated from" header (GH-29777) 
(GH-29792) (GH-29797)
https://github.com/python/cpython/commit/93a540d74c654819ad04d72fcdcf827d0e25


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45904] Pasting the U00FF character into Python REPL misinterprets character

[George King]

New submission from George King :

Using macOS 11.6 Terminal.app with Python 3.10.0 installed directly from 
python.org.

I open the REPL. If I enter `char(0xff)` I get back 'ÿ' as expected (U00FF 
LATIN SMALL LETTER Y WITH DIAERESIS).

However, If I copy this character with surrounding quotes, and then paste it 
into the REPL, it pastes as '' and evaluates to the empty string.

If I copy it without quotes and then paste into the REPL, I see nothing. When I 
hit return, the prompt renders as `>>> ^M>>>`. This suggests that the character 
is getting misinterpreted as a control character or something.

If I paste it into the terminal shell when the Python REPL is not running, it 
appears as the latin1 letter that I expect.

If I run `python3 -c 'print("ÿ")'` the character prints fine.

It seems to me that the python REPL is setting some terminal mode that fails on 
this particular character. Perhaps this is a problem with the macOS 
readline/libedit implementation?

It seems that only U00FF is problematic; U00FE and U01000 both paste in just 
fine.

I verified that my terminal profile is set to UTF-8 encoding. I also repeated 
this experiment in the Kitty terminal emulator, and got identical results.


Here is the readline version:
>>> readline._READLINE_LIBRARY_VERSION
'EditLine wrapper'
>>> readline._READLINE_RUNTIME_VERSION
1026
>>> readline._READLINE_VERSION
1026

--
messages: 407065
nosy: gwk
priority: normal
severity: normal
status: open
title: Pasting the U00FF character into Python REPL misinterprets character
type: behavior
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45904] Pasting the U00FF character into Python REPL misinterprets character

[George King]

George King  added the comment:

Edit: `chr(0xff)`

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45866] Out of tree build of Python 3.11.0a2+ breaks regen-frozen

[STINNER Victor]

STINNER Victor  added the comment:

It seems like all bugs reported in this issue has been fixed, so I close the 
issue. Thanks for the bug report ;-)

> -# @generated by pegen from ./Tools/peg_generator/pegen/metagrammar.gram
> +# @generated by pegen from ../../Tools/peg_generator/pegen/metagrammar.gram

This issue is now fixed in 3.9, 3.10 and main branches.


> ERROR: missing _freeze_module

I'm no longer able to reproduce this error on an up-to-date main branch. The 
issue has been fixed by this change:

commit 5be98e57b3c3b36d1a1176b49c73b8822c6380e7
Author: Guido van Rossum 
Date:   Tue Nov 23 08:56:06 2021 -0800

bpo-45873: Get rid of bootstrap_python (#29717)

Instead we use $(PYTHON_FOR_REGEN) .../deepfreeze.py with the
frozen .h file as input, as we did for Windows in bpo-45850.

We also get rid of the code that generates the .h files
when make regen-frozen is run (i.e., .../make_frozen.py),
and the MANIFEST file.

Restore Python 3.8 and 3.9 as Windows host Python again

Co-authored-by: Kumar Aditya 
<59607654+kumaraditya...@users.noreply.github.com>

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4322] function with modified __name__ uses original name when there's an arg error

[Irit Katriel]

Irit Katriel  added the comment:

Reproduced on 3.11: 

>>> def foo(): pass
... 
>>> foo.__name__ = 'bar'
>>> foo(1)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: foo() takes 0 positional arguments but 1 was given

--
nosy: +iritkatriel
versions: +Python 3.11 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45905] Provide a C API for introspectable frames for Cython and similar tools

[Mark Shannon]

New submission from Mark Shannon :

See https://github.com/cython/cython/issues/4484

--
messages: 407069
nosy: Mark.Shannon
priority: normal
severity: normal
status: open
title: Provide a C API for introspectable frames for Cython and similar tools

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45905] Provide a C API for introspectable frames for Cython and similar tools

[Mark Shannon]

Change by Mark Shannon :


--
assignee:  -> Mark.Shannon
type:  -> enhancement
versions: +Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22684] message.as_bytes() produces recursion depth exceeded

[Irit Katriel]

Irit Katriel  added the comment:

I am unable to reproduce this on 3.11.

--
nosy: +iritkatriel
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45019] Freezing modules has manual steps but could be automated.

[Guido van Rossum]

Guido van Rossum  added the comment:


New changeset b0b10e146b1cbf9c5dfa44af116a2eeb0f210e8b by Kumar Aditya in 
branch 'main':
bpo-45019: Cleanup module freezing and deepfreeze (#29772)
https://github.com/python/cpython/commit/b0b10e146b1cbf9c5dfa44af116a2eeb0f210e8b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Eric V. Smith]

Eric V. Smith  added the comment:

Given that the normal sort() machinery wouldn't use this code, I don't think 
there's any advantage to adding .sort() methods to bytes and bytesarray. The 
downside to adding these methods is the increased complexity in the stdlib.

I think the better approach is to put bytes_sort on PyPI.

--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32936] RobotFileParser.parse() should raise an exception when the robots.txt file is invalid

[Irit Katriel]

Irit Katriel  added the comment:

Please reopen this or create a new issue if this is still a problem and you can 
provide the missing information.

--
resolution:  -> rejected
stage:  -> resolved
status: pending -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43137] webbrowser to support "gio open "

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

The next 3.10 and 3.11 releases are very soon so unfortunately if this is not 
fixed by then I will need to revert PR29154

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45866] Out of tree build of Python 3.11.0a2+ breaks regen-frozen

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +28033
pull_request: https://github.com/python/cpython/pull/29798

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[STINNER Victor]

STINNER Victor  added the comment:

There are other byte-like objects like memoryview or array.array (for some 
array types). I would suggest writing a function rather than a method.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45902] Bytes and bytesarrays can be sorted with a much faster count sort.

[Brett Cannon]

Change by Brett Cannon :


--
nosy:  -brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44452] Allow paths to be joined without worrying about a leading slash

[lutecki]

lutecki  added the comment:

So how this should work? 
I'm testing this simple example on Windows:

a = Path("/a/b")
b = Path("c/d")

and b / a gives me WindowsPath('/a/b'). So I'm like "ok, a seems like absolute, 
I will test for that" but on Windows a.is_absolute() is False.
???

Regards

--
nosy: +lutecki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45847] Port module setup to PY_STDLIB_MOD() macro and addext()

[Erlend E. Aasland]

Erlend E. Aasland  added the comment:

Some info regarding _multiprocessing:

Modules/_multiprocessing/semaphore.c is currently conditionally included in 
setup.py (HAVE_SEM_OPEN && !POSIX_SEMAPHORES_NOT_ENABLED), but always included 
in Modules/Setup.

Here's some historical bpo's (more or less) relevant to 
Modules/_multiprocessing/semaphore.c:

- bpo-7272
- bpo-5545
- bpo-3770


Commit 99c48a8d315fe55c8e1e8eac2e01cd930cb89686 message says:
Author: Hye-Shik Chang 
Date:   Sat Jun 28 01:04:31 2008 +

Give information for compililation of _multiprocessing.SemLock on FreeBSD:

FreeBSD's P1003.1b semaphore support is highly experimental and
it's disabled by default.  Even if a user loads the experimental
kernel module manually, _multiprocessing doesn't work correctly due
to several known incompatibilities around sem_unlink and sem_getvalue,
yet.


Excerpt from the FreeBSD 9.0 release notes[^1], released four years later 
(2012-01-12):

kern.features.sysv_sem.   System V semaphores support
kern.features.p1003_1b_mqueue.POSIX P1003.1B message queues 
support
kern.features.p1003_1b_semaphores.POSIX P1003.1B semaphores support
kern.features.kposix_priority_scheduling  POSIX P1003.1B real-time 
extensions


I haven't checked OpenBSD or NetBSD yet.


Other relevant commits:

  - c4920e86ef7511b4e858028e870b1811437a71d0
  - 40a6164afa79f6b97e7e40e0f35f6081fde437c2



[^1]: https://www.freebsd.org/releases/9.0R/relnotes-detailed/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45901] store app file type ignores command-line arguments

[Steve Dower]

Change by Steve Dower :


--
keywords: +patch
pull_requests: +28034
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/29799

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45876] Improve accuracy of stdev functions in statistics

[Mark Dickinson]

Mark Dickinson  added the comment:

Since I've failed to find a coherent statement and proof of the general 
principle I articulated anywhere online, I've included one below. (To be 
absolutely clear, the principle is not new - it's very well known, but oddly 
hard to find written down anywhere.)

--

Setup: introducing jots
===

*Notation.* R is the set of real numbers, Z is the set of integers, operators
like *, / and ^ have their mathematical interpretations, not Python ones.

Fix a precision p > 0 and an IEEE 754 binary floating-point format with
precision p; write F for the set of representable values in that format,
including zeros and infinities. (We don't need NaNs, but feel free to include
them if you want to.)

Let rnd : R -> F be the rounding function corresponding to any of the standard
IEEE 754 rounding modes. We're not ignoring overflow and underflow here: rnd is
assumed to round tiny values to +/-0 and large values to +/-infinity as normal.
(We only really care about round-ties-to-even, but all of the below is
perfectly general.)

*Definition.* For the given fixed precision p, a *jot* is a subinterval of the
positive reals of the form (m 2^e, (m+1) 2^e) for some integers m and e, with
m satisfying 2^p <= m < 2^(p+1).

This is a definition-of-convenience, invented purely for the purposes of this
proof. (And yes, the name is silly. Suggestions for a better name to replace
"jot" are welcome. Naming things is hard.)

We've chosen the size of a jot so that between each consecutive pair a and b of
positive normal floats in F, there are exactly two jots: one spanning from a to
the midpoint (a+b)/2, and another spanning from (a+b)/2 to b. (Since jots are
open, the midpoint itself and the floats a and b don't belong to any jot.)

Now here's the key point: for values that aren't exactly representable and
aren't perfect midpoints, the standard rounding modes, whether directed or
round-to-nearest, only ever care about which side of the midpoint the value to
be rounded lies. In other words:

*Observation.* If x and y belong to the same jot, then rnd(x) = rnd(y).

This is the point of jots: they represent the wiggle-room that we have to
perturb a real number without affecting the way that it rounds under any
of the standard rounding modes.

*Note.* Between any two consecutive *subnormal* values, we have 4 or more
jots, and above the maximum representable float we have infinitely many, but
the observation that rnd is constant on jots remains true at both ends of the
spectrum. Also note that jots, as defined above, don't cover the negative
reals, but we don't need them to for what follows.

Here's a lemma that we'll need shortly.

*Lemma.* Suppose that I is an open interval of the form (m 2^e, (m+1) 2^e)
for some integers m and e satisfying 2^p <= m. Then I is either a jot, or a
subinterval of a jot.

*Proof.* If m < 2^(p+1) then this is immediate from the definition. In
the general case, m satisfies 2^q <= m < 2^(q+1) for some integer q with
p <= q. Write n = floor(m / 2^(q-p)). Then:

n <= m / 2^(q-p) < n + 1, so
n * 2^(q-p) <= m < (n + 1) * 2^(q-p), so
n * 2^(q-p) <= m and m + 1 <= (n + 1) * 2^(q-p)

so

   n * 2^(e+q-p) <= m * 2^e and (m + 1) * 2^e <= (n + 1) * 2^(e+q-p)

So I is a subinterval of (n * 2^(e+q-p), (n+1) * 2^(e+q-p)), which is a jot.


The magic of round-to-odd
=

*Definition.* The function to-odd : R -> Z is defined by:

- to-odd(x) = x if x is an integer
- to-odd(x) = floor(x) if x is not an integer and floor(x) is odd
- to-odd(x) = ceil(x) if x is not an integer and floor(x) is even

*Properties.* Some easy monotonicity properties of to-odd, with proofs left
to the reader:

- If x < 2n for real x and integer n, then to-odd(x) < to-odd(2n)
- If 2n < x for real x and integer n, then to-odd(2n) < to-odd(x)

Here's a restatement of the main principle.

*Proposition.* With p and rnd as in the previous section, suppose that x is a
positive real number and that e is any integer satisfying 2^e <= x. Define a
real number y by:

y = 2^(e-p-1) to-odd(x / 2^(e-p-1))

Then rnd(y) = rnd(x).


Proof of the principle
==

In a nutshell, we show that either

- y = x, or
- x and y belong to the same jot

Either way, since rnd is constant on jots, we get rnd(y) = rnd(x).

Case 1: x = m * 2^(e-p) for some integer m. Then x / 2^(e-p-1) = 2m is
an (even) integer, so to-odd(x / 2^(e-p-1)) = (x / 2^(e-p-1)) and y = x.
Hence rnd(y) = rnd(x).

Case 2: m * 2^(e-p) < x < (m + 1) * 2^(e-p) for some integer m.

Then rearranging, 2m < x / 2^(e-p-1) < 2(m+1). So from the monotonicity
properties of to-odd we have:

   2m < to-odd(x / 2^(e-p-1)) < 2(m+1)

And multiplying through by 2^(e-p-1) we get

   m * 2^(e-p) < y < (m+1) * 2^(e-p).

So both x and y belong to the interval I = (m*2^(e-p), (m+1)*2^(e-p-1)).

Furthermore, 2^e <= x < (m + 1) * 2^(e-p), so 2^p < m + 1, and 2^p <= m.
So by the lemma above, I is either a jo

[issue45901] store app file type ignores command-line arguments

[Steve Dower]

Steve Dower  added the comment:

For future reference, in case someone stumbles over this issue, here's how I 
tested this change:

* set $env:IncludeUWP="true"
* did a regular PCbuild/build.bat
* generated a Store layout with "./python.bat PC/layout --preset-appx --copy 
./out"
* mock-installed it with "Add-AppxPackage -Register ./out/appxmanifest.xml"
* used "Default Programs" to change my default to the new installed Python 3.11
* put "import sys; print(sys.argv); input()" into test.py
* ran "test.py arg1 arg2" and inspected the output

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33381] [doc] Incorrect documentation for strftime()/strptime() format code %f

[Vishal Pandey]

Change by Vishal Pandey :


--
keywords: +patch
nosy: +vishalpandeyvip
nosy_count: 5.0 -> 6.0
pull_requests: +28035
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/29801

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40280] Consider supporting emscripten/webassembly as a build target

[Christian Heimes]

Christian Heimes  added the comment:


New changeset 4ebde73b8e416eeb1fd5d2ca3283f7ddb534c5b1 by Christian Heimes in 
branch 'main':
bpo-40280: Move hard-coded feature checks to configure (GH-29789)
https://github.com/python/cpython/commit/4ebde73b8e416eeb1fd5d2ca3283f7ddb534c5b1


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26577] inspect.getclosurevars returns incorrect variable when using class member with the same name as other variable

[Irit Katriel]

Change by Irit Katriel :


--
versions: +Python 3.11 -Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25341] File mode wb+ appears as rb+

[Irit Katriel]

Irit Katriel  added the comment:

Reproduced on 3.11.

--
nosy: +iritkatriel
versions: +Python 3.11 -Python 2.7, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26577] inspect.getclosurevars returns incorrect variable when using class member with the same name as other variable

[Irit Katriel]

Irit Katriel  added the comment:

Reproduced on 3.11.

--
nosy: +iritkatriel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45876] Improve accuracy of stdev functions in statistics

[Tim Peters]

Tim Peters  added the comment:

Mark, ya, MS's Visual Studio's ldexp() has, far as I know, always worked this 
way. The code I showed was run under the 2019 edition, which we use to build 
the Windows CPython.

Raymond,

x = float(i)

is screamingly obvious at first glance.

x = i/1

looks like a coding error at first. The "reason" for different spellings in 
different branches looked obvious in the original: one branch needs to divide, 
and the other doesn't. So the original code was materially clearer to me.

Both, not sure it helps, but this use of round-to-odd appears akin to the 
decimal module's ROUND_05UP, which rounds an operation result in such a way 
that, if it's rounded again - under any rounding mode - to a narrower 
precision, you get the same narrower result as if you had used that rounding 
mode on the original operation to that narrower precision to begin with.

Decimal only needs to adjust the value of the last retained digit to, 
effectively, "encode" all possibilities, but binary needs two trailing bits. 
"Round" and "sticky" are great names for them :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27281] unpickling an xmlrpc.client.Fault raises TypeError

[Irit Katriel]

Irit Katriel  added the comment:

Reproduced in 3.11.

--
nosy: +iritkatriel
versions: +Python 3.11 -Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28422] multiprocessing Manager mutable type member access failure

[Irit Katriel]

Irit Katriel  added the comment:

I don't get any errors from the attached script. Is there still a problem here?

--
nosy: +iritkatriel
resolution:  -> works for me
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45872] Turtle: invalid example for write doc

[Éric Araujo]

Éric Araujo  added the comment:

The doc for the font parameter is:  a triple (fontname, fontsize, fonttype)
but the default value in the signature is:  font='Arial'

I do not know what’s font name vs font type!
Do you know if it’s valid to have font='Arial', in which case the doc should be 
amended?  or is the default value wrong?  (also not sure if it’s a default 
value or an example)

--
nosy: +eric.araujo
title: Turtle documentation, write() -> Turtle: invalid example for write doc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45889] pathlib: Path.match does not work on paths

[Éric Araujo]

Éric Araujo  added the comment:

FWIW I think in the same way as Ronald.

A pattern is not a path, it’s a string expressing rules.
If it matches, the results are paths, but that does not make the pattern a path.

--
nosy: +eric.araujo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45898] ctypes cfield.c defines duplicate ffi_type_* symbols

[Steve Dower]

Steve Dower  added the comment:

This is because we use libffi as a DLL and you can't statically reference 
dynamically loaded addresses on Windows.

You could change that format table to be initialised on first use, or resolve 
the FFI type lazily. Or change ctypes to statically link libffi (including 
updating our FFI build, which IIRC is the one that requires Cygwin, to generate 
static libraries instead of dynamic) - we can't backport this change, either, 
because it affects the layout on disk.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45906] Python github installatiomn issue

[Some User]

New submission from Some User :

Whenever i try to setup python3.9 configure via github, It raises a configure 
error.
``
$ ./configure
checking build system type... type
checking host system type... type
checking for python3.9... no
checking for python3... no
checking for python... no
checking for --enable-universalsdk... no
checking for --with-universal-archs... no
checking MACHDEP... "linux"
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... configure: error: in 
`/path/folder_which_contains_cpython_folder/cpython-3.9':
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details
```

--
components: Installation
messages: 407089
nosy: mazen001.ahmed001
priority: normal
severity: normal
status: open
title: Python github installatiomn issue
type: compile error
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45906] Python github installation issue

[Some User]

Change by Some User :


--
title: Python github installatiomn issue -> Python github installation issue

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28422] multiprocessing Manager mutable type member access failure

[Vilnis Termanis]

Vilnis Termanis  added the comment:

If you un-comment the print_exc() call, you'll see that it still fails - for 
queue.Queue:

Manager failure (for Queue)
Traceback (most recent call last):
  File "fish.py", line 74, in main
add_type_in_own_process(mgr, type_name)
  File "fish.py", line 61, in add_type_in_own_process
use_manager(obj, type_name, mgr=mgr)
  File "fish.py", line 35, in use_manager
obj.append(getattr(mgr, type_name)())
  File "", line 2, in append
  File "/usr/lib/python3.8/multiprocessing/managers.py", line 850, in 
_callmethod
raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError: 
---
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/managers.py", line 243, in 
serve_client
request = recv()
  File "/usr/lib/python3.8/multiprocessing/connection.py", line 251, in recv
return _ForkingPickler.loads(buf.getbuffer())
  File "/usr/lib/python3.8/multiprocessing/managers.py", line 959, in 
RebuildProxy
return func(token, serializer, incref=incref, **kwds)
TypeError: AutoProxy() got an unexpected keyword argument 'manager_owned'
---

--
resolution: works for me -> 
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28422] multiprocessing Manager mutable type member access failure

[Irit Katriel]

Irit Katriel  added the comment:

Version 3.8 is in security fix only mode. Is this a problem on newer versions?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28422] multiprocessing Manager mutable type member access failure

[Vilnis Termanis]

Vilnis Termanis  added the comment:

Apologies, my mistake - it does indeed work with 3.9 & 3.10. Feel free to close.

--
resolution:  -> works for me
status: open -> pending
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28422] multiprocessing Manager mutable type member access failure

[Irit Katriel]

Irit Katriel  added the comment:

Thank you.

--
stage:  -> resolved
status: pending -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44452] Allow paths to be joined without worrying about a leading slash

[Eryk Sun]

Eryk Sun  added the comment:

> and b / a gives me WindowsPath('/a/b'). So I'm like "ok, a seems
> like absolute, I will test for that" but on Windows a.is_absolute()
> is False.

Path.is_absolute() is true if a path has a `root` and, for a Windows path, also 
a `drive`.

In C++, the filesystem::path is_absolute() method is similar. In Windows it's 
true if both has_root_name() (i.e. a drive) and has_root_directory() are true. 
In POSIX it just depends on has_root_directory(). pathlib.Path is also 
consistent with C++ filesystem::path with regard to appending paths with the 
slash operator [1]. I would prefer for it to remain so.

FYI, in Windows, "/a/b" is resolved using the drive of the process current 
working directory, which may be a UNC path. The drive of a UNC path is the 
share path, such as "\\server\share". 

Another type of relative path in Windows is a drive-relative path, which 
applies to drive-letter drives only. For example:

>>> Path('C:spam') / "eggs"
WindowsPath('C:spam/eggs')
>>> Path('C:spam') / "/eggs"
WindowsPath('C:/eggs')

"C:spam" is relative to the current working directory on drive "C:". The API 
gets the working directory for the target drive either from the process current 
working directory, if it's a path on the target drive, or from an "=:" environment variable, such as "=Z:". (The Windows API allows 
environment variable names to begin with "=".) If the process current working 
directory is on a different drive, and the environment variable for the target 
drive isn't set, the API defaults to using the root directory on the target 
drive. Setting these per-drive working directory environment variables is up to 
the application. Python's os.chdir() supports them. 

---
[1] https://en.cppreference.com/w/cpp/filesystem/path/append

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45876] Improve accuracy of stdev functions in statistics

[Raymond Hettinger]

Raymond Hettinger  added the comment:

[Tim]
> Note that, on Windows, ldexp() in the presence of 
> denorms can truncate. Division rounds, so
>
>assert x / 2**i == ldexp(x, -i)
>
> can fail.

Objects/longobject.c::long_true_divide() uses ldexp() internally.  Will it 
suffer the same issues with subnormals on Windows?  Is CPython int/int true 
division guaranteed to be correctly rounded?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45876] Improve accuracy of stdev functions in statistics

[Mark Dickinson]

Mark Dickinson  added the comment:

> Will it suffer the same issues with subnormals on Windows?

No, it should be fine. All the rounding has already happened at the point where 
ldexp is called, and the result of the ldexp call is exact.

> Is CPython int/int true division guaranteed to be correctly rounded?

Funny you should ask. :-) There's certainly no documented guarantee, and there 
_is_ a case (documented in comments) where the current code may not return 
correctly rounded results on machines that use x87: there's a fast path where 
both numerator and denominator fit into an IEEE 754 double without rounding, 
and we then do a floating-point division.

But we can't hit that case with the proposed code, since the numerator will 
always have at least 55 bits, so the fast path is never taken.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45876] Improve accuracy of stdev functions in statistics

[Tim Peters]

Tim Peters  added the comment:

> Objects/longobject.c::long_true_divide() uses ldexp() internally.
> Will it suffer the same issues with subnormals on Windows?

Doesn't look like it will. In context, looks like it's ensuring that ldexp can 
only lose trailing 0 bits, so that _whatever_ ldexp does in the way of rounding 
is irrelevant. But it's not doing this because of Windows - it's to prevent 
"double-rounding" errors regardless of platform.

> Is CPython int/int true division guaranteed to be correctly rounded?

If there's some promise of that in the docs, I don't know where it is. But the 
code clearly intends to strive for correct rounding. Ironically, PEP 238 
guarantees that if it is correctly rounded, that's purely by accident ;-) :

"""
True division for ints and longs will convert the arguments to float and then 
apply a float division. That is, even 2/1 will return a float 
"""

But i/j is emphatically not implemented via float(i)/float(j).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45876] Improve accuracy of stdev functions in statistics

[Mark Dickinson]

Mark Dickinson  added the comment:

> All the rounding has already happened at the point where ldexp is called, and 
> the result of the ldexp call is exact.

Sketch of proof:

[Here](https://github.com/python/cpython/blob/4ebde73b8e416eeb1fd5d2ca3283f7ddb534c5b1/Objects/longobject.c#L3929)
 we have:

shift = Py_MAX(diff, DBL_MIN_EXP) - DBL_MANT_DIG - 2;

from which (assuming IEEE 754 as usual) shift >= -1076. (DBL_MIN_EXP = -1021, 
DBL_MANT_DIG = 53)

[Here](https://github.com/python/cpython/blob/4ebde73b8e416eeb1fd5d2ca3283f7ddb534c5b1/Objects/longobject.c#L4008)
 we round away the last two or three bits of x, after which x is guaranteed to 
be a multiple of 4:

x->ob_digit[0] = low & ~(2U*mask-1U);

Then after converting the PyLong x to a double dx with exactly the same value 
[here](https://github.com/python/cpython/blob/4ebde73b8e416eeb1fd5d2ca3283f7ddb534c5b1/Objects/longobject.c#L4020)
 we make the ldexp call:

result = ldexp(dx, (int)shift);

At this point dx is a multiple of 4 and shift >= -1076, so the result of the 
ldexp scaling is a multiple of 2**-1074, and in the case of a subnormal result, 
it's already exactly representable.


For the int/int division possibly not being correctly rounded on x87, see 
[here](https://github.com/python/cpython/blob/4ebde73b8e416eeb1fd5d2ca3283f7ddb534c5b1/Objects/longobject.c#L3889-L3892).

It won't affect _this_ application, but possibly we should fix this anyway. 
Though the progression of time is already effectively fixing it for us, as x87 
becomes less and less relevant.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45866] Out of tree build of Python 3.11.0a2+ breaks regen-frozen

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 9a7611a7c4da6406383f8ffcea272ded6f890f1e by Victor Stinner in 
branch 'main':
bpo-45866: Fix typo in the NEWS entry (GH-29798)
https://github.com/python/cpython/commit/9a7611a7c4da6406383f8ffcea272ded6f890f1e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45866] Out of tree build of Python 3.11.0a2+ breaks regen-frozen

[miss-islington]

Change by miss-islington :


--
pull_requests: +28037
pull_request: https://github.com/python/cpython/pull/29804

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45866] Out of tree build of Python 3.11.0a2+ breaks regen-frozen

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 4.0 -> 5.0
pull_requests: +28036
pull_request: https://github.com/python/cpython/pull/29803

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45876] Improve accuracy of stdev functions in statistics

[Mark Dickinson]

Mark Dickinson  added the comment:

Concrete example of int/int not being correctly rounded on systems using x87 
instructions: on those systems, I'd expect to see 1/2731 return a result of 
0.00036616623947272064 (0x1.7ff4005ffd002p-12), as a result of first rounding 
to 64-bit precision and then to 53-bit. The correctly-rounded result is 
0.0003661662394727206 (0x1.7ff4005ffd001p-12).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45866] Out of tree build of Python 3.11.0a2+ breaks regen-frozen

[miss-islington]

miss-islington  added the comment:


New changeset fc16ea9c8b8769af8a4c0c16fed7eba2e8bf4019 by Miss Islington (bot) 
in branch '3.10':
bpo-45866: Fix typo in the NEWS entry (GH-29798)
https://github.com/python/cpython/commit/fc16ea9c8b8769af8a4c0c16fed7eba2e8bf4019


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45866] Out of tree build of Python 3.11.0a2+ breaks regen-frozen

[miss-islington]

miss-islington  added the comment:


New changeset b3f14dacfea54332e2ddde792142d818b3554dbc by Miss Islington (bot) 
in branch '3.9':
bpo-45866: Fix typo in the NEWS entry (GH-29798)
https://github.com/python/cpython/commit/b3f14dacfea54332e2ddde792142d818b3554dbc


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42268] ./configure failing when --with-memory-sanitizer specified

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +28038
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/29806

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21944] Allow copying of CodecInfo objects

[Irit Katriel]

Change by Irit Katriel :


--
type: behavior -> enhancement
versions: +Python 3.11 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12020] [doc] Attribute error with flush on stdout,stderr

[Irit Katriel]

Change by Irit Katriel :


--
keywords: +easy
title: Attribute error with flush on stdout,stderr -> [doc] Attribute error 
with flush on stdout,stderr
versions: +Python 3.11 -Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com