[issue31834] BLAKE2: the (pure) SSE2 impl forced on x86_64 is slower than reference

[Michał Górny]

New submission from Michał Górny :

The setup.py file for Python states:

if (not cross_compiling and
os.uname().machine == "x86_64" and
sys.maxsize >  2**32):
# Every x86_64 machine has at least SSE2.  Check for sys.maxsize
# in case that kernel is 64-bit but userspace is 32-bit.
blake2_macros.append(('BLAKE2_USE_SSE', '1'))

While the assertion about having SSE2 is true, it doesn't mean that it's 
worthwhile to use. I've tested pure (i.e. without SSSE3 and so on) on three 
different machines, getting the following results:

Athlon64 X2 (SSE2 is the best supported variant), 540 MiB of data:

SSE2: [5.18998800443, 5.070812243997352]
ref:  [2.0161159170020255, 2.0475422790041193]

Core i3, same data file:

SSE2: [1.92442592602, 1.9246174693, 1.929803750191]
ref:  [1.794020974667, 1.790085556976, 1.783553876418]

Xeon E5630 server, 230 MiB data file:

SSE2: [0.7671358410007088, 0.7797677099879365, 0.7648976119962754]
ref:  [0.5784736709902063, 0.5717909929953748, 0.5717219939979259]

So in all the tested cases, pure SSE2 implementation is *slower* than the 
reference implementation. SSSE3 and other variants are faster and AFAIU they 
are enabled automatically based on CFLAGS, so it doesn't matter for most of the 
systems.

However, for old CPUs that do not support SSSE3, the choice of SSE2 makes the 
algorithm prohibitively slow -- it's 2.5 times slower than the reference 
implementation!

--
components: Extension Modules
messages: 304696
nosy: mgorny
priority: normal
severity: normal
status: open
title: BLAKE2: the (pure) SSE2 impl forced on x86_64 is slower than reference
type: performance
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31828] Support Py_tss_NEEDS_INIT outside of static initialisation

[Nick Coghlan]

Nick Coghlan  added the comment:

Right, the cases we were aiming to cover were:

- C variable declarations ("static Py_tss_t tss_key = Py_tss_NEEDS_INIT;")
- dynamic allocation with PyThread_tss_alloc
- resetting a key back to the uninitialised state with PyThread_tss_delete

The problem we have is that the second field in Py_tss_t is platform dependent, 
and not all platforms define a safe "unused" value for their NATIVE_TSS_KEY_T, 
which means Py_tss_NEEDS_INIT ends up being only a partial initialiser (earlier 
versions of the PEP used a field initialiser, but we were asked to switch it to 
a partial initialiser in order to support more compilers).

We *could* offer a `PyThread_tss_reset` (to reset a key back to 
Py_tss_NEEDS_INIT), but that's confusingly similar to PyThread_tss_delete.

Another option would be to check for typed partial initialiser support in the 
configure script, and declare Py_tss_NEEDS_INIT accordingly. However, that 
wouldn't solve the problem for any clients that are themselves also attempting 
to write cross-platform code.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16737] Different behaviours in script run directly and via runpy.run_module

[Nick Coghlan]

Nick Coghlan  added the comment:

Is there a relevant discrepancy other than __file__ sometimes being absolute?

If code wants to be certain that __file__ is relative to the current directory, 
they need to run it through os.relpath() - there's no requirement for 
implementations one way or the other as to whether __file__ is absolute or 
relative

If we changed anything in CPython, it would be to make __main__.__file__ always 
absolute, even for scripts - we already changed plain imports to work that way.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31834] BLAKE2: the (pure) SSE2 impl forced on x86_64 is slower than reference

[Michał Górny]

Change by Michał Górny :


--
keywords: +patch
pull_requests: +4036
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31314] email throws exception with oversized header input

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +4037

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31829] Portability issues with pickle

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +4038
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31314] email throws exception with oversized header input

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests:  -4037

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25083] Python can sometimes create incorrect .pyc files

[tzickel]

tzickel  added the comment:

OK, This issue has been biting me a few more times in production, so for now 
I've added the environment variable PYTHONDONTWRITEBYTECODE which resolves it 
(but it's a hack). I'm sure I am not the only one with it (recall that this is 
happening in a complex setup where I run python in windows via a network drive 
hosted in netapp, and runs thousands of times from many machines).

So, I've been looking for a simple way to show you it's a major fault at the 
python 2 import I/O error checking, and found that new strace has fault 
injection capability :)

In this demo I'll be running under debian sid (has strace version high enough 
for fault injection and latest python 2), you can use docker if you don't have 
it.

On my mac, I'm running (this example is on one of python's init modules, but of 
course can happen on any .py file):

user$ docker run -it --cap-add SYS_PTRACE debian:sid

The cap-add is needed for strace to run.

root@2dcc36934ea6:/# apt-get update && apt-get install -y strace python

root@2dcc36934ea6:/# python
Python 2.7.14 (default, Sep 17 2017, 18:50:44)
[GCC 7.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()

Python works just fine.

root@2dcc36934ea6:/# strace -P /usr/lib/python2.7/sre_parse.pyc -P 
/usr/lib/python2.7/sre_parse.py -e trace=read -e fault=read python
read(7, 0x55c3ac2ad900, 4096)   = -1 EPERM (Operation not permitted) 
(INJECTED)
...
read(6, 0x55c3ac2cb680, 4096)   = -1 EPERM (Operation not permitted) 
(INJECTED)
...
Traceback (most recent call last):
  File "/usr/lib/python2.7/site.py", line 554, in 
main()
...
  File "/usr/lib/python2.7/sre_compile.py", line 572, in compile
p = sre_parse.parse(p, flags)
AttributeError: 'module' object has no attribute 'parse'
+++ exited with 1 +++

This command simply causes the python process to fail the read command on the 
files sre_parse.py and sre_parse.pyc (the .pyc btw already existed from 
previous run).
This should be OK, since it can't read a required module form disk.

root@2dcc36934ea6:/# python
Traceback (most recent call last):
  File "/usr/lib/python2.7/site.py", line 554, in 
main()
...
  File "/usr/lib/python2.7/sre_compile.py", line 572, in compile
p = sre_parse.parse(p, flags)
AttributeError: 'module' object has no attribute 'parse'

This is already bad, python does not work anymore, now even without an I/O 
error :(

root@2dcc36934ea6:/# ls -l /usr/lib/python2.7/sre_parse.pyc
-rw-r--r-- 1 root root 118 Oct 21 09:20 /usr/lib/python2.7/sre_parse.pyc

If we check, we see that the previous python instance with I/O error created an 
empty byte code valid sre_parse.pyc (you can check it by dis.dis it, and see 
it's empty code object), this is the crux of the bug.

root@2dcc36934ea6:/# rm /usr/lib/python2.7/sre_parse.pyc

let's delete the bad .pyc file

root@2dcc36934ea6:/# python
Python 2.7.14 (default, Sep 17 2017, 18:50:44)
[GCC 7.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()

yey, python works now again

root@2dcc36934ea6:/# ls -l /usr/lib/python2.7/sre_parse.pyc
-rw-r--r-- 1 root root 21084 Oct 21 09:20 /usr/lib/python2.7/sre_parse.pyc

We can see now that the .pyc file has a much bigger size (21084 bytes compared 
to 118 from before)

root@2dcc36934ea6:/# strace -P /usr/lib/python2.7/sre_parse.pyc -P 
/usr/lib/python2.7/sre_parse.py -e trace=read -e fault=read python -B
read(7, 0x55ceb72a7900, 4096)   = -1 EPERM (Operation not permitted) 
(INJECTED)
...
read(6, 0x55ceb72c5680, 4096)   = -1 EPERM (Operation not permitted) 
(INJECTED)
...
Traceback (most recent call last):
  File "/usr/lib/python2.7/site.py", line 554, in 
main()
...
AttributeError: 'module' object has no attribute 'parse'
+++ exited with 1 +++

We can now try this issue with python -B which should not try to create .pyc 
files

root@2dcc36934ea6:/# python
Python 2.7.14 (default, Sep 17 2017, 18:50:44)
[GCC 7.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()

yey, python still works (hopefully if this is an network I/O error, it will 
stop occurring on a future run with a watchdog for an server app)


A less likely variant (but possible) is that if the .pyc does not exist, and 
you have an I/O error on importing the .py, it will produce an bad .pyc file, 
you can try it:
root@2dcc36934ea6:/# rm /usr/lib/python2.7/sre_parse.pyc
root@2dcc36934ea6:/# strace -P /usr/lib/python2.7/sre_parse.py -e trace=read -e 
fault=read python
root@2dcc36934ea6:/# python
Python will not work until you delete the .pyc file.



(*) The I/O error needs to occur when the AST parser will still produce a valid 
code object, this easily happens when the error is just when starting to read 
the file (will produce an empty code AST which will create an empty code object)

-

[issue31829] Portability issues with pickle

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

PR 4067 fixes following issues when unpickle on Unix or in binary mode files 
written with protocol 0 in text mode on Windows:

* ints were unpickled as longs by cPickle.
* bools were unpickled as longs by cPickle and as ints by pickle.
* floats couldn't be unpickled by cPickle.
* strings couldn't be unpickled by pickle.
* instances and globals couldn't be unpickled. And error messages were 
confusing due to invisible \r.
* pickles with protocol 0 containing Unicode string with \x1a couldn't be 
unpickled on Windows in text mode.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25083] Python can sometimes create incorrect .pyc files

[tzickel]

tzickel  added the comment:

Ignore the hash append / link at the start of each shell command (it's the 
output from docker, and not related to python commits).

BTW, forgot to mention, of course when doing the fault injection on the .py 
files, the error is bad as well, it should be I/O error, and instead it shows 
that it's an empty module:
AttributeError: 'module' object has no attribute 'parse'

(The patch actually fixes that).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31835] _PyFunction_FastCallDict and _PyFunction_FastCallKeywords: fast path not used

[Anselm Kruis]

New submission from Anselm Kruis :

Just a minor performance issue.

The C functions _PyFunction_FastCallDict() and _PyFunction_FastCallKeywords() 
(branch 'master', Objects/call.c) and their predecessors fast_function() and 
_PyFunction_FastCallDict() in Python/ceval.c all contain the following 
sub-expression in the "if"-statement for the fast-path. For instance 
Objects/call.c:318

 co->co_flags == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE)

Now, if co_flags has any of the CO_FUTURE_... bits set, the expression is 
always False and the fast path is not used.

Currently this affects only Python 3.6 and Python 2.7, because other Python 
versions do not use the __future__ mechanism.

The fix is simple. Replace the faulty sub-expression by

 (co->co_flags & (~PyCF_MASK)) == (CO_OPTIMIZED | CO_NEWLOCALS | CO_NOFREE))

I discovered this issue while debugging reference leaks in Stackless Python a 
few month ago. It is hard to write a test case, but one can compare C call 
stacks using a debugger.

$ ulimit -c unlimited  # enable core dumps
$ python3.6 -c 'from __future__ import generator_stop; import os; (lambda: 
os.abort())()'
$ gdb -batch -ex bt  python3.6 core > trace_with_future
$ python3.6 -c 'import os; (lambda: os.abort())()'
$ gdb -batch -ex bt  python3.6 core > trace_without_future

If you compare the traces, the difference is in stack frame #9. Same for 
python2.7.

--
components: Interpreter Core
messages: 304702
nosy: anselm.kruis
priority: normal
severity: normal
status: open
title: _PyFunction_FastCallDict and _PyFunction_FastCallKeywords: fast path not 
used
type: performance
versions: Python 2.7, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16737] Different behaviours in script run directly and via runpy.run_module

[Jason R. Coombs]

Jason R. Coombs  added the comment:

The other major difference and the only one that's affected me is the presence 
of sys.path[0] == ''.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26858] setting SO_REUSEPORT fails

[Xavier de Gaye]

Xavier de Gaye  added the comment:

Philip, 9791c5d55f52 was commited in november 2013 and issue19901 was closed in 
december 2013, so how can they be an appropriate solution for a problem 
reported on 2017-01-13 in msg285392 ?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31831] EmailMessage.add_attachment(filename="long or spécial") crashes or produces invalid output

[calimeroteknik]

calimeroteknik  added the comment:

I confirm that as for the crash, the patch in gh-3488 fixes it.
The first code excerpt in my initial report now outputs the following, valid 
headers:

Content-Type: text/plain
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
 filename*0*=utf-8''I%20thought%20I%20could%20put%20a%20few%20words%20in%20th;
 filename*1*=e%20filename%20but%20apparently%20it%20does%20not%20go%20so%20we;
 filename*2*=ll.txt
MIME-Version: 1.0


However, when Unicode is added and the filename is short, things don't look 
right, this code:

import email.message
mail = email.message.EmailMessage()
mail.add_attachment(b"test", maintype="text", subtype="plain", filename="é.txt")
print(mail)

Results in these headers:

Content-Type: text/plain
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="é.txt"
MIME-Version: 1.0

To begin with, it is easy to deduce that there is no way to know that this 'é' 
character is UTF-8.
And it's two 8-bit values at east one of which is detectably outside of 7-bit 
US-ASCII.


Quoting https://tools.ietf.org/html/rfc2231#page-4:
>a lightweight encoding mechanism is needed to accommodate 8-bit information in 
>parameter values.

The 8-bit encoding goes straight through instead of undergoing the encoding 
process, which seems required in my interpretation of RFC2231.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31831] EmailMessage.add_attachment(filename="long or spécial") crashes or produces invalid output

[R. David Murray]

R. David Murray  added the comment:

You are correct, that is a bug.  Presumably I forgot to check for non-ascii 
when the parameter value doesn't need to be folded.  I'm not sure when I'll 
have time to look at this, unfortunately :(.  If you can see how to fix it, you 
could submit a PR against my PR branch, I think.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20210] Support the *disabled* marker in Setup files

[Xavier de Gaye]

Change by Xavier de Gaye :


--
pull_requests:  -2537

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16737] Different behaviours in script run directly and via runpy.run_module

[Nick Coghlan]

Nick Coghlan  added the comment:

Yeah, that one I definitely think could be improved. Could you file a separate 
RFE suggesting that we update sys.path[0] based on __main__.__spec__.origin 
after we look up __main__.__spec__?

That way it will only stay as the current directory if the module being 
executed is in a subdirectory of the current directory, and will otherwise be 
updated appropriately for wherever we actually found the main module.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26858] setting SO_REUSEPORT fails

[Guido van Rossum]

Guido van Rossum  added the comment:

I assume he means that a similar piece of code should be inserted elsewhere. I 
am still waiting for Philip's PR though.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31836] test_code_module fails after test_idle

[Serhiy Storchaka]

New submission from Serhiy Storchaka :

$ ./python -m test -uall test_idle test_code_module
Run tests sequentially
0:00:00 load avg: 0.42 [1/2] test_idle
0:00:02 load avg: 0.42 [2/2] test_code_module
test test_code_module failed -- Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/test/test_code_module.py", line 35, in 
test_ps1
self.assertEqual(self.sysmod.ps1, '>>> ')
AssertionError:  != '>>> '

test_code_module failed
1 test OK.

1 test failed:
test_code_module

Total duration: 2 sec
Tests result: FAILURE

--
assignee: terry.reedy
components: IDLE, Tests
messages: 304709
nosy: serhiy.storchaka, terry.reedy
priority: normal
severity: normal
status: open
title: test_code_module fails after test_idle
type: behavior
versions: Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25083] Python can sometimes create incorrect .pyc files

[tzickel]

tzickel  added the comment:

Added a script to check if the bug exists (provided you have an updated strace 
4.15 or above).

Without patch:
# ./import_io_check.sh
strace: Requested path 'tmp.py' resolved into '/root/tmp.py'
read(3, 0x55fc3a71cc50, 4096)   = -1 ENOSYS (Function not implemented) 
(INJECTED)
read(3, 0x55fc3a71cc50, 4096)   = -1 ENOSYS (Function not implemented) 
(INJECTED)
Traceback (most recent call last):
  File "", line 1, in 
ImportError: No module named py
+++ exited with 1 +++
Traceback (most recent call last):
  File "", line 1, in 
ImportError: No module named py
Bug exists an incorrect .pyc has been produced

With patch:
# PYTHON=Python-2.7.14-with-patch/python ./import_io_check.sh
strace: Requested path 'tmp.py' resolved into '/root/tmp.py'
read(3, 0x55a8ff7d3020, 4096)   = -1 ENOSYS (Function not implemented) 
(INJECTED)
read(3, 0x55a8ff7d3020, 4096)   = -1 ENOSYS (Function not implemented) 
(INJECTED)
Traceback (most recent call last):
  File "", line 1, in 
  File "tmp.py", line 1

^
SyntaxError: unexpected EOF while parsing
+++ exited with 1 +++
Script finished successfully

--
nosy: +brett.cannon
Added file: https://bugs.python.org/file47229/import_io_check.sh

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30156] PYTHONDUMPREFS segfaults on exit

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Result on the same computer:
 
Mean +- std dev: [python0] 146 ns +- 5 ns -> [python] 206 ns +- 2 ns: 1.41x 
slower (+41%)

The difference now is smaller (41% instead of 58%) because calling with a tuple 
now is 16% slower.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31486] calling a _json.Encoder object raises a SystemError in case obj.items() returned a tuple

[Oren Milman]

Oren Milman  added the comment:

ISTM that PR 3840 resolved this issue (as part of bpo-28280).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28326] multiprocessing.Process depends on sys.stdout being open

[Antoine Pitrou]

Antoine Pitrou  added the comment:

In issue31804, we see that sys.stdout and sys.stderr can sometimes simply be 
None, so we must check for that too.

--
nosy: +pitrou
versions:  -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6721] Locks in the standard library should be sanitized on fork

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I think we should somehow move forward on this, at least for logging locks 
which can be quite an annoyance.

There are two possible approaches:
- either a generic mechanism as posted by sbt in reinit_locks_2.diff
- or a logging-specific fix using os.register_at_fork()

What do you think?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31837] ParseError in test_all_project_files()

[Serhiy Storchaka]

New submission from Serhiy Storchaka :

test_all_project_files() in test_lib2to3 emits warnings in verbose mode.

/home/serhiy/py/cpython/Lib/lib2to3/tests/test_parser.py:415: UserWarning: 
ParseError on file /home/serhiy/py/cpython/Lib/lib2to3/main.py (bad input: 
type=22, value='=', context=('', (130, 38)))
  warnings.warn('ParseError on file %s (%s)' % (filepath, err))
/home/serhiy/py/cpython/Lib/lib2to3/tests/test_parser.py:415: UserWarning: 
ParseError on file 
/home/serhiy/py/cpython/Lib/lib2to3/tests/pytree_idempotency.py (bad input: 
type=22, value='=', context=('', (49, 33)))
  warnings.warn('ParseError on file %s (%s)' % (filepath, err))

ParseError is raised by lines like:

print("Use --help to show usage.", file=sys.stderr)

Seems "from __future__ import print_function" doesn't work.

--
components: 2to3 (2.x to 3.x conversion tool), Tests
messages: 304715
nosy: benjamin.peterson, serhiy.storchaka
priority: normal
severity: normal
status: open
title: ParseError in test_all_project_files()
type: behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29802] A possible null-pointer dereference in struct.s_unpack_internal()

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +4039

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6721] Locks in the standard library should be sanitized on fork

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Oh, I forgot that IO buffered objects also have a lock.  So we would have to 
special-case those as well, unless we take the generic approach...

A problem with the generic approach is that it would leave higher-level 
synchronization objects such as RLock, Event etc. in an inconsistent state.  
Not to mention the case where the lock is taken by the thread calling fork()...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28280] Always return a list from PyMapping_Keys/PyMapping_Values/PyMapping_Items

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31486] calling a _json.Encoder object raises a SystemError in case obj.items() returned a tuple

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
stage:  -> needs patch
versions: +Python 3.6 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28655] Tests altered the execution environment in isolated mode

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
dependencies: +ParseError in test_all_project_files()

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31589] Links for French documentation PDF is broken: LaTeX issue with non-ASCII characters?

[jfbu]

Change by jfbu :


--
pull_requests: +4040

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31589] Links for French documentation PDF is broken: LaTeX issue with non-ASCII characters?

[jfbu]

jfbu  added the comment:

I have made a PR at https://github.com/python/cpython/pull/4069 which enhances 
`conf.py` with some pdflatex extra Unicode configuration. I tested it with 
building PDF English documentation at master (at 
https://github.com/python/cpython/tree/db60a5bfa5d5f7a6f1538cc1fe76f0fda57b524e)
 and at 3.6 branch (at 
https://github.com/python/cpython/tree/1e78ed6825701029aa45a68f9e62dd3bb8d4e928)
 and also French documentation at 3.6 (at 
https://github.com/python/python-docs-fr/commit/76b522b79c3caa26658920c714acf8fac0c20eeb).
 The changes are only for ``pdflatex`` builds: if `latex_engine` is set to 
`xelatex`, `lualatex`, or `platex` (automatic if language is `ja`), nothing is 
modified.

--
nosy: +jfbu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29802] A possible null-pointer dereference in struct.s_unpack_internal()

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 73c4708630f99b94c35476529748629fff1fc63e by Serhiy Storchaka in 
branch 'master':
Fix bytes warnings in test_struct (added in bpo-29802). (#4068)
https://github.com/python/cpython/commit/73c4708630f99b94c35476529748629fff1fc63e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31827] Remove os.stat_float_times()

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

stat_result is a named 10-tuple, containing several additional attributes. The 
last three items are st_atime, st_mtime and st_ctime as integers. Accessing 
them by name returns floats. Isn't a time to make them floats when access 
stat_result as a tuple?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31836] test_code_module fails after test_idle

[Terry J. Reedy]

Change by Terry J. Reedy :


--
keywords: +patch
pull_requests: +4041
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31836] test_code_module fails after test_idle

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Running test_code_module afte test_idle, which somewhere executes idle code 
that sets sys.ps1, exposed a deficiency in test_code_module.

TestInteractiveConsole.test_ps1 is intended to test this code in 
InteractiveConsole.interact.

try:
sys.ps1
except AttributeError:
sys.ps1 = ">>> "

The existing test only tried to test the except branch, but without insuring 
that the AttributeError occurs.  PR 4070 fixes that and adds a test of the try 
branch, that sys.ps1 is respected and left alone (and later used as it) when 
present.  Ditto for test_ps2.

--
components: +Library (Lib) -IDLE
stage: patch review -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27640] add the '--disable-test-suite' option to configure

[Xavier de Gaye]

Xavier de Gaye  added the comment:

> Ah, the discussion restarted on the other issue: 
> http://bugs.python.org/issue20210#msg287516

Issue 20210 has ended up as a change in the build system eventually and not the 
installation process, only a tiny step towards the configuration of a smaller 
Python distribution :(

After browsing again the last discussion on "[Python-ideas] size of the 
installation of Python on mobile devices" [1], I find all the arguments in 
favor of such changes still relevant. I would also add the following new 
arguments:

* One frequent argument used against those changes is that "[packagers for 
mobile and embedded devices] will have to patch the code base anyway to get 
things working in such an environment". But this is not true, see issue 30386 
(Add a build infrastructure for Android) that proposes an addition to Python 
that does not modify a SINGLE LINE in the existing source code. So it is not 
true (or at least not anymore true) that packagers for mobile and embedded 
devices need to patch the code base anyway.

* When Python is installed by the termux [2] Android application on an Android 
device, the size of the installed standard library is 34M. The termux 
installation script does not install the following files [3]:

TERMUX_PKG_RM_AFTER_INSTALL="
bin/python${_MAJOR_VERSION}m bin/idle*
lib/python${_MAJOR_VERSION}/idlelib
lib/python${_MAJOR_VERSION}/test
lib/python${_MAJOR_VERSION}/tkinter
lib/python${_MAJOR_VERSION}/turtledemo
"

But it fails to not install also:
lib/python${_MAJOR_VERSION}/config-${_MAJOR_VERSION}m
lib/python${_MAJOR_VERSION}/ctypes/test
lib/python${_MAJOR_VERSION}/distutils/tests
lib/python${_MAJOR_VERSION}/sqlite3/test
lib/python${_MAJOR_VERSION}/unittest/test

This is not consistent and should be fixed by a proper (endorsed by Python) 
installation process.

[1] https://mail.python.org/pipermail/python-ideas/2016-July/
[2] https://termux.com/
[3] 
https://github.com/termux/termux-packages/blob/master/packages/python/build.sh

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6721] Locks in the standard library should be sanitized on fork

[Gregory P. Smith]

Change by Gregory P. Smith :


--
pull_requests: +4042

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6721] Locks in the standard library should be sanitized on fork

[Gregory P. Smith]

Gregory P. Smith  added the comment:

logging is pretty easy to deal with so I created a PR.

bufferedio.c is a little more work as we either need to use the posixmodule.c 
os.register_at_fork API or expose it as an internal C API to be able to call it 
to add acquires and releases around the buffer's self->lock member when 
non-NULL.  either way, that needs to be written safely so that it doesn't crash 
if fork happens after a buffered io struct is freed.  (unregister at fork 
handlers when freeing it? messy)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6721] Locks in the standard library should be sanitized on fork

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Actually, we already have a doubly-linked list of buffered IO objects
(for another purpose), so we can reuse that and register a single set of
global callbacks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25083] Python can sometimes create incorrect .pyc files

[Gregory P. Smith]

Change by Gregory P. Smith :


--
nosy:  -gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25083] Python can sometimes create incorrect .pyc files

[Gregory P. Smith]

Gregory P. Smith  added the comment:

I'm un-cc'ing myself as I don't use Python 2 in an environment where we allow 
it to write .pyc files at application runtime.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31838] Python 3.4 supported SSL version

[Mauro]

New submission from Mauro :

Hello and sorry to bother. This is my first message to the list.

I'm trying to build python 3.4.7 downloaded from python.org (released in August 
this year) and while following the exact same steps detailed in  
https://bugs.python.org/issue29027, I get getting the exact same error he does.

Doesn't Python 3.4 support OpenSSL v 1.1.0? Is this is the case, I guess it 
will never do, right?

More importantly, is there a place to get build dependencies (with appropriate 
versions) for each CPython version? I scrawled the source and the GitHub 
repositories without luck.

I guess this has been asked before, but after several hours, I couldn't find 
anything.

Cheers,
Mauro

--
assignee: christian.heimes
components: SSL
messages: 304725
nosy: christian.heimes, sabian2008
priority: normal
severity: normal
status: open
title: Python 3.4 supported SSL version
versions: Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31831] EmailMessage.add_attachment(filename="long or spécial") crashes or produces invalid output

[calimeroteknik]

calimeroteknik  added the comment:

Eventually there is no bug, I was just confused at the output of print() on the 
EmailMessage.

I noticed that in email/_header_value_parser.py policy.utf8 was True.
The reason is found in email/message.py line 970 (class MIMEPart):

def __str__(self):
return self.as_string(policy=self.policy.clone(utf8=True)

print() will use __str__() and this is why it happens.

I didn't dig out the exact reason since there are so many delegated calls.
In any case, the flattened message in smtplib.SMTP does contain what 
as_string() returns, which means that the policy.utf8 is only forced when using 
print().

Sorry for the false alert.
I can guess that the intention in forcing policy.utf8=True in __str__() was 
that SMTPUTF8 output is visually prettier than any ASCII-armored text.

After additional fuzzing, checking the output with EmailMessage.as_string(), 
everything seems OK.

That's a +1 for gh-3488, which fixes this bug.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25083] Python can sometimes create incorrect .pyc files

[Nick Coghlan]

Nick Coghlan  added the comment:

Adding a couple of Red Hat folks to the nosy list, as even though this was 
originally reported for Windows, the reproducers show that it's a 
cross-platform issue.

tzickel would you mind signing the Python CLA and turning your patch into a PR 
on Github?

--
nosy: +cstratak, encukou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31839] datetime: add method to parse isoformat() output

[Oren Tirosh]

New submission from Oren Tirosh :

The object types in the datetime module can produce a standard string 
representation (rfc3339, a subset of iso8601) but they do not provide a method 
to parse it.

Proposed method names: isoparse or fromisoformat.

In addition, a constructor call with a single string argument may also be 
supported. This would make the behavior of datetime/date/time more similar to 
other value types such as int, str or float and maintain the same invariant of 
x == type(x)(str(x))

Requirements: support lossless round-tripping of all valid datetime object 
values to str and back, with the exception of objects having a custom tzinfo 
(not None or an instance of datetime.timezone).

The _strptime format of '%Y-%m-%d %H:%M:S.%f%z' comes close, but fails to meet 
these requirements in the following ways:

1. %z matches +HHMM, not +HH:MM (addressed by issue 31800, currently discussed 
on datetime-sig)
2. %z does not match the empty string, indicating a naive datetime object 
(tzinfo=None)
3. .%f requires a fraction part, while isoformat drops it when the timestamp is 
an exact second (microsecond=0).

--
messages: 304728
nosy: orent
priority: normal
severity: normal
status: open
title: datetime: add method to parse isoformat() output
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31840] `ImportError` is` raised` only when `python -m unittest discover` is given

[Hiroki “h2” Horiuchi]

New submission from Hiroki “h2” Horiuchi :

Create a package `pkg`. Put `test.py` there. Write `none = None` in 
`pkg/__init__.py`.
In the directory containing `pkg/`, `python [23] -m unittest pkg.test` will 
terminate normally, but `python [23] -m unittest discover pkg` will raise 
`ImportError`. The phenomenon is the same in macOS Sierra, Debian 9.0, Windows 
7. I do not think this is a very kind behavior.
Thank you.

--
components: Library (Lib), Tests, Windows, macOS
files: test.py
messages: 304729
nosy: Hiroki “h2” Horiuchi, ned.deily, paul.moore, ronaldoussoren, steve.dower, 
tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: `ImportError` is` raised` only when `python -m unittest discover` is 
given
type: behavior
versions: Python 2.7, Python 3.6
Added file: https://bugs.python.org/file47230/test.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30744] Local variable assignment is broken when combined with threads + tracing + closures

[Nick Coghlan]

Nick Coghlan  added the comment:

I rewrote the PEP based on the latest iteration of the design concept: 
https://github.com/python/peps/commit/9a8e590a523bdeed0598084a0782fb07fc15dfc0

(That initial commit has some minor errors in it that I've already fixed, so 
check the latest version in the repo if you spot anything else - if I'd thought 
it through properly, I would have submitted the update as a PR, so folks had a 
place to more easily make line-by-line comments)

I genuinely like this version, and I think it should be reasonably 
straightforward to implement (given that types.MappingProxyType already exists).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31840] `ImportError` is` raised` only when `python -m unittest discover` is given

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Run

python -m unittest discover --start-directory pkg --top-level-directory .

By default the top level directory is the same as the start directory (i.e. 
pkg).

--
nosy: +serhiy.storchaka
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com