[issue33611] Fatal Python error: Py_Initialize: unable to load the file system codec

[s]

New submission from s :

Fatal Python error: Py_Initialize: unable to load the file system codec
ModuleNotFoundError: No module named 'encodings'
###
Process:   Python [1889]
Path:  
/Library/Frameworks/Python.framework/Versions/3.6/Resources/Python.app/Contents/MacOS/Python
Identifier:Python
Version:   3.6.5 (3.6.5)
Code Type: X86-64 (Native)
Parent Process:bash [722]
Responsible:   Python [1889]
User ID:   501

Date/Time: 2018-05-22 23:52:40.997 -0700
OS Version:Mac OS X 10.13.4 (17E202)
Report Version:12
Anonymous UUID:F667AFCE-E0F9-BF0F-FB74-372904C44DB5


Time Awake Since Boot: 2000 seconds

System Integrity Protection: disabled

Crashed Thread:0  Dispatch queue: com.apple.main-thread

Exception Type:EXC_CRASH (SIGABRT)
Exception Codes:   0x, 0x
Exception Note:EXC_CORPSE_NOTIFY

Application Specific Information:
abort() called

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib  0x0001018eeb6e __pthread_kill + 10
1   libsystem_pthread.dylib 0x000101928080 pthread_kill + 333
2   libsystem_c.dylib   0x0001015c51ae abort + 127
3   org.python.python   0x000100877611 Py_FatalError + 65
4   org.python.python   0x0001008793c1 
_Py_InitializeEx_Private + 1281
5   org.python.python   0x000100896b40 Py_Main + 1360
6   org.python.python   0x00010dfe 0x1 + 3582
7   org.python.python   0x00010c34 0x1 + 3124

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x  rbx: 0x00010192e380  rcx: 0x7fff5bffef58  
rdx: 0x
  rdi: 0x0307  rsi: 0x0006  rbp: 0x7fff5bffef90  
rsp: 0x7fff5bffef58
   r8: 0x   r9: 0x0006  r10: 0x  
r11: 0x0202
  r12: 0x0307  r13: 0x0002  r14: 0x0006  
r15: 0x002d
  rip: 0x0001018eeb6e  rfl: 0x0202  cr2: 0x000101bd7020
  
Logical CPU: 0
Error Code:  0x02000148
Trap Number: 133


Binary Images:
   0x1 -0x10ff7 +org.python.python (3.6.5 - 3.6.5) 
 
/Library/Frameworks/Python.framework/Versions/3.6/Resources/Python.app/Contents/MacOS/Python
   0x13000 -0x10049dff7  com.apple.CoreFoundation (6.9 - 
1452.23) <358C547D-E227-3228-8218-62327F4605C8> 
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
   0x10075 -0x100977fef +org.python.python (3.6.5, [c] 
2001-2018 Python Software Foundation. - 3.6.5) 
 
/Library/Frameworks/Python.framework/Versions/3.6/Python
   0x100ac -0x100ac1ffb  libSystem.B.dylib (1252.50.4) 
 /usr/lib/libSystem.B.dylib
   0x100ac8000 -0x100ac9ff3  libDiagnosticMessagesClient.dylib 
(104) <9712E980-76EE-3A89-AEA6-DF4BAF5C0574> 
/usr/lib/libDiagnosticMessagesClient.dylib
   0x100acf000 -0x100cf6ffb  libicucore.A.dylib (59173.0.1) 
 /usr/lib/libicucore.A.dylib
   0x100dc8000 -0x1011b93b7  libobjc.A.dylib (723) 
 /usr/lib/libobjc.A.dylib
   0x10136a000 -0x10137cffb  libz.1.dylib (70) 
<48C67CFC-940D-3857-8DAD-857774605352> /usr/lib/libz.1.dylib
   0x101382000 -0x101386ff7  libcache.dylib (80) 
<092479CB-1008-3A83-BECF-E115F24D13C1> /usr/lib/system/libcache.dylib
   0x10138c000 -0x101396ff3  libcommonCrypto.dylib (60118.50.1) 
<029F5985-9B6E-3DCB-9B96-FD007678C6A7> /usr/lib/system/libcommonCrypto.dylib
   0x1013a4000 -0x1013abfff  libcompiler_rt.dylib (62) 
<968B8E3F-3681-3230-9D78-BB8732024F6E> /usr/lib/system/libcompiler_rt.dylib
   0x1013b9000 -0x1013c2ffb  libcopyfile.dylib (146.50.5) 
<3885083D-50D8-3EEC-B481-B2E605180D7F> /usr/lib/system/libcopyfile.dylib
   0x1013c9000 -0x10144efff  libcorecrypto.dylib (562.50.17) 
<67007279-24E1-3F30-802D-A55CD5C27946> /usr/lib/system/libcorecrypto.dylib
   0x10146b000 -0x1014a4ff7  libdispatch.dylib (913.50.12) 
<848EEE57-4235-3A61-9A52-C0097DD2AB5E> /usr/lib/system/libdispatch.dylib
   0x1014df000 -0x1014fcff7  libdyld.dylib (551.3) 
 /usr/lib/system/libdyld.dylib
   0x101517000 -0x101517ffb  libkeymgr.dylib (28) 
 /usr/lib/system/libkeymgr.dylib
   0x10151c000 -0x10151cff7  liblaunch.dylib (1205.50.76) 
<4D52BB64-C568-3A36-8935-2480427EE2A2> /usr/lib/system/liblaunch.dylib
   0x101523000 -0x101527ffb  libmacho.dylib (906) 
<1902A611-081A-3452-B11E-EBD1B166E831> /usr/lib/system/libmacho.dylib
   0x10152e000 -0x101530ff3  libquarantine.dylib (86) 
<26C0BA22-8F93-3A07-9A4E-C8D53D2CE42E> /usr/lib/system/libquarantine.dylib
   0x

[issue33565] strange tracemalloc results

[Alexander Mohr]

Alexander Mohr  added the comment:

INADA Naoki: Unfortunately you'll need to use credentials from a free AWS 
account: https://aws.amazon.com/free/.  Then create a credentials file in 
~/.aws/credentials: 
https://docs.aws.amazon.com/cli/latest/userguide/cli-config-files.html

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33612] Assertion failure in PyThreadState_Clear

[Serhiy Storchaka]

New submission from Serhiy Storchaka :

I got the following output when run tests in the huntrleaks mode.

$ ./python -We -m test -R 3:3 -x test_builtin -x test_urlparse
...
1:24:54 load avg: 2.48 [225/414] test_multiprocessing_fork
beginning 6 repetitions
123456
..python: Python/pystate.c:589: PyThreadState_Clear: Assertion 
`tstate->exc_info->previous_item == NULL' failed.
Fatal Python error: Aborted

Current thread 0x7fdd3e07e700 (most recent call first):
  File "/home/serhiy/py/cpython/Lib/multiprocessing/popen_fork.py", line 70 in 
_launch
  File "/home/serhiy/py/cpython/Lib/multiprocessing/popen_fork.py", line 20 in 
__init__
  File "/home/serhiy/py/cpython/Lib/multiprocessing/context.py", line 277 in 
_Popen
  File "/home/serhiy/py/cpython/Lib/multiprocessing/process.py", line 112 in 
start
  File "/home/serhiy/py/cpython/Lib/multiprocessing/pool.py", line 241 in 
_repopulate_pool
  File "/home/serhiy/py/cpython/Lib/multiprocessing/pool.py", line 248 in 
_maintain_pool
  File "/home/serhiy/py/cpython/Lib/multiprocessing/pool.py", line 412 in 
_handle_workers
  File "/home/serhiy/py/cpython/Lib/threading.py", line 865 in run
  File "/home/serhiy/py/cpython/Lib/threading.py", line 917 in _bootstrap_inner
  File "/home/serhiy/py/cpython/Lib/threading.py", line 885 in _bootstrap
..python: Python/pystate.c:589: PyThreadState_Clear: Assertion 
`tstate->exc_info->previous_item == NULL' failed.
Fatal Python error: Aborted

Current thread 0x7fdd6cb6c700 (most recent call first):
  File "/home/serhiy/py/cpython/Lib/multiprocessing/popen_fork.py", line 70 in 
_launch
  File "/home/serhiy/py/cpython/Lib/multiprocessing/popen_fork.py", line 20 in 
__init__
  File "/home/serhiy/py/cpython/Lib/multiprocessing/context.py", line 277 in 
_Popen
  File "/home/serhiy/py/cpython/Lib/multiprocessing/process.py", line 112 in 
start
  File "/home/serhiy/py/cpython/Lib/multiprocessing/pool.py", line 241 in 
_repopulate_pool
  File "/home/serhiy/py/cpython/Lib/multiprocessing/pool.py", line 248 in 
_maintain_pool
  File "/home/serhiy/py/cpython/Lib/multiprocessing/pool.py", line 412 in 
_handle_workers
  File "/home/serhiy/py/cpython/Lib/threading.py", line 865 in run
  File "/home/serhiy/py/cpython/Lib/threading.py", line 917 in _bootstrap_inner
  File "/home/serhiy/py/cpython/Lib/threading.py", line 885 in _bootstrap
..
1:34:07 load avg: 1.01 [226/414] test_multiprocessing_forkserver -- 
test_multiprocessing_fork passed in 9 min 13 sec
...

It happens after 1.5 hours of running tests. Running the 
test_multiprocessing_fork test only doesn't expose it. Likely there are already 
opened related issues, but the assertion failure in PyThreadState_Clear is new, 
it was added in 3.7. It may help to identify the problem.

--
messages: 317363
nosy: Mark.Shannon, davin, pitrou, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Assertion failure in PyThreadState_Clear
type: crash
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33612] Assertion failure in PyThreadState_Clear

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33610] IDLE: Make multiple improvements to CodeContext

[Terry J. Reedy]

Terry J. Reedy  added the comment:

11. Use read-only Text instead of Label for context. 
Text.index(f'@{e.x},{e.y}', where e is event, converts mouse click to line 
number.  That can be mapped to text line number for 8. above.  Text also enables

12. If context box is too short for all lines, add scrollbar.

13. When horizontal scrollbars are added to editor, scroll context along with 
text scroll.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19251] bitwise ops for bytes of equal length

[Matthias Gilch]

Change by Matthias Gilch :


--
nosy: +MGilch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33612] Assertion failure in PyThreadState_Clear

[Antoine Pitrou]

Change by Antoine Pitrou :


--
priority: normal -> critical

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26826] Expose new copy_file_range() syscall in os module.

[Giampaolo Rodola']

Giampaolo Rodola'  added the comment:

This is a great addition. I have a working patch adding sendfile() support for 
shutil.copyfileobj() which speeds it up by a factor of 1.3x on Linux. According 
to this 
https://lists.kernelnewbies.org/pipermail/kernelnewbies/2016-March/015999.html 
copy_file_range() may result in even better performances (but we may still want 
to use sendfile() for other UNIXes where file->file copy is supported - not 
sure which ones at this point).
As for the patch attached to this ticket, is there anything missing in order to 
push it forward?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33330] Better error handling in PyImport_Cleanup()

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +6696

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26826] Expose new copy_file_range() syscall in os module.

[STINNER Victor]

STINNER Victor  added the comment:

> As for the patch attached to this ticket, is there anything missing in order 
> to push it forward?

IMHO the next step would be to create a pull request on GitHub.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33330] Better error handling in PyImport_Cleanup()

[STINNER Victor]

STINNER Victor  added the comment:

I reopen the issue because I created the PR 7068 to propose further checks (in 
debug mode).

--
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33612] Assertion failure in PyThreadState_Clear

[STINNER Victor]

STINNER Victor  added the comment:

The assertion has been added by bpo-25612:

commit ae3087c6382011c47db82fea4d05f8bbf514265d
Author: Mark Shannon 
Date:   Sun Oct 22 22:41:51 2017 +0100

Move exc state to generator. Fixes bpo-25612 (#1773)

Move exception state information from frame objects to coroutine 
(generator/thread) object where it belongs.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33612] Assertion failure in PyThreadState_Clear

[STINNER Victor]

STINNER Victor  added the comment:

Failing assertion:

/* The stack of exception states should contain just this thread. */
assert(tstate->exc_info->previous_item == NULL);

"test_multiprocessing_fork passed in 9 min 13 sec"

Oh. A child process crashed on "self.pid = os.fork()" in 
"/Lib/multiprocessing/popen_fork.py", line 70 in _launch", but the test still 
pass.

os.fork() calls PyOS_AfterFork_Child() which starts by calling 
_PyGILState_Reinit() which ends with 
"_PyThreadState_DeleteExcept(current_tstate);".

_PyThreadState_DeleteExcept() creates a "garbage" list of thread states, and 
then call PyThreadState_Clear() on each thread. Finally, one call to 
PyThreadState_Clear() fails on the assertion.

I'm not sure that the assertion takes in account the case of specific case of 
fork().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33611] Fatal Python error: Py_Initialize: unable to load the file system codec

[INADA Naoki]

INADA Naoki  added the comment:

It seems not a bug of Python.
Maybe, your Python installation is broken, or you have wrong PYTHONPATH envvar.

--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33611] Fatal Python error: Py_Initialize: unable to load the file system codec

[STINNER Victor]

STINNER Victor  added the comment:

> Maybe, your Python installation is broken, or you have wrong PYTHONPATH 
> envvar.

We should maybe dump sys.path into stderr on the specific "unable to load the 
file system codec" error.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[STINNER Victor]

New submission from STINNER Victor :

test_semaphore_tracker_sigint() emits a warning. If the test is run with -W 
error, the test fails.

vstinner@apu$ ./python -m test test_multiprocessing_fork -v -m 
test_semaphore_tracker_sigint
(...)
test_semaphore_tracker_sigint 
(test.test_multiprocessing_fork.TestSemaphoreTracker) ... 
/home/vstinner/prog/python/master/Lib/multiprocessing/semaphore_tracker.py:55: 
UserWarning: semaphore_tracker: process died unexpectedly, relaunching.  Some 
semaphores might leak.
  warnings.warn('semaphore_tracker: process died unexpectedly, '
ok
(...)
Tests result: SUCCESS

vstinner@apu$ ./python  -Werror  -m test test_multiprocessing_fork -v -m 
test_semaphore_tracker_sigint
(...)
==
ERROR: test_semaphore_tracker_sigint 
(test.test_multiprocessing_fork.TestSemaphoreTracker)
--
Traceback (most recent call last):
  File "/home/vstinner/prog/python/master/Lib/test/_test_multiprocessing.py", 
line 4533, in test_semaphore_tracker_sigint
self.check_semaphore_tracker_death(signal.SIGINT, False)
  File "/home/vstinner/prog/python/master/Lib/test/_test_multiprocessing.py", 
line 4521, in check_semaphore_tracker_death
sem = ctx.Semaphore()
  File "/home/vstinner/prog/python/master/Lib/multiprocessing/context.py", line 
82, in Semaphore
return Semaphore(value, ctx=self.get_context())
  File "/home/vstinner/prog/python/master/Lib/multiprocessing/synchronize.py", 
line 127, in __init__
SemLock.__init__(self, SEMAPHORE, value, SEM_VALUE_MAX, ctx=ctx)
  File "/home/vstinner/prog/python/master/Lib/multiprocessing/synchronize.py", 
line 81, in __init__
register(self._semlock.name)
  File 
"/home/vstinner/prog/python/master/Lib/multiprocessing/semaphore_tracker.py", 
line 83, in register
self._send('REGISTER', name)
  File 
"/home/vstinner/prog/python/master/Lib/multiprocessing/semaphore_tracker.py", 
line 90, in _send
self.ensure_running()
  File 
"/home/vstinner/prog/python/master/Lib/multiprocessing/semaphore_tracker.py", 
line 55, in ensure_running
warnings.warn('semaphore_tracker: process died unexpectedly, '
UserWarning: semaphore_tracker: process died unexpectedly, relaunching.  Some 
semaphores might leak.
(...)
Tests result: FAILURE

--
components: Tests
messages: 317372
nosy: davin, pitrou, vstinner
priority: normal
severity: normal
status: open
title: test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W 
error
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33612] Assertion failure in PyThreadState_Clear

[STINNER Victor]

STINNER Victor  added the comment:

I created bpo-33613: "test_multiprocessing_fork: 
test_semaphore_tracker_sigint() fails with -W error". I don't think that it's 
directly related.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[STINNER Victor]

STINNER Victor  added the comment:

Note: I found this bug while working on bpo-33612.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[STINNER Victor]

STINNER Victor  added the comment:

Attached bug.py reproduces the bug with a daemonic thread but without fork():

$ ./python bug.py 
ls coredpython: Python/pystate.c:589: PyThreadState_Clear: Assertion 
`tstate->exc_info->previous_item == NULL' failed.
Aborted (core dumped)

(You might have to run the script a few times to get the crash, it's not 
deterministic.)

--
Added file: https://bugs.python.org/file47611/bug.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[STINNER Victor]

Change by STINNER Victor :


--
Removed message: https://bugs.python.org/msg317375

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[STINNER Victor]

STINNER Victor  added the comment:

Using bug.py, the assertion fails at:

python: Python/pystate.c:589: PyThreadState_Clear: Assertion 
`tstate->exc_info->previous_item == NULL' failed.
Aborted (core dumped)
(...)
vstinner@apu$ gdb ./python -c coredump-python.24130
GNU gdb (GDB) Fedora 8.0.1-36.fc27
(..)
Core was generated by `./python bug.py'.
Program terminated with signal SIGABRT, Aborted.

#0  0x7f5136075660 in raise () from /lib64/libc.so.6
[Current thread is 1 (Thread 0x7f5136f64040 (LWP 24130))]

(gdb) where
#0  0x7f5136075660 in raise () from /lib64/libc.so.6
#1  0x7f5136076c41 in abort () from /lib64/libc.so.6
#2  0x7f513606df7a in __assert_fail_base () from /lib64/libc.so.6
#3  0x7f513606dff2 in __assert_fail () from /lib64/libc.so.6
#4  0x00591dde in PyThreadState_Clear (tstate=0x1b7a430) at 
Python/pystate.c:589
#5  0x00590274 in PyInterpreterState_Clear (interp=0x1b23930) at 
Python/pystate.c:204
#6  0x0058cf4c in Py_FinalizeEx () at Python/pylifecycle.c:1153
#7  0x004257af in pymain_main (pymain=0x7ffe5f82f200) at 
Modules/main.c:2664
#8  0x00425915 in _Py_UnixMain (argc=2, argv=0x7ffe5f82f448) at 
Modules/main.c:2697
#9  0x0041f1e7 in main (argc=2, argv=0x7ffe5f82f448) at 
./Programs/python.c:15

(gdb) info threads
  Id   Target Id Frame 
  1Thread 0x7f5136f64040 (LWP 24130) 0x7f5136075660 in raise () from 
/lib64/libc.so.6
* 2Thread 0x7f512e8c6700 (LWP 24138) 0x7f5136b56be6 in 
pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  3Thread 0x7f512e0c5700 (LWP 24139) 0x7f5136b56be6 in 
pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  4Thread 0x7f512d8c4700 (LWP 24147) 0x7f5136b56be6 in 
pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  5Thread 0x7f512f0c7700 (LWP 24137) 0x7f5136b56be6 in 
pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0

(gdb) t 2
[Switching to thread 2 (Thread 0x7f512e8c6700 (LWP 24138))]
#0  0x7f5136b56be6 in pthread_cond_timedwait@@GLIBC_2.3.2 () from 
/lib64/libpthread.so.0

(gdb) where
#0  0x7f5136b56be6 in pthread_cond_timedwait@@GLIBC_2.3.2 () from 
/lib64/libpthread.so.0
#1  0x0053458a in PyCOND_TIMEDWAIT (cond=0xa07248 <_PyRuntime+1224>, 
mut=0xa07278 <_PyRuntime+1272>, us=5000) at Python/condvar.h:90
#2  0x005349a3 in take_gil (tstate=0x1b87980) at Python/ceval_gil.h:208

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33612] Assertion failure in PyThreadState_Clear

[STINNER Victor]

STINNER Victor  added the comment:

Attached bug.py reproduces the bug with a daemonic thread but without fork():

$ ./python bug.py 
ls coredpython: Python/pystate.c:589: PyThreadState_Clear: Assertion 
`tstate->exc_info->previous_item == NULL' failed.
Aborted (core dumped)

(You might have to run the script a few times to get the crash, it's not 
deterministic.)

vstinner@apu$ gdb ./python -c coredump-python.24130
GNU gdb (GDB) Fedora 8.0.1-36.fc27
(..)
Core was generated by `./python bug.py'.
Program terminated with signal SIGABRT, Aborted.

#0  0x7f5136075660 in raise () from /lib64/libc.so.6
[Current thread is 1 (Thread 0x7f5136f64040 (LWP 24130))]

(gdb) where
#0  0x7f5136075660 in raise () from /lib64/libc.so.6
#1  0x7f5136076c41 in abort () from /lib64/libc.so.6
#2  0x7f513606df7a in __assert_fail_base () from /lib64/libc.so.6
#3  0x7f513606dff2 in __assert_fail () from /lib64/libc.so.6
#4  0x00591dde in PyThreadState_Clear (tstate=0x1b7a430) at 
Python/pystate.c:589
#5  0x00590274 in PyInterpreterState_Clear (interp=0x1b23930) at 
Python/pystate.c:204
#6  0x0058cf4c in Py_FinalizeEx () at Python/pylifecycle.c:1153
#7  0x004257af in pymain_main (pymain=0x7ffe5f82f200) at 
Modules/main.c:2664
#8  0x00425915 in _Py_UnixMain (argc=2, argv=0x7ffe5f82f448) at 
Modules/main.c:2697
#9  0x0041f1e7 in main (argc=2, argv=0x7ffe5f82f448) at 
./Programs/python.c:15

(gdb) info threads
  Id   Target Id Frame 
  1Thread 0x7f5136f64040 (LWP 24130) 0x7f5136075660 in raise () from 
/lib64/libc.so.6
* 2Thread 0x7f512e8c6700 (LWP 24138) 0x7f5136b56be6 in 
pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  3Thread 0x7f512e0c5700 (LWP 24139) 0x7f5136b56be6 in 
pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  4Thread 0x7f512d8c4700 (LWP 24147) 0x7f5136b56be6 in 
pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  5Thread 0x7f512f0c7700 (LWP 24137) 0x7f5136b56be6 in 
pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0

(gdb) t 2
[Switching to thread 2 (Thread 0x7f512e8c6700 (LWP 24138))]
#0  0x7f5136b56be6 in pthread_cond_timedwait@@GLIBC_2.3.2 () from 
/lib64/libpthread.so.0

(gdb) where
#0  0x7f5136b56be6 in pthread_cond_timedwait@@GLIBC_2.3.2 () from 
/lib64/libpthread.so.0
#1  0x0053458a in PyCOND_TIMEDWAIT (cond=0xa07248 <_PyRuntime+1224>, 
mut=0xa07278 <_PyRuntime+1272>, us=5000) at Python/condvar.h:90
#2  0x005349a3 in take_gil (tstate=0x1b87980) at Python/ceval_gil.h:208

--
Added file: https://bugs.python.org/file47612/bug.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[STINNER Victor]

Change by STINNER Victor :


Removed file: https://bugs.python.org/file47611/bug.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[STINNER Victor]

Change by STINNER Victor :


--
Removed message: https://bugs.python.org/msg317376

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[STINNER Victor]

STINNER Victor  added the comment:

(Oops, I attached a file and added two comments to this issue, whereas I wanted 
to comment bpo-33612.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33612] Assertion failure in PyThreadState_Clear

[STINNER Victor]

STINNER Victor  added the comment:

I created PR 7069 to remove the assertion. With this change, attached bug.py 
doesn't crash anymore, but it still logs warnings in verbose mode:

vstinner@apu$ ./python -v bug.py 
(...)
PyThreadState_Clear: warning: thread still has a frame
PyThreadState_Clear: warning: thread still has a frame
PyThreadState_Clear: warning: thread still has a generator
PyThreadState_Clear: warning: thread still has a frame
PyThreadState_Clear: warning: thread still has a generator
PyThreadState_Clear: warning: thread still has a frame

The root issue is that Python doesn't join deamonic threads. But it's by design 
no?

I hate daemonic threads :-)

For the specific case of a fork(), it can be possible that a thread calls 
os.fork() while another thread is running a generator, no?

--

Since this issue seems to be a regression and I have a PR fixing it, I propose 
to mark this issue as a release blocker. Sorry Ned :-(

--
nosy: +ned.deily
priority: critical -> release blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33612] Assertion failure in PyThreadState_Clear

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +6697
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33613] test_multiprocessing_fork: test_semaphore_tracker_sigint() fails with -W error

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

See also some information about this in issue31463.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25612] nested try..excepts don't work correctly for generators

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +6698

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33330] Better error handling in PyImport_Cleanup()

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

See also issue33379 about the bug in subinterpreters exposed when add yet one 
PyErr_WriteUnraisable() in PyImport_Cleanup() (it was not added because of 
this).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31463] test_multiprocessing_fork hangs test_subprocess

[STINNER Victor]

STINNER Victor  added the comment:

> It looks like test_multiprocessing_fork doesn't clean up some of its 
> subprocesses and then test_subprocess hangs on waitpid(0) forever.

Oh strange. This issue was supposed to be fixed. I spent a lot of time to fix 
such bug, but also make sure that not test leak any child process.

"test_subprocess hangs on waitpid(0) forever" I recall a similar bug that I 
fixed recently.

I spent time on test.support.reap_children() and 
test.support.threading_cleanup() to make sure that tests don't leak threads no 
child processes. It seems like I missed some tests.

In Lib/test/libregrtest/runtest.py, I added:

def post_test_cleanup():
support.reap_children()

which is run after each test.

But currently, support.reap_children() calls os.waitpid(-1, os.WNOHANG) to 
check for child processes which already completed. It doesn't warn if there are 
child processes which are still running.

During my tests, I modified reap_children() to add a sleep(1).

Maybe on Linux we could use a cgroup to really check for all child processes?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33514] async and await as keywords not mentioned in What’s New In Python 3.7

[Miro Hrončok]

Miro Hrončok  added the comment:

This was fixed via issue32996

--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33603] Subprocess Thread handles grow with each call and aren't released [Windows]

[GranPrego]

GranPrego  added the comment:

I'm now pretty convinced that sounddevice 0.3.11 library is the culprit, which 
may in turn point to the portaudio library, or CFFI.

I make a call to sd.play() just before calling subprocess to run the dos cmd, 
the timing was such that process explorer would make it look like the leak was 
occurring during the subprocess call, but isolating the sd.play  shows that it 
causing the two additional Thread handles to be created and never released 
until the script ends (which could be 1-48 hours or more)

Another section of the code was using sd._terminate() and sd._initialize() to 
work around a buffersize problem with sounddevice and these calls also leak 
thread handles.

I've cut the program down as much as possible and the following now shows the 
problem without the call to subprocess.  You'll need to change the 
sd.default.device to an appropriate sound card.

Thanks for the quick responses.  If you're happy that this is the correct 
analysis of issue then perhaps it could be reclassified as a different 
component or I can get in touch with the sounddevice author.

Regards,
Joe

--
Added file: https://bugs.python.org/file47613/soundeviceLeaker.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33612] Assertion failure in PyThreadState_Clear

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

The assertion looked good to me when I reviewed this code, and I don't 
understand why it fails. Maybe it would be better to fix the case in which 
tstate->exc_info->previous_item != NULL before calling PyThreadState_Clear(). 
Otherwise we can leak some things.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33612] Assertion failure in PyThreadState_Clear

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Yes, simply removing the assertion feels more like a copout than an actual fix.
(perhaps it *is* the right fix to the issue, but it would be nice to find out 
why :-))

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33603] Subprocess Thread handles grow with each call and aren't released [Windows]

[Giampaolo Rodola']

Giampaolo Rodola'  added the comment:

FYI, there's psutil.Process().num_handles() which you can use to count handles 
before and after subprocess invocation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33612] Assertion failure in PyThreadState_Clear

[STINNER Victor]

STINNER Victor  added the comment:

> Yes, simply removing the assertion feels more like a copout than an actual 
> fix.

We are very close to the 3.7rc1, so I suggest to remove the assertion, just to 
get more time to fix the issue.

--

I agree that in a perfect world, Python should cleanup everything properly, but 
in the current world, daemon threads and fork are a mess full of corner cases 
like this one.

Python doesn't join daemonic threads at shutdown, so I don't see how Python 
could ensure that the state of the Python state of these daemonic threads is 
consistent (ready to be cleared). For fork, it seems like the issue is similar. 
A fork removes immediately all threads except of the thread which called fork. 
How would it be possible for Python to ensure that the Python state of the 
other destroyed threads is consistent?

tstate->exc_info is a new feature of Python 3.7, to fix bpo-25612. But I don't 
think that Python 3.6 is better to cleanup daemon threads at shutdown or to 
cleanup threads on fork. If you run the attached bug.py with python3.6 -v 
bug.py, I also see warnings:

PyThreadState_Clear: warning: thread still has a frame
PyThreadState_Clear: warning: thread still has a frame
PyThreadState_Clear: warning: thread still has a frame
PyThreadState_Clear: warning: thread still has a frame

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33612] Assertion failure in PyThreadState_Clear

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Le 23/05/2018 à 13:57, STINNER Victor a écrit :
> 
> I agree that in a perfect world, Python should cleanup everything properly, 
> but in the current world, daemon threads and fork are a mess full of corner 
> cases like this one.

For now, we don't know for sure that it's a cleanup issue.  What if it's
something else?

I agree that, if it's simply a cleanup issue with daemon threads, we can
remove the assertion.  But perhaps we should first diagnose the issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33547] Relative imports do not replace local variables

[Rolf Campbell]

Rolf Campbell  added the comment:

OK, while I understand what you are saying, that is NOT how absolute imports 
work.  I'll give an example:

./main.py:import func
./main.py:print(f"Value of func.func after import func:{func.func}")
./main.py:import func.func
./main.py:print(f"Value of func.func after import func.func:{func.func}")
./func/__init__.py:func = 1
./func/__init__.py:from . import func
./func/__init__.py:print(f"Value of func after from . import func:{func}")
./func/func.py:print("Module imported")

Here, the relative import inside __init__.py does NOT load the "func.py" module 
because there is already an object called "func".

But, the absolute "import func.func" does load "func.py" even though there is 
already a "func.func" object.

Are these supposed to work differently?  That seems strange to me.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27485] urllib.splitport -- is it official or not?

[Cheryl Sabella]

Change by Cheryl Sabella :


--
keywords: +patch
pull_requests: +6699
stage: resolved -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27485] urllib.splitport -- is it official or not?

[Cheryl Sabella]

Cheryl Sabella  added the comment:

Serhiy,

Thanks for finding this.  I've submitted a PR to fix the tests.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33614] Compilation of Python fails on AMD64 Windows8.1 Refleaks 3.x

[STINNER Victor]

Change by STINNER Victor :


--
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33614] Compilation of Python fails on AMD64 Windows8.1 Refleaks 3.x

[STINNER Victor]

New submission from STINNER Victor :

http://buildbot.python.org/all/#/builders/80/builds/244

   (Link target) -> 
 python3_d.def : error LNK2001: unresolved external symbol 
PyImport_GetModule 
[D:\buildarea\3.x.ware-win81-release.refleak\build\PCbuild\python3dll.vcxproj]
 python3_d.def : error LNK2001: unresolved external symbol Py_UTF8Mode 
[D:\buildarea\3.x.ware-win81-release.refleak\build\PCbuild\python3dll.vcxproj]
 
D:\buildarea\3.x.ware-win81-release.refleak\build\PCbuild\amd64\python3_d.lib : 
fatal error LNK1120: 2 unresolved externals 
[D:\buildarea\3.x.ware-win81-release.refleak\build\PCbuild\python3dll.vcxproj]

Maybe it's related to this recent change:

commit 4e29f566e8821c09d8274eadcdd355e8b1284b8b
Author: Serhiy Storchaka 
Date:   Tue May 22 20:59:42 2018 +0300

Add missed details of the C API introduced in 3.7. (GH-7047)

* Set the limited API version for PyImport_GetModule and PyOS_*Fork
  functions.
* Add PyImport_GetModule and Py_UTF8Mode in PC/python3.def.
* Add several functions in Doc/data/refcounts.dat.

The two missing symbols are defined by:

#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x0307
PyAPI_FUNC(PyObject *) PyImport_GetModule(PyObject *name);
#endif

and

#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x0307
PyAPI_DATA(int) Py_UTF8Mode;
#endif

They are not part of the stable ABI.

--
messages: 317392
nosy: vstinner
priority: normal
severity: normal
status: open
title: Compilation of Python fails on AMD64 Windows8.1 Refleaks 3.x

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33614] Compilation of Python fails on AMD64 Windows8.1 Refleaks 3.x

[STINNER Victor]

Change by STINNER Victor :


--
components: +Build, Windows
nosy: +paul.moore, serhiy.storchaka, steve.dower, tim.golden, zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33565] strange tracemalloc results

[INADA Naoki]

INADA Naoki  added the comment:

Alexander Mohr:

Thanks, I can reproduce.

I found ResourceWarning is happened when head_object.
re may be used for filtering warning.

You can use `-W always` to see the warnings.

I don't know why the leak happened yet.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33615] test__xxsubinterpreters crashed on x86 Gentoo Refleaks 3.x

[STINNER Victor]

New submission from STINNER Victor :

http://buildbot.python.org/all/#/builders/1/builds/232

(...)
3:28:16 load avg: 3.67 [401/416/3] test__xxsubinterpreters crashed (Exit code 
-6) -- running: test_asyncio (4631 sec)
python: Modules/gcmodule.c:277: visit_decref: Assertion `_PyGCHead_REFS(gc) != 
0' failed.
Fatal Python error: Aborted

Current thread 0xb748a700 (most recent call first):
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/case.py", 
line 615 in run
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/case.py", 
line 663 in __call__
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", 
line 122 in run
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", 
line 84 in __call__
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", 
line 122 in run
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", 
line 84 in __call__
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", 
line 122 in run
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", 
line 84 in __call__
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/support/__init__.py",
 line 1781 in run
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/support/__init__.py",
 line 1882 in _run_suite
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/support/__init__.py",
 line 1972 in run_unittest
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/runtest.py",
 line 175 in test_runner
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/runtest.py",
 line 176 in runtest_inner
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/runtest.py",
 line 140 in runtest
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/runtest_mp.py",
 line 67 in run_tests_slave
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/main.py",
 line 517 in _main
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/main.py",
 line 510 in main
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/main.py",
 line 585 in main
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/regrtest.py", 
line 46 in _main
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/regrtest.py", 
line 50 in 
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/runpy.py", 
line 85 in _run_code
  File "/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/runpy.py", 
line 193 in _run_module_as_main
(...)
Re-running test 'test__xxsubinterpreters' in verbose mode
(...)
test_recv_not_found (test.test__xxsubinterpreters.ChannelTests) ... ok
test_run_string_arg_resolved (test.test__xxsubinterpreters.ChannelTests) ... 
python: Modules/gcmodule.c:277: visit_decref: Assertion `_PyGCHead_REFS(gc) != 
0' failed.
Fatal Python error: Aborted
Current thread 0xb74b9700 (most recent call first):
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/case.py", 
line 615 in run
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/case.py", 
line 663 in __call__
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", 
line 122 in run
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", 
line 84 in __call__
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", 
line 122 in run
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", 
line 84 in __call__
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", 
line 122 in run
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/suite.py", 
line 84 in __call__
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/unittest/runner.py", 
line 176 in run
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/support/__init__.py",
 line 1882 in _run_suite
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/support/__init__.py",
 line 1972 in run_unittest
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/runtest.py",
 line 175 in test_runner
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/runtest.py",
 line 176 in runtest_inner
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/runtest.py",
 line 140 in runtest
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/main.py",
 line 291 in rerun_failed_tests
  File 
"/buildbot/buildarea/3.x.ware-gentoo-x86.refleak/build/Lib/test/libregrtest/main.py",
 

[issue33616] typing.NoReturn is undocumented

[Sebastian Rittau]

New submission from Sebastian Rittau :

This exists at least in Python 3.6.5's typing module. 
https://github.com/python/typing/issues/165 has background on why it was added.

--
assignee: docs@python
components: Documentation
messages: 317395
nosy: docs@python, srittau
priority: normal
severity: normal
status: open
title: typing.NoReturn is undocumented
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33565] strange tracemalloc results

[STINNER Victor]

STINNER Victor  added the comment:

> Thanks, I can reproduce.

Are you testing the master branch? I fixed a memory leak in the _warnings 
module for ignored ResourceWarning warnings.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33565] strange tracemalloc results

[INADA Naoki]

INADA Naoki  added the comment:

I confirmed the leak on 3.6.5, and I can't reproduce it on 3.7.0b4.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33614] Compilation of Python fails on AMD64 Windows8.1 Refleaks 3.x

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

My bad, I didn't test that change on Windows myself, and CI seems was broken at 
that moment.

But I have no ideas why the build is failed. These names are part of stable ABI 
since version 3.7. I don't see any difference between PyImport_GetModule and 
say PyInterpreterState_GetID.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31463] test_multiprocessing_fork hangs test_subprocess

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Let's take another look at the issue: why does test_subprocess need to know 
about all child processes, rather than those that were launched during 
test_subprocess?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33565] strange tracemalloc results

[INADA Naoki]

INADA Naoki  added the comment:

Hmm, GH-4587 is merged in 3.6 branch in last November, but 3.6.5 doesn't 
include it?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33565] strange tracemalloc results

[Antoine Pitrou]

Change by Antoine Pitrou :


--
nosy:  -pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33565] strange tracemalloc results

[INADA Naoki]

INADA Naoki  added the comment:

I was wrong.  It is not merged into 3.6 branch.
The pull request is just closed.

Now nothing strange in tracemalloc.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32710] test_asyncio leaked [4, 4, 3] memory blocks, sum=11 on AMD64 Windows8.1 Refleaks 3.x

[Giampaolo Rodola']

Change by Giampaolo Rodola' :


--
nosy: +giampaolo.rodola

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32622] Implement loop.sendfile

[Giampaolo Rodola']

Change by Giampaolo Rodola' :


--
nosy: +giampaolo.rodola

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23188] Provide a C helper function to chain raised (but not yet caught) exceptions

[Nick Coghlan]

Nick Coghlan  added the comment:

Also see 
https://github.com/python/cpython/blob/55edd0c185ad2d895b5d73e47d67049bc156b654/Objects/exceptions.c#L2713
 for the version we use in a few places to implicitly update the exception 
message, while keeping the exception type and state the same (and giving up and 
letting the exception through unchained if we can't work out how to do that in 
a reliable way).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33607] [subinterpreters] Explicitly track object ownership (and allocator).

[Nick Coghlan]

Nick Coghlan  added the comment:

Rather than tracking this per object, you could potentially track it per arena 
at the memory allocator level instead. Then if you really need the info (e.g. 
when running the debug allocator), you can check it in a reliable way, but in 
the normal case, you assume the associations are being managed correctly and 
avoid any significant bookkeeping overhead.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33608] [subinterpreters] Add a cross-interpreter-safe mechanism to indicate that an object may be destroyed.

[Nick Coghlan]

Nick Coghlan  added the comment:

Adding a low level callback based mechanism to ask another interpreter to do 
work seems like a good way to go to me.

As an example of why that might be needed, consider the case of sharing a 
buffer exporting object with another subinterpreter: when the memoryview in the 
subinterpreter is destroyed, it needs to request that the buffer view be 
released in the source interpreter that actually owns the original object.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33547] Relative imports do not replace local variables

[Nick Coghlan]

Nick Coghlan  added the comment:

Yes, while weird, that's expected behaviour.

Rather than being due to absolute vs relative imports, the difference arises 
from the fact that in "import pkg.module", the request is explicitly for a 
submodule, so the submodule import always happens, whereas if you write "from 
func import attr", the child module import is only attempted if "func.attr" 
fails to resolve after "func" is imported.

$ echo "print(__name__)" > pkg/__init__.py
$ echo "print(__name__)" > pkg/submodule.py

$ python3 -c "import pkg; pkg.submodule = 1; import pkg.submodule; 
print(pkg.submodule)"
pkg
pkg.submodule


$ python3 -c "import pkg; pkg.submodule = 1; from pkg import submodule; 
print(pkg.submodule)"
pkg
1

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33614] Compilation of Python fails on AMD64 Windows8.1 Refleaks 3.x

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

It succeeds after removing directories amd64 and obj in PCbuild. Seems they are 
left from old builds and the clean up script did nor remove them.

This may be a cause of other weird bugs on Windows.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30437] SSL_shutdown needs SSL_read() until SSL_ERROR_ZERO_RETURN

[Christian Heimes]

Christian Heimes  added the comment:

The session ticket issue in TLS 1.3 handshake will be fixed by upstream patch 
https://github.com/openssl/openssl/pull/6340.

We still need to drain the SSL socket to remove pending application data before 
the second SSL_shutdown() call, but it's no longer critical to get basic TLS 
1.3 support working.

Ned, I'm downgrading this issue from blocker to high.

--
priority: deferred blocker -> high

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33617] subprocess.Popen etc do not accept os.PathLike in passed sequence on Windows

[Kyle Altendorf]

New submission from Kyle Altendorf :

PS C:\Users\FSTAB\Desktop\g\20\grid-tied> py -3.6 -c "import os, pathlib, 
subprocess; subprocess.run([pathlib.Path()])"
Traceback (most recent call last):
  File "", line 1, in 
  File "C:\Program Files\Python36\lib\subprocess.py", line 403, in run
with Popen(*popenargs, **kwargs) as process:
  File "C:\Program Files\Python36\lib\subprocess.py", line 707, in __init__
restore_signals, start_new_session)
  File "C:\Program Files\Python36\lib\subprocess.py", line 964, in 
_execute_child
args = list2cmdline(args)
  File "C:\Program Files\Python36\lib\subprocess.py", line 461, in list2cmdline
needquote = (" " in arg) or ("\t" in arg) or not arg
TypeError: argument of type 'WindowsPath' is not iterable


PR to follow soon.

--
messages: 317408
nosy: altendky
priority: normal
severity: normal
status: open
title: subprocess.Popen etc do not accept os.PathLike in passed sequence on 
Windows

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33617] subprocess.Popen etc do not accept os.PathLike in passed sequence on Windows

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> subprocess._execute_child doesn't accept a single PathLike 
argument for args

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33617] subprocess.Popen etc do not accept os.PathLike in passed sequence on Windows

[STINNER Victor]

STINNER Victor  added the comment:

That's a feature request for Python 3.8 ;-)

> PR to follow soon.

Cool!

--
components: +Library (Lib), Windows
nosy: +paul.moore, steve.dower, tim.golden, vstinner, zach.ware
resolution: duplicate -> 
stage: resolved -> 
status: closed -> open
superseder: subprocess._execute_child doesn't accept a single PathLike argument 
for args -> 
type:  -> enhancement
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33565] strange tracemalloc results

[STINNER Victor]

STINNER Victor  added the comment:

> Hmm, GH-4587 is merged in 3.6 branch in last November, but 3.6.5 doesn't 
> include it?

We decided to not fix the memory leak in Python 3.6:
https://github.com/python/cpython/pull/4587#issuecomment-347659163

"""
This change makes warning.warn() 1.4x slower on Python 3.6 for the ignore 
action: https://bugs.python.org/issue27535#msg307165

I don't think that the memory leak issue is important enough to justify such 
slowdown in a stable branch. So I close my PR.
"""

So yes, this issue is a bug in Python: it's bpo-27535. It's a bug in the 
_warnings module. tracemalloc only tells you the truth, it's just that it's 
hard to understand the output :-)

The bug occurs when you get ResourceWarning warnings. Such warning means that 
the code has a bug: the bug should be fixed. If you fix the ResourceWarning, 
the bug should disappear.

As Naoki INADA wrote: use python3 -Walways (I prefer python3 -Wdefault, but 
it's almost the same ;-)) to see such warnings.

Python 3.7 comes with a new -X dev option which enables these warnings but also 
other runtime checks to ease development.

I suggest to close this issue as a duplicate of bpo-27535.

Thanks Alexander Mohr for your bug report, and good luck to fix your warning 
;-) If you are motivated, you can propose changes to the tracemalloc 
documentation!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33617] subprocess.Popen etc do not accept os.PathLike in passed sequence on Windows

[Kyle Altendorf]

Change by Kyle Altendorf :


--
keywords: +patch
pull_requests: +6700
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33617] subprocess.Popen etc do not accept os.PathLike in passed sequence on Windows

[Kyle Altendorf]

Kyle Altendorf  added the comment:

I totally failed to fill out the metadata, sorry.  I would personally consider 
this more of a bugfix than a feature enhancement, but I don't know how that's 
decided exactly.  It's also quite small.  There are a couple other open issues 
related to full os.PathLike protocol support.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27485] urllib.splitport -- is it official or not?

[Guido van Rossum]

Change by Guido van Rossum :


--
nosy:  -gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33618] Support TLS 1.3

[Christian Heimes]

New submission from Christian Heimes :

Epic for various TLS 1.3 related tasks and improvements

TLS 1.3 requires some changes to the SSL module and tests. The TLS 1.3 
handshake behaves slightly differently, which causes some tests to fail. The 
new handshake and deferred non-application data also affect applications. 

* TLS 1.3 cipher suites are now set by SSL_CTX_set_ciphersuites(), while SSL to 
TLS 1.2 cipher suites are still set by SSL_CTX_set_cipher_list(). Therefore 
SSLContext.set_ciphers() no longer fails with invalid cipher suites, because 
TLS 1.3 are still available. TLS 1.3 cipher suites also cannot be changed or 
disabled by SSLContext.set_ciphers().

* TLS client cert authentication occurs after SSL_do_handshake() has finished. 
SSLSocket.connect() / handshake no longer fail, when the server requests a 
client cert or the available client cert is invalid. The actual authentication 
occurs when the client performs the first SSL_read() / SSL_write().

* Session tickets are exchanged after the handshake, too. On the client side, 
the session ticket is only available after the first SSL_read() or other 
operations that perform a read(). The session ticket class and code no longer 
works with TLS 1.3.

* TLS 1.3 sends two session tickets instead of one.

* Server-side handshake can fail with ConnectionResetError or BrokenPipeError, 
when the client closes the fd while the server is still send non-application 
data like new session ticket or client cert request.

* Client-side unwrap() / shutdown used to fail when a session ticket was stuck 
on the wire. This problem will be fixed by OpenSSL 1.1.1-pre7, see 
https://github.com/openssl/openssl/pull/6340 


I'll add a TLS 1.3 section to the ssl module documentation. TLS 1.3 will be a 
tech-preview and not production-ready until at least OpenSSL 1.1.1-final and 
Python 3.7.1. Ned, Benjamin, are you OK with that?

--
assignee: christian.heimes
messages: 317413
nosy: alex, benjamin.peterson, christian.heimes, dstufft, janssen, ned.deily
priority: high
severity: normal
stage: needs patch
status: open
title: Support TLS 1.3
type: enhancement
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33618] Support TLS 1.3

[Christian Heimes]

Christian Heimes  added the comment:

More:

* We also need a new API to request TLS client cert authentication *after* some 
application data was requested. The use case is e.g. HTTP web server. A client 
sends a GET request and then the server gets to decide if the route requires 
authentication or not.

* Renegotiation is no longer available (good). TLS 1.3 has a new re-keying 
mechanism to establish a new master key.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33616] typing.NoReturn is undocumented

[Ivan Levkivskyi]

Change by Ivan Levkivskyi :


--
nosy: +levkivskyi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33569] dataclasses InitVar does not maintain any type info

[Ivan Levkivskyi]

Change by Ivan Levkivskyi :


--
nosy: +levkivskyi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33615] test__xxsubinterpreters crashed on x86 Gentoo Refleaks 3.x

[Eric Snow]

Eric Snow  added the comment:

I'll take a look.  Thanks!

--
assignee:  -> eric.snow

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33421] Missing documentation for typing.AsyncContextManager

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:

This is now fixed.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33618] Support TLS 1.3

[Chih-Hsuan Yen]

Change by Chih-Hsuan Yen :


--
nosy: +yan12125

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29427] Option to skip padding for base64 urlsafe encoding/decoding

[Guillaume]

Change by Guillaume :


--
keywords: +patch
pull_requests: +6702
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33612] Assertion failure in PyThreadState_Clear

[STINNER Victor]

STINNER Victor  added the comment:


New changeset b6dccf54fd3bac9c87348d96f9d6b571608c15bc by Victor Stinner in 
branch 'master':
bpo-33612: Remove PyThreadState_Clear() assertion (#7069)
https://github.com/python/cpython/commit/b6dccf54fd3bac9c87348d96f9d6b571608c15bc


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33612] Assertion failure in PyThreadState_Clear

[miss-islington]

Change by miss-islington :


--
pull_requests: +6704

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25612] nested try..excepts don't work correctly for generators

[miss-islington]

Change by miss-islington :


--
pull_requests: +6705

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32436] Implement PEP 567

[Yury Selivanov]

Change by Yury Selivanov :


--
pull_requests: +6703

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25612] nested try..excepts don't work correctly for generators

[STINNER Victor]

STINNER Victor  added the comment:


New changeset b6dccf54fd3bac9c87348d96f9d6b571608c15bc by Victor Stinner in 
branch 'master':
bpo-33612: Remove PyThreadState_Clear() assertion (#7069)
https://github.com/python/cpython/commit/b6dccf54fd3bac9c87348d96f9d6b571608c15bc


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33565] strange tracemalloc results

[Alexander Mohr]

Alexander Mohr  added the comment:

I'll try with that fix and see if the leak is fixed, is the reasoning that if 
the warning happens after the try/except scope and that's why the callstack has 
it?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33614] Compilation of Python fails on AMD64 Windows8.1 Refleaks 3.x

[Steve Dower]

Steve Dower  added the comment:

I guess that means there is an invalid date comparison going on to detect when 
the file has changed. This file doesn't change often, so it's easy to miss :)

All the critical builds should forcibly rebuild it, so this isn't urgent, but 
I'll take a look next time I get a chance to work on the build files.

--
assignee:  -> steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27535] Ignored ResourceWarning warnings leak memory in warnings registries

[Alexander Mohr]

Alexander Mohr  added the comment:

not fixing this means that 3.6 slowly leaks for many people in prod.  It's not 
often possible to fix all the warnings on large dynamic applications, I highly 
suggest finding a way to get this into 3.6.  I bet there are a lot of 
frustrated people out there who don't know why their applications slowly leak.

--
nosy: +thehesiod

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33462] reversible dict

[Rémi Lapeyre]

Rémi Lapeyre  added the comment:

I updated the pull request, now reversed work on the dict and dict views:

➜  cpython git:(master) ./python.exe 
Python 3.8.0a0 (heads/master-dirty:128576b88c, May 23 2018, 16:33:46) 
[Clang 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> d = dict(a=1, b=2)
>>> list(reversed(d))
['b', 'a']
>>> list(reversed(d.keys()))
['b', 'a']
>>> list(reversed(d.values()))
[2, 1]
>>> list(reversed(d.items()))
[('b', 2), ('a', 1)]

reversed on dict and dict views is not symmetric and applying twice will  raise 
TypeError which is also the behavior on list. I'm not sure why this behavior 
has been chosen but it's coherent.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33609] Document that dicts preserve insertion order

[Cheryl Sabella]

Cheryl Sabella  added the comment:

As reference, issue32337 made some changes documenting that the dicts preserve 
insertion order.  Also, issue33218 was marked as being superseded by #32337.

--
nosy: +cheryl.sabella

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33619] libffi detection via pkg-config is broken

[Chih-Hsuan Yen]

New submission from Chih-Hsuan Yen :

The cause is that ./configure contains unexpanded m4 macro PKG_PROG_PKG_CONFIG:

https://github.com/python/cpython/blob/3055c94/configure#L9909

For unknown reasons, after GH-6850 the PKG_PROG_PKG_CONFIG macro is removed 
from aclocal.m4, and thus the macro is kept unexpanded in ./configure. As a 
result, $PKG_CONFIG is not correctly set, libffi can't be detected and thus 
_ctypes fails to build.

Run autoreconf again fixes the issue on my machine. I have pkg-config 0.29.2-1, 
autoconf 2.69-4 and automake 1.15.1-1 on Arch Linux.

cc the author of GH-6850

--
components: Build
messages: 317424
nosy: benjamin.peterson, yan12125
priority: normal
severity: normal
status: open
title: libffi detection via pkg-config is broken
type: compile error
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33620] requests.Session doesn't properly handle closed keep-alive sessions

[Jonathan Lynch]

New submission from Jonathan Lynch :

When a server reaps a keep-alive session it sends a FIN packet to the client. 
Normally, requests handles this fine and rebuilds the session on the next 
request. However, there is an edge case involving network latency that is not 
properly handled:

If python sends a request at roughly the same time as the server closes the 
session, then the server will send a RST (as the session is closed). Python 
receives this RST on what it thought was a valid session and throws an error:

requests.exceptions.ConnectionError: ('Connection aborted.', 
RemoteDisconnected('Remote end closed connection without response',))

The reason I consider this a bug is because python received the FIN packet 
before it received the RST. As a result, it shouldn't be surprised when the 
connection is subsequently aborted. It is an edge case, but the client has 
enough information available to it that it could have handled it correctly.

The workaround is to set max_retries on the Session via an HTTPAdaptor, but I 
believe the correct behavior when the FIN is received is to rebuild the session 
and re-send any requests that were in-flight (rather than throwing an error). 
Requests correctly handles the FIN packet if there are no in-flight requests, 
but if there are in-flight requests it ignores it and instead throws an error.

--
components: Library (Lib)
messages: 317425
nosy: Jonathan Lynch
priority: normal
severity: normal
status: open
title: requests.Session doesn't properly handle closed keep-alive sessions
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33614] Compilation of Python fails on AMD64 Windows8.1 Refleaks 3.x

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

If "PCbuild\build.bat -p x64" builds Python, "PCbuild\clean.bat -p x64" removes 
all files in PCbuild\amd64 but python3stub.exp and python3stub.lib.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33620] requests.Session doesn't properly handle closed keep-alive sessions

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

requests is a third-party package. Can you reproduce your issue with the 
standard Python library?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33565] strange tracemalloc results

[Alexander Mohr]

Alexander Mohr  added the comment:

Ok I've verified that the patch does indeed fix the leak detected.  Thank you 
very much INADA for knowing that there was a leak in the warnings module, I 
would have never guessed, especially given the tracemalloc stack given.  Had it 
showed a callstack where the warning was created it would have made a lot more 
sense.

I agree this can be closed, however can the leak fix PLEASE be put into 3.6 
(any any other version that needs it)?  Who cares if warnings are 1.4x slower 
with the fix?  Are you going to rationally tell me that keeping warnings fast 
is more important than fixing leaks? In most applications there should be no 
warnings so it doesn't really matter. This particular leak was causing our 
application to fail after running for a few days which makes it unusable in 
production.  It's caused me a lot of days wasted in investigation.  If speed 
was really a problem that would have been a much worthier thing to spend time 
on than finding leaks.

leaks should be highest priority, then speed.  No rational developer would have 
complained that warnings got slower, that's when you fix warnings, not because 
of leaks! :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33619] libffi detection via pkg-config is broken

[Chih-Hsuan Yen]

Change by Chih-Hsuan Yen :


--
keywords: +patch
pull_requests: +6706
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32436] Implement PEP 567

[Yury Selivanov]

Yury Selivanov  added the comment:


New changeset 28b9178023a445b1da2694774c265cd4b7a244ec by Yury Selivanov in 
branch 'master':
bpo-32436: Document PEP 567 changes to asyncio. (GH-7073)
https://github.com/python/cpython/commit/28b9178023a445b1da2694774c265cd4b7a244ec


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32436] Implement PEP 567

[miss-islington]

Change by miss-islington :


--
pull_requests: +6707

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33621] repr(threading._DummyThread) always fails.

[Fabio Zadrozny]

New submission from Fabio Zadrozny :

Doing the following throws an exception:

import threading
repr(threading._DummyThread())


Or, in a more contrived example (I actually had this in a QThread, so, 
reproducing using getting the current_thread using a thread created with the 
_thread module):

import threading
import traceback
finished = threading.Event()
worked = []
def method():
try:
repr(threading.current_thread())
worked.append(True)
except:
traceback.print_exc()
worked.append(False)
finally:
finished.set()

import _thread
_thread.start_new_thread(method, ())

finished.wait()
assert worked[0]

--
components: Library (Lib)
messages: 317430
nosy: fabioz
priority: normal
severity: normal
status: open
title: repr(threading._DummyThread) always fails.
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33622] Fix and improve errors handling in the garbage collector

[Serhiy Storchaka]

New submission from Serhiy Storchaka :

There are following bugs in the garbage collector.

* If the garbage collector fails to add an object with __del__ or referenced by 
an object with __del__ to gc.garbage (in handle_legacy_finalizers()), it leaks 
it and other not added objects with __del__ and referenced by them. They become 
no longer accessible by the garbage collector.

* PyGC_Collect() is not documented, but it is a public C API. And it can be 
called by user with an exception set. PyGC_Collect() then can either crash or 
just silent the exception. It is safer to safe possible exception and restore 
it after collecting.

* A pointer to released member can be used (compared with NULL) in 
invoke_gc_callback(). This is an undefined behavior.

--
components: Interpreter Core
messages: 317431
nosy: pitrou, serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Fix and improve errors handling in the garbage collector
type: behavior
versions: Python 2.7, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33620] requests.Session doesn't properly handle closed keep-alive sessions

[Jonathan Lynch]

Jonathan Lynch  added the comment:

Ah, I'm sorry! I'll open the report over there, closing this.

--
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33622] Fix and improve errors handling in the garbage collector

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +6708
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32436] Implement PEP 567

[miss-islington]

miss-islington  added the comment:


New changeset 2fc443c469fb15033b6b96acd727e2e7cc147adc by Miss Islington (bot) 
in branch '3.7':
bpo-32436: Document PEP 567 changes to asyncio. (GH-7073)
https://github.com/python/cpython/commit/2fc443c469fb15033b6b96acd727e2e7cc147adc


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com