[issue38588] Use-after-free in dict/list

[Inada Naoki]

Inada Naoki  added the comment:

If we can not add INCREF and DECREF in the PyObject_RichCompare, we can add v 
== w check in the caller side.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39162] setup.py not picking up tkinter headers

[anthony shaw]

Change by anthony shaw :


--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39162] setup.py not picking up tkinter headers

[anthony shaw]

New submission from anthony shaw :

./configure && make -j4 is returning:

Failed to build these modules:
_tkinter   

I'm running macOS 10.15.2, with the SDK installed using `xcode-select 
--install` (no funny business)

/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/Tk.framework/

xcrun --show-sdk-path
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk

If I debug setup.py, I can see it's eagerly returning the wrong path (/System)

On a REPL, if I do:
>>> import setup
>>> setup.macosx_sdk_root()
'/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk'

So the path is correct, but I can see in setup.py that detect_tkinter_darwin() 
will break inside the loop if 
/System/Library/Frameworks/Tk.framework/Versions/Current  exists (which it 
does), but it doesn't contain the headers, it contains 3 directories:

Resources  Tk _CodeSignature

I think detect_tkinter_darwin should be updated so that framework_dirs scans 
macosx_sdk_root first, but I don't know what other scenarios this might break.

I'd be happy to submit a patch for my scenario but it looks like this whole 
function needs tests so each platform change and scenario as they come up. 
There's loads of other issues on BPO related.

--
components: Build
messages: 359026
nosy: anthonypjshaw
priority: normal
severity: normal
status: open
title: setup.py not picking up tkinter headers
type: compile error

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39162] setup.py not picking up tkinter headers

[anthony shaw]

anthony shaw  added the comment:

I also can't see how this condition would ever match, since macosx_sdk_root() 
is not in the list being iterated upon,
https://github.com/python/cpython/blob/master/setup.py#L1774

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39163] Perfect Exchange Migration tool

[georgepaul bj]

New submission from georgepaul bj :

EdbMails Exchange Migration tool is the perfect tool which migrates your entire 
mailbox items  like mails, calendars, contacts etc. without any data loss. and 
it supports  Exchange versions 2003,  2007, 2010, 2013, 2016 and 2019.  The 
migration will be incremental .ie. no duplicate items during consecutive 
migration on the same system.

Exchange Migration


The tool provides risk less migration which helps to migrate direct Exchange to 
Exchange server and Exchange to Office 365 Migration. It also handles public 
folders and Archive mailboxes migration.

The software supports cut over , staged and hybrid migration.  It easily 
performs cross forest and cross domain migration.
To know more : 
https://www.edbmails.com/pages/exchange-server-migration-tool.html

--
files: exchange-migration.jpg
messages: 359028
nosy: georgepaul123
priority: normal
severity: normal
status: open
title: Perfect Exchange Migration tool
type: performance
Added file: https://bugs.python.org/file48809/exchange-migration.jpg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39164] PyErr_GetExcInfo does not allow to retrieve for an arbitrary thread

[Julien Danjou]

Julien Danjou  added the comment:

Relevant python-dev thread: 
https://mail.python.org/archives/list/python-...@python.org/thread/QVMFP76R35SXUIM2WPPVPV5XCVMKPDEB/#TLP6MWNML4PRKFFGXHCKNEUMN6UIQ4MT

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39164] PyErr_GetExcInfo does not allow to retrieve for an arbitrary thread

[Julien Danjou]

New submission from Julien Danjou :

PyErr_GetExcInfo does not allow to retrieve exception information for an 
arbitrary thread. As it calls `_PyThreadState_GET` itself, it's impossible to 
get exception information for a different thread.

--
components: C API
messages: 359029
nosy: jd
priority: normal
severity: normal
status: open
title: PyErr_GetExcInfo does not allow to retrieve for an arbitrary thread
type: enhancement
versions: Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39029] TestMaildir.test_clean fails randomly under parallel tests

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

> Should not parallel tests be ran in different directories?

yes, the test uses support.TESTFN which has the value of 
"{}_{}_tmp".format(TESTFN, os.getpid()) in the setUp [0] to create tempdir. 
Under parallel tests the pid value of TESTFN is initialised with a pid once and 
used always in the setup calls. So setup acts under parallel tests with same 
test folder. The tempdir can be dynamically generated but I guess this pattern 
to use support.TESTFN is common and there could be other flaky tests.

# create tempdir at setup each time and clean it up.

diff --git Lib/test/test_mailbox.py Lib/test/test_mailbox.py
index 36a265390e..aa8ae9ae20 100644
--- Lib/test/test_mailbox.py
+++ Lib/test/test_mailbox.py
@@ -51,13 +51,15 @@ class TestMailbox(TestBase):
 _template = 'From: foo\n\n%s\n'

 def setUp(self):
-self._path = support.TESTFN
+self._tempdir = tempfile.TemporaryDirectory()
+self._path = self._tempdir.name
 self._delete_recursively(self._path)
 self._box = self._factory(self._path)

 def tearDown(self):
 self._box.close()
 self._delete_recursively(self._path)
+self._tempdir.cleanup()

 def test_add(self):
 # Add copies of a sample message

[0] 
https://github.com/python/cpython/blob/88dce26da6bc4838092128d9a6f1c98bf48b7c90/Lib/test/test_mailbox.py#L54

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39164] PyErr_GetExcInfo does not allow to retrieve for an arbitrary thread

[Julien Danjou]

Change by Julien Danjou :


--
keywords: +patch
pull_requests: +17188
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/17752

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39156] Break up COMPARE_OP into logically distinct operations.

[Mark Shannon]

Mark Shannon  added the comment:

Moving work from the interpreter to the compiler is always a good idea.

Performance: The compiler is run once per code unit, the interpreter thousands 
or millions of times.

The compiler is easier to test. Just match the expected bytecode with the 
actual bytecode. 
The output can be sanity checked by visual inspection.


Although I expect a performance boost, I think this is a worthwhile improvement 
whether or not it helps 
performance as it makes the instructions better focused.


Pablo, currently there are 117 opcodes, increasing that to 120 is not a problem.
Also there is no reason why we are limited to 256 opcodes in the long term.
Plus, I'm 4 opcodes in credit, thanks to https://bugs.python.org/issue33387 :)

Raymond, regarding the performance of COMPARE_OP, it is not just branch 
prediction that matters. With this change, the number of (hardware) 
instructions executed is always reduced, even if branch prediction is no better.

Serhiy, the benefit of having a special opcode for exception matching is not 
really to speed up exception matching, but to avoid slowing down other tests.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38967] Improve error message in enum for member name surrounded by underscore.

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +ethan.furman

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39162] setup.py not picking up tkinter headers

[anthony shaw]

Change by anthony shaw :


--
keywords: +patch
pull_requests: +17189
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/17753

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33777] dummy_threading: .is_alive method returns True after execution has completed

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

FWIW, dummy_threading and _dummy_thread were removed with 
8bf08ee45b7c2341f0d0175b91892843a37c23da in Python 3.9.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39162] setup.py not picking up tkinter headers

[anthony shaw]

anthony shaw  added the comment:

This comes down to a logic bug in detect_tkinter_darwin(), shown in 
https://github.com/python/cpython/pull/17753/files#r361956102

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39156] Break up COMPARE_OP into logically distinct operations.

[Mark Shannon]

Change by Mark Shannon :


--
keywords: +patch
pull_requests: +17190
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/17754

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39163] Perfect Exchange Migration tool

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


Removed file: https://bugs.python.org/file48809/exchange-migration.jpg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39163] Perfect Exchange Migration tool

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
Removed message: https://bugs.python.org/msg359028

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39163] spam

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
resolution:  -> wont fix
stage:  -> resolved
status: open -> closed
title: Perfect Exchange Migration tool -> spam
type: performance -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39037] Fix the trial order of the __exit__ and __enter__ methods in the with statement documentation

[Géry]

Change by Géry :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33777] dummy_threading: .is_alive method returns True after execution has completed

[Inada Naoki]

Change by Inada Naoki :


--
resolution:  -> wont fix
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33387] Simplify bytecodes for try-finally, try-except and with blocks.

[Mark Shannon]

Change by Mark Shannon :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39114] Python 3.9.0a2 changed how finally/return is traced

[Mark Shannon]

Mark Shannon  added the comment:

Ned, I think this is fixed. Feel free to re-open if you disagree.

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34790] Deprecate passing coroutine objects to asyncio.wait()

[Andrew Svetlov]

Andrew Svetlov  added the comment:


New changeset 89aa7f0ede1a11c020e83f24394593c577a61509 by Andrew Svetlov (Kyle 
Stanley) in branch 'master':
bpo-34790: Implement deprecation of passing coroutines to asyncio.wait() 
(GH-16977)
https://github.com/python/cpython/commit/89aa7f0ede1a11c020e83f24394593c577a61509


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39162] setup.py not picking up tkinter headers

[Ned Deily]

Change by Ned Deily :


--
assignee:  -> ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39114] Python 3.9.0a2 changed how finally/return is traced

[Ned Batchelder]

Ned Batchelder  added the comment:

Thanks, that fixes the original case in this issue.

Here is another problem which seems related enough to append here instead of 
opening a new issue:

--- 8< 
import linecache, sys

def trace(frame, event, arg):
lineno = frame.f_lineno
print("{} {}: {}".format(event[:4], lineno, linecache.getline(__file__, 
lineno).rstrip()))
return trace

print(sys.version)
sys.settrace(trace)

def b(exc):
print("exc is {!r}".format(exc))
try:
raise Exception(exc)
except Exception as e:
if exc != 'expected':
raise   # line 17
q = "removing this line changes the behavior"
r = "is this executed?" # line 19
return

b('expected')
try:
b('unexpected')
except:
pass
--- 8< 

With your fix, it reports that the raise on line 17 is followed by line 19, 
which can't be right:

3.9.0a2+ (heads/pr/17737:f2545467fa, Dec 29 2019, 15:38:57)
[Clang 10.0.0 (clang-1000.10.44.4)]
call 11: def b(exc):
line 12: print("exc is {!r}".format(exc))
exc is 'expected'
line 13: try:
line 14: raise Exception(exc)
exce 14: raise Exception(exc)
line 15: except Exception as e:
line 16: if exc != 'expected':
line 18: q = "removing this line changes the behavior"
line 19: r = "is this executed?"
line 20: return
retu 20: return
call 11: def b(exc):
line 12: print("exc is {!r}".format(exc))
exc is 'unexpected'
line 13: try:
line 14: raise Exception(exc)
exce 14: raise Exception(exc)
line 15: except Exception as e:
line 16: if exc != 'expected':
line 17: raise
line 19: r = "is this executed?"
retu 19: r = "is this executed?"


Python 3.8 reported this:

3.8.1 (default, Dec 19 2019, 08:38:38)
[Clang 10.0.0 (clang-1000.10.44.4)]
call 11: def b(exc):
line 12: print("exc is {!r}".format(exc))
exc is 'expected'
line 13: try:
line 14: raise Exception(exc)
exce 14: raise Exception(exc)
line 15: except Exception as e:
line 16: if exc != 'expected':
line 18: q = "removing this line changes the behavior"
line 19: r = "is this executed?"
line 20: return
retu 20: return
call 11: def b(exc):
line 12: print("exc is {!r}".format(exc))
exc is 'unexpected'
line 13: try:
line 14: raise Exception(exc)
exce 14: raise Exception(exc)
line 15: except Exception as e:
line 16: if exc != 'expected':
line 17: raise
retu 17: raise

--
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39114] Python 3.9.0a2 changed how finally/return is traced

[Ned Batchelder]

Ned Batchelder  added the comment:

BTW: this is not a regression in your fix. 3.9a2 behaved this way also.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39165] Completeness and symmetry in RE, avoid `findall(...)[0]`

[Juancarlo Añez]

New submission from Juancarlo Añez :

The problematic `findall(...)[0]` is a common anti-pattern in Python programs. 
The reason is lack of symmetry and completeness in the `re` module.

The original proposal in `python-ideas` was to add `re.findfirst(pattern, 
string, flags=0, default=_mark)` with more or less the semantics of 
`next(findall(pattern, string, flags=flags), default=default)`. 

The referenced PR adds `findalliter(pattern, string, flags=0)` with the value 
semantics of `findall()` over a generator, implements `findall()` as `return 
list(findalliter(...))`, and implements `findfirst()`. 

Consistency and correctness are likely because all tests pass with the 
redefined `findall()`.

--
components: Library (Lib)
messages: 359039
nosy: apalala
priority: normal
pull_requests: 17191
severity: normal
status: open
title: Completeness and symmetry in RE, avoid `findall(...)[0]`
type: enhancement
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39166] Python 3.9.0a2 changed how "async for" traces its final iteration

[Ned Batchelder]

New submission from Ned Batchelder :

3.9.0a2 changed how the final iteration of "async for" is traced.  The body of 
the loop is traced when the body is not executed.  Standard "for" loops don't 
show the same effect.

In the output below, notice that 3.9.0a2 and 3.9.0a2+ both show one last 
execution of line 32, but that line is not actually executed (there is no 
output).  The standard for loop doesn't show line 27 doing that in any version.

--- 8< 
import linecache, sys

def trace(frame, event, arg):
# The weird globals here is to avoid a NameError on shutdown...
if frame.f_code.co_filename == globals().get("__file__"):
lineno = frame.f_lineno
print("{} {}: {}".format(event[:4], lineno, linecache.getline(__file__, 
lineno).rstrip()))
return trace

import asyncio

class AsyncIteratorWrapper:
def __init__(self, obj):
self._it = iter(obj)

def __aiter__(self):
return self

async def __anext__(self):
try:
return next(self._it)
except StopIteration:
raise StopAsyncIteration

def doit_sync():
for letter in "ab":
print(letter)
print(".")

async def doit_async():
async for letter in AsyncIteratorWrapper("ab"):
print(letter)
print(".")

print(sys.version)
sys.settrace(trace)

doit_sync()

loop = asyncio.new_event_loop()
loop.run_until_complete(doit_async())
loop.close()
--- 8< 



$ /usr/local/pythonz/pythons/CPython-3.9.0a1/bin/python3.9 /tmp/bpo2.py
3.9.0a1 (default, Nov 20 2019, 18:52:14)
[Clang 10.0.0 (clang-1000.10.44.4)]
call 25: def doit_sync():
line 26: for letter in "ab":
line 27: print(letter)
a
line 26: for letter in "ab":
line 27: print(letter)
b
line 26: for letter in "ab":
line 28: print(".")
.
retu 28: print(".")
call 30: async def doit_async():
line 31: async for letter in AsyncIteratorWrapper("ab"):
call 13: def __init__(self, obj):
line 14: self._it = iter(obj)
retu 14: self._it = iter(obj)
call 16: def __aiter__(self):
line 17: return self
retu 17: return self
call 19: async def __anext__(self):
line 20: try:
line 21: return next(self._it)
retu 21: return next(self._it)
exce 31: async for letter in AsyncIteratorWrapper("ab"):
line 32: print(letter)
a
line 31: async for letter in AsyncIteratorWrapper("ab"):
call 19: async def __anext__(self):
line 20: try:
line 21: return next(self._it)
retu 21: return next(self._it)
exce 31: async for letter in AsyncIteratorWrapper("ab"):
line 32: print(letter)
b
line 31: async for letter in AsyncIteratorWrapper("ab"):
call 19: async def __anext__(self):
line 20: try:
line 21: return next(self._it)
exce 21: return next(self._it)
line 22: except StopIteration:
line 23: raise StopAsyncIteration
exce 23: raise StopAsyncIteration
retu 23: raise StopAsyncIteration
exce 31: async for letter in AsyncIteratorWrapper("ab"):
line 33: print(".")
.
retu 33: print(".")

$ /usr/local/pythonz/pythons/CPython-3.9.0a2/bin/python3.9 /tmp/bpo2.py
3.9.0a2 (default, Dec 19 2019, 08:42:29)
[Clang 10.0.0 (clang-1000.10.44.4)]
call 25: def doit_sync():
line 26: for letter in "ab":
line 27: print(letter)
a
line 26: for letter in "ab":
line 27: print(letter)
b
line 26: for letter in "ab":
line 28: print(".")
.
retu 28: print(".")
call 30: async def doit_async():
line 31: async for letter in AsyncIteratorWrapper("ab"):
call 13: def __init__(self, obj):
line 14: self._it = iter(obj)
retu 14: self._it = iter(obj)
call 16: def __aiter__(self):
line 17: return self
retu 17: return self
call 19: async def __anext__(self):
line 20: try:
line 21: return next(self._it)
retu 21: return next(self._it)
exce 31: async for letter in AsyncIteratorWrapper("ab"):
line 32: print(letter)
a
line 31: async for letter in AsyncIteratorWrapper("ab"):
call 19: async def __anext__(self):
line 20: try:
line 21: return next(self._it)
retu 21: return next(self._it)
exce 31: async for letter in AsyncIteratorWrapper("ab"):
line 32: print(letter)
b
line 31: async for letter in AsyncIteratorWrapper("ab"):
call 19: async def __anext__(self):
line 20: try:
line 21: return next(self._it)
exce 21: return next(self._it)
line 22: except StopIteration:
line 23: raise StopAsyncIteration
exce 23: raise StopAsyncIteration
retu 23: raise StopAsyncIteration
exce 31: async for letter in AsyncIteratorWrapper("ab"):
line 32: print(letter)
line 33: print(".

[issue39165] Completeness and symmetry in RE, avoid `findall(...)[0]`

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +ezio.melotti, mrabarnett, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38314] Implement unix read_pipe.is_reading() method

[Callum Ward]

Change by Callum Ward :


--
pull_requests: +17192
pull_request: https://github.com/python/cpython/pull/17755

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39125] Type signature of @property not shown in help()

[Nguyễn Gia Phong]

Nguyễn Gia Phong  added the comment:

Relating to this, should there also be indication about the mode (get, set, 
del) the property?  Currently there is no way to tell if the *attribute* is 
read-only, read-write or write-only.

--
versions:  -Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39119] email/_header_value_parser.py:parse_message_id: UnblondLocalError

[Drew DeVault]

Drew DeVault  added the comment:

Sorry for the delay, was travelling for the holidays. I'll check that this is 
not an issue with 3.8.1. Thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36051] Drop the GIL during large bytes.join operations?

[Bruce Merry]

Change by Bruce Merry :


--
keywords: +patch
pull_requests: +17193
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/17757

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36220] LOAD_NAME and LOAD_GLOBAL, STORE_GLOBAL handle dict subclasses for globals() differently

[Paul Sokolovsky]

Change by Paul Sokolovsky :


--
nosy: +pfalcon
title: LOAD_NAME and LOAD_GLOBAL handle dict subclasses for globals() 
differently -> LOAD_NAME and LOAD_GLOBAL, STORE_GLOBAL handle dict subclasses 
for globals() differently

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39019] Missing class getitems in standard library classes

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:


New changeset 4dc5a9df59837446ec1dc5b7a0e6ce95ae5b5cec by Ivan Levkivskyi 
(Batuhan Taşkaya) in branch 'master':
bpo-39019: Implement missing __class_getitem__ for subprocess classes (GH-17558)
https://github.com/python/cpython/commit/4dc5a9df59837446ec1dc5b7a0e6ce95ae5b5cec


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39019] Missing class getitems in standard library classes

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:


New changeset 09c482fad11c769be38b2449f1056e264b701bb7 by Ivan Levkivskyi 
(Batuhan Taşkaya) in branch 'master':
bpo-39019: Implement missing __class_getitem__ for SpooledTemporaryFile 
(GH-17560)
https://github.com/python/cpython/commit/09c482fad11c769be38b2449f1056e264b701bb7


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36220] LOAD_NAME and LOAD_GLOBAL, STORE_GLOBAL handle dict subclasses for globals() differently

[Paul Sokolovsky]

Paul Sokolovsky  added the comment:

> I wanted to write a sandbox for Python.

Sandbox indeed, it is.

class NS(dict):

def __setitem__(self, k, v):
if not isinstance(v, type(lambda: 0)):
raise RuntimeError("Global variables considered harmful")

globals = NS()

exec("foo = 1", globals)

But then:

exec("""
global foo
foo = "watch me escaping your sandboxes"
""", globals)

This is due to STORE_GLOBAL not handling dict subclasses, unlike STORE_NAME.


While @Kevin Shweh's issue with LOAD_NAME, and @vstinner's concerns of adding 
yet another branch to LOAD_NAME implementation may be one matter, issue with 
STORE_GLOBAL is both related and somewhat different. STORE_GLOBAL's should be 
relatively infrequent, so adding another branch to it unlikely will be 
quantifiable in performance. But lack of its support disallows to write to 
tools which police/constrain Python's overdynamicity, which is useful in the 
age.

Anyway, I decided to not open a separate ticket for this, but add here. Fairly 
speaking, as long as work to support dict subclasses as globals/locals started, 
the only reasonable way forward seems to implement it completely.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39167] argparse boolean type bug

[Trenton Bricken]

New submission from Trenton Bricken :

This is a bug with argparse. 

Say I have:

parser.add_argument('--verbose', type=bool, action='store', nargs='+',
default = [False],
help='turns on verbosity')

If in the command line I have "--verbose False' the default will then evaluate 
to True and return the value "True" rather than the value of "False" that I 
tried to set it to!

When a developer has lots of arguments to pass in, they may not remember if an 
argument defaults to False or True. Setting the value to False and having it 
then return True is very confusing and should not occur. 

Right now I have a work-around where I have a new type=buildBool where:

def buildBool(arg):
   return bool(arg)

and do:

parser.add_argument('--verbose', type=buildBool, action='store', nargs='+',
default = ['False'],
help='turns on verbosity')

but this means I have to have this type and have my default value be a string 
which is suboptimal and this bug remains a trap for other developers.

--
messages: 359045
nosy: Trenton Bricken
priority: normal
severity: normal
status: open
title: argparse boolean type bug
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39167] argparse boolean type bug

[Trenton Bricken]

Trenton Bricken  added the comment:

Update: 

I was being dumb before, the problem still remains but my work around 
previously was wrong. This is the new workaround: 

def buildBool(arg):
if arg == 'False':
return False
else:
return True

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39167] argparse boolean type bug

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

It seems like this is a common problem : 
https://stackoverflow.com/questions/15008758/parsing-boolean-values-with-argparse
 . I guess you want to store verbose=True when --verbose is passed and 
verbose=False when --verbose is not passed where store_true might help.

--
nosy: +paul.j3, rhettinger, xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Ruslan Dautkhanov]

New submission from Ruslan Dautkhanov :

Reported originally here - 
https://twitter.com/__zero323__/status/1210911632953692162

See details here
https://asciinema.org/a/290643

In [4]: class Foo: pass
In [5]: %timeit -n1_000_000 Foo()
88.5 ns ± 3.44 ns per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [6]: T = TypeVar("T")
In [7]: class Bar(Generic[T]): pass
In [8]: %timeit -n1_000_000 Bar()
883 ns ± 3.46 ns per loop (mean ± std. dev. of 7 runs, 100 loops each)

Same effect in Python 3.6 and 3.8

--
messages: 359049
nosy: Ruslan Dautkhanov
priority: normal
severity: normal
status: open
title: Generic type subscription is a huge toll on Python performance
type: performance
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Ruslan Dautkhanov]

Ruslan Dautkhanov  added the comment:

Python typing gives an order of magnitude slow down in this case

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39167] argparse boolean type bug

[Trenton Bricken]

Trenton Bricken  added the comment:

Thank you for your quick and helpful reply. 

The problem with your solution is twofold: 
1. it adds some cognitive load in needing to remember whether or not the flag 
defaults to True or False and thus whether or not you need to add it. It is 
easier for me to have everything as a flag with a value that follows. 
2. More importantly, I am using argparse for trying lots of different 
combinations of inputs to a function so I pass them in as a list and then 
permute the possible values for each input. 

For example: --flag1 a b
--flag2 c d
 
I want to then run a command with the parameter combinations: 
a, c; a, d; b, c; b, d;

And I don't think store_true store_false would allow me to pass in having 
combinations of True and False like --flag True False would. 

I am fine now with my current work around, but was very confused for some time 
why my flag would be true when I set it to false. And think this is a 
counterintuitive pitfall other developers can fall into.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Ned Deily]

Change by Ned Deily :


--
nosy: +gvanrossum, levkivskyi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36220] LOAD_NAME and LOAD_GLOBAL, STORE_GLOBAL handle dict subclasses for globals() differently

[ppperry]

ppperry  added the comment:

Duplicate of issue32615

--
nosy: +ppperry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24955] webbrowser broken on Mac OS X when using the BROWSER variable

[Eitan Adler]

Change by Eitan Adler :


--
nosy: +eitan.adler

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36220] LOAD_NAME and LOAD_GLOBAL, STORE_GLOBAL handle dict subclasses for globals() differently

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Closed as duplicate of issue32615.

--
nosy: +pablogsal
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39167] argparse boolean type bug

[paul j3]

paul j3  added the comment:

Despite the name, the 'type' parameter specifies a function, not a Python 
class.  

The only string that produces False is the empty one: bool('').  So 'type=bool' 
is valid Python, even if it isn't useful.

With `nargs='+'` there's no problem with providing strings like 'False', 
'true', 'no', 'oui', 'niet', but if you want to convert those to boolean 
True/False values, you need to write your own 'type' function.


There was a recent bug/issue that proposed providing such a function (or 
importing it from another module), and shadowing the existing 'bool' function, 
but that has been rejected (I think).  It isn't really needed, and the proposed 
solution was too language specific.

Seems to me that expecting your user to provide an open ended list of 'True 
False False True' strings would be rather confusing, or at least require a 
complicated 'help' string.  In any case it's not a common enough case to 
require any changes to the core argparse functionality.

In sum, it isn't clear what the bug is, or what patch you expect.  This sounds 
more like a StackOverflow question, than a bug report.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38610] use-after-free in list object function

[miss-islington]

Change by miss-islington :


--
pull_requests: +17194
pull_request: https://github.com/python/cpython/pull/17758

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38610] use-after-free in list object function

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset d9e561d23d994e3ed15f4fcbd7ee5c8fe50f190b by Pablo Galindo 
(Zackery Spytz) in branch 'master':
bpo-38610: Fix possible crashes in several list methods (GH-17022)
https://github.com/python/cpython/commit/d9e561d23d994e3ed15f4fcbd7ee5c8fe50f190b


--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38610] use-after-free in list object function

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
pull_requests: +17195
pull_request: https://github.com/python/cpython/pull/17759

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38610] use-after-free in list object function

[miss-islington]

miss-islington  added the comment:


New changeset fcaf14cd9179bb48850f8f81ce8d5cee28129745 by Miss Islington (bot) 
in branch '3.8':
bpo-38610: Fix possible crashes in several list methods (GH-17022)
https://github.com/python/cpython/commit/fcaf14cd9179bb48850f8f81ce8d5cee28129745


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38610] use-after-free in list object function

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38610] use-after-free in list object function

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset 296d45ec10fb55532bc3fac2311a3f91299ecf59 by Pablo Galindo in 
branch '3.7':
[3.7] bpo-38610: Fix possible crashes in several list methods (GH-17022) 
(GH-17759)
https://github.com/python/cpython/commit/296d45ec10fb55532bc3fac2311a3f91299ecf59


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38870] Expose ast.unparse in the ast module

[Batuhan]

Change by Batuhan :


--
pull_requests: +17196
pull_request: https://github.com/python/cpython/pull/17760

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39167] argparse boolean type bug

[paul j3]

paul j3  added the comment:

The rejected boolean type proposal:

https://bugs.python.org/issue37564

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38967] Improve error message in enum for member name surrounded by underscore.

[Ethan Furman]

Ethan Furman  added the comment:

Rubén, good idea.

I wonder, though, if we should say:

  _sunder_ names, such as "%s", are reserved for future Enum use

since `_sunder_` is used in the Enum docs and is unlikely to be confused with 
one's own variable name.

--
assignee:  -> ethan.furman

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39167] argparse boolean type bug

[Mark Dickinson]

Mark Dickinson  added the comment:

This looks like a duplicate of #37564.

--
nosy: +mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39167] argparse boolean type bug

[Mark Dickinson]

Change by Mark Dickinson :


--
Removed message: https://bugs.python.org/msg359060

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34118] Fix some class entries in 'Built-in Functions'

[Terry J. Reedy]

Change by Terry J. Reedy :


--
pull_requests: +17197
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/17761

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Ruslan Dautkhanov]

Ruslan Dautkhanov  added the comment:

In [12]: cProfile.run("for _ in range(100_000): Bar()") 

   
 23 function calls in 0.136 seconds 

   


   
   Ordered by: standard name

   


   
   ncalls  tottime  percall  cumtime  percall filename:lineno(function) 

   
10.0470.0470.1360.136 :1()  

   
   100.0790.0000.0890.000 typing.py:865(__new__)

   
   100.0100.0000.0100.000 {built-in method __new__ of type 
object at 0x55ab65861ac0}   

10.0000.0000.1360.136 {built-in method builtins.exec}   

   
10.0000.0000.0000.000 {method 'disable' of 
'_lsprof.Profiler' objects} 



   


   


   
In [13]: # And typing.py:865 points to  

   


   
In [14]: inspect.getsourcelines(Generic.__new__)

   
Out[14]:

   
(['def __new__(cls, *args, **kwds):\n', 

   
  'if cls in (Generic, Protocol):\n',   

   
  'raise TypeError(f"Type {cls.__name__} cannot be instantiated; 
"\n',   
  
  '"it can be used only as a base class")\n',   

   
  'if super().__new__ is object.__new__ and cls.__init__ is not 
object.__init__:\n',
   
  'obj = super().__new__(cls)\n',   

[issue34118] Fix some class entries in 'Built-in Functions'

[Terry J. Reedy]

Terry J. Reedy  added the comment:


New changeset ee9ff05ec22ecd47dbffdd361967ccd55963dad2 by Terry Jan Reedy in 
branch 'master':
 bpo-34118: memoryview, range, and tuple are classes  (GH-17761)
https://github.com/python/cpython/commit/ee9ff05ec22ecd47dbffdd361967ccd55963dad2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34118] Fix some class entries in 'Built-in Functions'

[miss-islington]

Change by miss-islington :


--
pull_requests: +17198
pull_request: https://github.com/python/cpython/pull/17762

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34118] Fix some class entries in 'Built-in Functions'

[Terry J. Reedy]

Terry J. Reedy  added the comment:

I started with the agreed-on 'function' => 'class' changes and intend to follow 
with a PR for text changes.

--
stage: patch review -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34118] Fix some class entries in 'Built-in Functions'

[miss-islington]

Change by miss-islington :


--
pull_requests: +17199
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/17763

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34118] Fix some class entries in 'Built-in Functions'

[miss-islington]

miss-islington  added the comment:


New changeset c9c17cc933dcffb9ed7f03e3f791d8cfd7acc54a by Miss Islington (bot) 
in branch '3.7':
bpo-34118: memoryview, range, and tuple are classes  (GH-17761)
https://github.com/python/cpython/commit/c9c17cc933dcffb9ed7f03e3f791d8cfd7acc54a


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32615] Inconsistent behavior if globals is a dict subclass

[Paul Sokolovsky]

Paul Sokolovsky  added the comment:

Some smart maintainer closed https://bugs.python.org/issue36220 as a duplicate 
of this one. That ticket might have more details of the underlying issues.

--
nosy: +pfalcon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34118] Fix some class entries in 'Built-in Functions'

[miss-islington]

miss-islington  added the comment:


New changeset ec941568bdb25e164a87a23cf1b8870ac047b4e3 by Miss Islington (bot) 
in branch '3.8':
bpo-34118: memoryview, range, and tuple are classes  (GH-17761)
https://github.com/python/cpython/commit/ec941568bdb25e164a87a23cf1b8870ac047b4e3


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32615] Inconsistent behavior if globals is a dict subclass

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Victor, Pablo, I added you two since you commented on the duplicate.

I believe this should be closed as 'not a bug'.  The doc for globals() says is 
returns a 'dictionary', meaning an instance of dict.  The doc for exec says 
globals "must be a dictionary (and not a subclass of dictionary)".  (This is 
mere implied when locals is also given, but it *is* implied.).  The behavior 
when globals is otherwise is undefined.

--
nosy: +pablogsal, vstinner
versions: +Python 3.9 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32615] Inconsistent behavior if globals is a dict subclass

[Paul Sokolovsky]

Paul Sokolovsky  added the comment:

> The doc for exec says globals "must be a dictionary (and not a subclass of 
> dictionary)"

Docs are full of mistakes and outdated information.

Fixing STORE_GLOBAL case from https://bugs.python.org/issue36220#msg359046 
would be trivial and cheap re: overheads. And useful, yeah.

And the gentleman who submitted the other ticket argues that:

> but the inconsistency between LOAD_NAME and LOAD_GLOBAL definitely seems 
> wrong.

Which is easy to agree with, though there're more concerns re: runtime overhead.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32615] Inconsistent behavior if globals is a dict subclass

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> The doc for exec says globals "must be a dictionary (and not a subclass of 
> dictionary)"

I agree with Terry, the moment you pass a dict subclass to exec you are out of 
contract. If any, we may need to sanitize the input to exec, although I don't 
think is worth paying the performance price for that.

> Some smart maintainer closed https://bugs.python.org/issue36220 as a 
> duplicate of this one.

I closed issue36220 as a duplicate. Paul, I don't know if you are being 
sarcastic here so I will assume that you are not but in case you are, I suggest 
you stop as this violates our CoC.

> Docs are full of mistakes and outdated information.

They are also full of contracts and as of today, what the docs document is what 
is considered the public API. 

In any case, my opinion is that LOAD_NAME and LOAD_GLOBAL should not pay the 
performance price to handle less-specific APIs just to support a very uncommon 
case (using dict subclasses as globals) that are also out of contract. For 
STORE_GLOBAL I feel less strongly about that, but I am still -1 on it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34938] Fix mimetype.init() to account for from import

[Ashley Whetter]

Ashley Whetter  added the comment:

Yes I'm happy to make those changes as part of this.

So clarify what needs to change in PR 16567:

1) Include a note in the docs for `inited` that outlines that if it is imported 
into the local scope, it will not be updated by calls to `init()`. Only 
`mimetypes.inited` will.

2) Add a clarification to the `add_type()` docs that changes made with 
`strict=True` means that they will persist after subsequent calls to `init()`.

3) Remove `_default_mime_types()` in favour of public global constants. The 
documentation currently refers to these globals as the "official MIME types". 
Should that wording change to use "default" instead of "official", or maybe 
should the names of the constants change to `OFFICIAL_SUFFIX_MAP` for example?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39169] TypeError: 'int' object is not callable if the signal handler is SIG_IGN

[Ronald Li]

New submission from Ronald Li :

The attached script ign2ndsig.py demonstrates an attempted way to handle 
signals at most once and ignore any subsequent signals. SIGINT and SIGTERM are 
used in the demo.


In Python 3.5, the subprocess would go into one of the "except 
KeyboardInterrupt:" or "except SystemExit:" blocks before doing "finally:" and 
exiting zero:


# execute the script with no args:
# ./ign2ndsig.py

subproc: sys.version_info(major=3, minor=5, micro=9, releaselevel='final', 
serial=0)
raising KeyboardInterrupt
handling KeyboardInterrupt
doing finally
rc: 0



In Python 3.6, 3.7 or 3.8, the subprocess would go into neither the "except 
KeyboardInterrupt:" nor "except SystemExit:" blocks before doing "finally:" and 
exiting non-zero, with a traceback like this:


subproc: sys.version_info(major=3, minor=6, micro=10, releaselevel='final', 
serial=0)
raising KeyboardInterrupt
doing finally
Traceback (most recent call last):
  File "./ign2ndsig.py", line 30, in subproc
input()
  File "./ign2ndsig.py", line 18, in handler
raise KeyboardInterrupt
KeyboardInterrupt

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./ign2ndsig.py", line 58, in 
subproc()
  File "./ign2ndsig.py", line 32, in subproc
printef('handling KeyboardInterrupt')
  File "./ign2ndsig.py", line 10, in printef
print(*objects, file=sys.stderr, flush=True)
TypeError: 'int' object is not callable
rc: 1



More on the behaviors of ign2ndsig.py :
1. Replacing SIG_IGN at line 14 and 15 with a no-op Python function "fixes" the 
problem - such that in Python 3.6, 3.7 or 3.8, it'd behave similarly to the 
Python 3.5 output above.
2. Sending a SIGTERM then a SIGINT (instead of a SIGINT followed by a SIGTERM) 
(i.e. reversing the order of SIGINT and SIGTERM) at line 49-50 results in a 
similar behavior (that the TypeError is raised).
3. Sending 2 SIGINTs or 2 SIGTERMs (instead of 1 SIGINT + 1 SIGTERM) (i.e. 
sending the same signal twice) at line 49-50 does NOT result in the TypeError.


Potentially related issues:

_thread.interrupt_main() errors if SIGINT handler in SIG_DFL, SIG_IGN
https://bugs.python.org/issue23395

Turn SIG_DFL and SIG_IGN into functions
https://bugs.python.org/issue23325

--
files: ign2ndsig.py
messages: 359071
nosy: Ronald Li
priority: normal
severity: normal
status: open
title: TypeError: 'int' object is not callable if the signal handler is SIG_IGN
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8
Added file: https://bugs.python.org/file48810/ign2ndsig.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Guido van Rossum]

Guido van Rossum  added the comment:

What Python version was used for the timings? If not 3.8, please do over in 3.8.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32615] Inconsistent behavior if globals is a dict subclass

[Paul Sokolovsky]

Paul Sokolovsky  added the comment:

> I agree with Terry, the moment you pass a dict subclass to exec you are out 
> of contract. If any, we may need to sanitize the input to exec, although I 
> don't think is worth paying the performance price for that.

exec() params are already checked, as a seconds param, only dict or dict 
subclasses are accepted. Seems like good enough contract.

> Paul, I don't know if you are being sarcastic here so I will assume that you 
> are not but in case you are, I suggest you stop as this violates our CoC.

I am not, and please stop your accusations, that violates CoC.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Guido van Rossum]

Guido van Rossum  added the comment:

Sorry, you already said 3.6 and 3.8 give the same effect. But what if you add a 
minimal __new__() to Foo?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32615] Inconsistent behavior if globals is a dict subclass

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> exec() params are already checked, as a seconds param, only dict or dict 
> subclasses are accepted. Seems like good enough contract.

As I said, the documentation is considered the public API contract and again, 
the documentation says that dict subclasses are not accepted:

>> it must be a dictionary (and not a subclass of dictionary)

My opinion is that as Terry said, this is a violation of the contract and 
supporting dict subclasses is a case unusual enough to not sacrifice 
performance for the general case. 

> I am not, and please stop your accusations, that violates CoC.

A respectful indication to keep the discussion civilized does not violate the 
CoC. If you are not being sarcastic, then everything is fine.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38588] Use-after-free in dict/list

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset 2d5bf568eaa5059402ccce9ba5a366986ba27c8a by Pablo Galindo 
(Dong-hee Na) in branch 'master':
bpo-38588: Fix possible crashes in dict and list when calling 
PyObject_RichCompareBool (GH-17734)
https://github.com/python/cpython/commit/2d5bf568eaa5059402ccce9ba5a366986ba27c8a


--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38588] Use-after-free in dict/list

[Dong-hee Na]

Change by Dong-hee Na :


--
pull_requests: +17200
pull_request: https://github.com/python/cpython/pull/17764

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38588] Use-after-free in dict/list

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> How about calling Py_INCREF and Py_DECREF in PyObject_RichCompare or 
> do_richcompare?

Apologies, I had missed this suggestion before merging the PR :(

If we decide to add the check to PyObject_RichCompare or do_richcompare we 
should also adapt the fix for https://bugs.python.org/issue38610

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38588] Use-after-free in dict/list

[Dong-hee Na]

Change by Dong-hee Na :


--
pull_requests: +17201
pull_request: https://github.com/python/cpython/pull/17765

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38588] Use-after-free in dict/list

[Inada Naoki]

Inada Naoki  added the comment:

$ ./python -m pyperf timeit -s 'a = ["a"]*100; b = ["a"]*100;' -- 'a == b'

master : Mean +- std dev: 276 ns +- 1 ns
patched: Mean +- std dev: 572 ns +- 3 ns

This makes list comparison 2x slower.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38588] Use-after-free in dict/list

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> This makes list comparison 2x slower.

Would you like to revert PR 17734? Calling Py_INCREF and Py_DECREF in 
PyObject_RichCompare or do_richcompare will take the same effect, no?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38588] Use-after-free in dict/list

[Dong-hee Na]

Dong-hee Na  added the comment:

> This makes list comparison 2x slower.

This is affected by PR 17734? or PyObject_RichCompare patched?

--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39167] argparse boolean type bug

[Josh Rosenberg]

Change by Josh Rosenberg :


--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> ArgumentParser should support bool type according to truth 
values

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39168] Generic type subscription is a huge toll on Python performance

[Guido van Rossum]

Guido van Rossum  added the comment:

Hm, here's what I measure in Python 3.8.1. (I don't use IPython or notebooks so 
this looks a little different.)


>>> timeit.timeit('Foo()', 'class Foo: pass')
0.3763025619934


>>> timeit.timeit('Foo()', 'class Foo:\n  def __new__(cls): return 
>>> super().__new__(cls)')
1.575319603864


>>> timeit.timeit('Foo()', 'from typing import Generic, TypeVar\nT = 
>>> TypeVar("T")\nclass Foo(Generic[T]): pass')
3.874873715068


>From this I conclude that adding a minimal __new__() method is responsible for 
>about 4x slowdown, and the functionality in typing.py for another factor 2.5.


While this isn't great I don't see an easy way to improve upon this without 
rewriting the entire typing module in C.  (Some of this may or may not happen 
for PEP 604.)

PS. I just realized my Python binary was built with debug options, so absolute 
numbers will look different (better) for you -- but relative numbers will look 
the same, and I get essentially the same factors with 3.9.0a1+.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38588] Use-after-free in dict/list

[Inada Naoki]

Change by Inada Naoki :


--
pull_requests: +17202
pull_request: https://github.com/python/cpython/pull/17766

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38588] Use-after-free in dict/list

[Dong-hee Na]

Dong-hee Na  added the comment:

Master
Mean +- std dev: 1.08 us +- 0.02 us

Before PR-17734
Mean +- std dev: 584 ns +- 12 ns

New suggested
.
Mean +- std dev: 578 ns +- 14 ns

diff --git a/Objects/object.c b/Objects/object.c
index 6fc1146..b42f41a 100644
--- a/Objects/object.c
+++ b/Objects/object.c
@@ -865,6 +865,8 @@ PyObject_RichCompareBool(PyObject *v, PyObject *w, int op)
 return 0;
 }

+Py_INCREF(v);
+Py_INCREF(w);
 res = PyObject_RichCompare(v, w, op);
 if (res == NULL)
 return -1;
@@ -873,6 +875,8 @@ PyObject_RichCompareBool(PyObject *v, PyObject *w, int op)
 else
 ok = PyObject_IsTrue(res);
 Py_DECREF(res);
+Py_DECREF(v);
+Py_DECREF(w);
 return ok;
 }

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38588] Use-after-free in dict/list

[Inada Naoki]

Inada Naoki  added the comment:

>> This makes list comparison 2x slower.
>
> This is affected by PR 17734? or PyObject_RichCompare patched?

Caused by PR 17734.


> Would you like to revert PR 17734? Calling Py_INCREF and Py_DECREF in 
> PyObject_RichCompare or do_richcompare will take the same effect, no?

Moving INCREF and DECREF is a huge change.  It is just a future idea to prevent 
same type of bugs.  I think it can not be backported.

To fix this issue with minimum performance regression, I think PR 17766 is the 
best solution.  So no need to revert PR 17734.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36051] Drop the GIL during large bytes.join operations?

[Josh Rosenberg]

Josh Rosenberg  added the comment:

This will introduce a risk of data races that didn't previously exist. If you 
do:

ba1 = bytearray(b'\x00') * 5
ba2 = bytearray(b'\x00') * 5
... pass references to thread that mutates them ...
ba3 = b''.join((ba1, ba2))

then two things will change from the existing behavior:

1. If the thread in question attempts to write to the bytearrays in place, then 
it could conceivably write data that is only partially picked up (ba1[0], 
ba1[4] = 2, 3 could end up copying the results of the second write without 
the first; at present, it could only copy the first without the second)

2. If the thread tries to change the size of the bytearrays during the join 
(ba1 += b'123'), it'll die with a BufferError that wasn't previously possible

#1 isn't terrible (as noted, data races in that case already existed, this just 
lets them happen in more ways), but #2 is a little unpleasant; code that 
previously had simple data races (the data might be inconsistent, but the code 
ran and produced some valid output) can now fail hard, nowhere near the actual 
call to join that introduced the behavioral change.

I don't think this sinks the patch (loudly breaking code that was silently 
broken before isn't awful), but I feel like a warning of some kind in the 
documentation (if only a simple compatibility note in What's New) might be 
appropriate.

--
nosy: +josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39146] too much memory consumption in re.compile unicode

[Zhipeng Xie]

Zhipeng Xie <775350...@qq.com> added the comment:

Hi, I tracked it down and found that this problem was introduced in python2.7.9 
by following commit:

https://hg.python.org/cpython/rev/ebd48b4f650d

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32615] Inconsistent behavior if globals is a dict subclass

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Guido, some people would like manipulation and use of globals to work with dict 
subclasses as well as dicts themselves.  I believe that the current 
restriction/optimization is at least partly your work.  Do you have any quick 
opinion, or do you think discussion should go to python-ideas?

--
nosy: +gvanrossum
stage:  -> test needed
type: behavior -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32615] Inconsistent behavior if globals is a dict subclass

[Terry J. Reedy]

Terry J. Reedy  added the comment:

There are more examples on duplicate #36220.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38588] Use-after-free in dict/list

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> Moving INCREF and DECREF is a huge change.  It is just a future idea to 
> prevent same type of bugs.  I think it can not be backported.

Now I am wondering how many other APIs are affected by the same pattern other 
than PyObject_RichCompareBool

> To fix this issue with minimum performance regression, I think PR 17766 is 
> the best solution.  So no need to revert PR 17734.

Thanks for the quick fix and the analysis. I reviewed PR 17734.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38588] Use-after-free in dict/list

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Sorry, I meant that I reviewed PR 17766.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38588] Use-after-free in dict/list

[Inada Naoki]

Inada Naoki  added the comment:


New changeset dfef986f12dd92bd6434117bba0db3cbb4e65243 by Inada Naoki in branch 
'master':
bpo-38588: Optimize list comparison. (GH-17766)
https://github.com/python/cpython/commit/dfef986f12dd92bd6434117bba0db3cbb4e65243


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32615] Inconsistent behavior if globals is a dict subclass

[Guido van Rossum]

Guido van Rossum  added the comment:

This is probably an issue for the new Steering Council.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37446] Undefined behavior in Python/hamt.c

[Benjamin Peterson]

Benjamin Peterson  added the comment:


New changeset d0c92e81aa2171228a23cb2bed36f7dab975257d by Benjamin Peterson 
(Batuhan Taşkaya) in branch 'master':
closes bpo-37446: resolve undefined behavior in Python/hamt.c (GH-17727)
https://github.com/python/cpython/commit/d0c92e81aa2171228a23cb2bed36f7dab975257d


--
nosy: +benjamin.peterson
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37446] Undefined behavior in Python/hamt.c

[miss-islington]

Change by miss-islington :


--
pull_requests: +17203
pull_request: https://github.com/python/cpython/pull/17767

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37446] Undefined behavior in Python/hamt.c

[miss-islington]

miss-islington  added the comment:


New changeset a278ae19b4daa1deb11e2a8eed38838027e90ece by Miss Islington (bot) 
in branch '3.8':
closes bpo-37446: resolve undefined behavior in Python/hamt.c (GH-17727)
https://github.com/python/cpython/commit/a278ae19b4daa1deb11e2a8eed38838027e90ece


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com