[issue40508] IDLE won't open

[Louis]

Change by Louis :


--
components: macOS
files: IDLE.app.zip
nosy: Louis, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: IDLE won't open
type: performance
versions: Python 3.6
Added file: https://bugs.python.org/file49120/IDLE.app.zip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40508] IDLE won't open

[Ned Deily]

Change by Ned Deily :


--
assignee:  -> terry.reedy
components: +IDLE -macOS
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40496] re.findall() takes a long time (100% cup usage) on Python 3.6.10

[Sergio Rael]

Sergio Rael  added the comment:

Thank you for your reply Rémi.

I agree with you that the reason can be that the pattern is too complex. I just 
noticed that in Python 3.7 using the same pattern finish the searchall almost 
instantaneously, but in 3.6 the CPU goes to 100% and it takes ages to finish. 
In fact I don't know if this can finish at all because it takes so long that I 
had to stop it.
I tough it would be a good idea to let you know this behaviour. Of course, 
after this, I don't use 3.6 anymore.

Thanks again!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40508] IDLE won't open

[Louis]

New submission from Louis :

Hi, I cannot open IDLE. When i try and open it from finder it says that the 
file could not be opened. When I try and open it from the dock, it bounces once 
and then nothing else happens. When i try and open it from terminal, it says 
that there is a bad cpu type in the executable.

-Louis

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36543] Remove old-deprecated ElementTree features (part 2)

[Miro Hrončok]

Miro Hrončok  added the comment:

Some stats: See 
https://bugzilla.redhat.com/showdependencytree.cgi?id=PYTHON39&hide_resolved=0 
and Ctrl+F for cElementTree.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40350] modulefinder chokes on numpy - dereferencing None in spec.loader

[Cajetan Rodrigues]

Change by Cajetan Rodrigues :


--
keywords: +patch
pull_requests: +19231
stage: test needed -> patch review
pull_request: https://github.com/python/cpython/pull/19917

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40350] modulefinder chokes on numpy - dereferencing None in spec.loader

[Cajetan Rodrigues]

Cajetan Rodrigues  added the comment:

Turns out using _PKG_DIRECTORY as a type for namespace packages ended up making 
`_find_module` try to search for an __init__.py within them, since it had no 
understanding of the diff. between a namespace package and a regular one (the 
lack of __init__.py), and caused it to break.

I've raised a PR with a new type _NSP_DIRECTORY for namespace-directories.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36543] Remove old-deprecated ElementTree features (part 2)

[Stefan Behnel]

Stefan Behnel  added the comment:

Thanks Miro, that's beautiful. Good to know that I'm not the only one who 
doesn't read documentation. ;-)

So, how do we deal with this? We can't warn about the deprecation without 
annoying everyone. We can't just remove the empty module without breaking lots 
of code. We also can't really let the module stick around forever and let 
people write more code that uselessly imports it.

Any ideas?

--
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40508] IDLE won't open

[Ronald Oussoren]

Ronald Oussoren  added the comment:

What version of macOS do you use?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40508] IDLE won't open

[Louis]

Louis  added the comment:

Mac OS Catalina 10.15.4

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40508] IDLE won't open

[Ned Deily]

Ned Deily  added the comment:

Thanks for submitting the version of IDLE you were using.  It appears to be a 
Python 3.6.1 from 2017; the version is very old-of-date and no longer 
supported.  Python 3.6.x is now in security fix mode only; Python 3.8.2 is now 
current.  Unless you have a reallygood reason to stick with it you should 
download and use the latest version from here:

https://www.python.org/downloads/

The reason you are getting the bad cpy type message is that the 3.6.1 variant 
you are trying to use was built for 32-bit architectures only.  macOS 10.15 no 
longer supports running 32-bit executables.  The most recent downloads from 
python.org for macOS are 64-bit executables and work fine on 10.15.

--
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40508] IDLE won't open

[Louis]

Louis  added the comment:

Thank You for your reply

-Louis

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36543] Remove old-deprecated ElementTree features (part 2)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Thank you Miro for information. It looks as awesome work!

The simplest solution would be to restore xml.etree.cElementTree and keep it 
forever. It will not harm anyone, it is just outdated import here and there.

We could also write an import hook which would analyze the code of the importer 
and warn only if xml.etree.cElementTree is imported without a common idiom. But 
it looks slightly overcomplicated to me.

And we can just let the users to fix they code for 3.9.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40496] re.findall() takes a long time (100% cup usage) on Python 3.6.10

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

It is hard to say what is the problem, but seems it was solved in 3.7. Either 
it was an optimization, or a bug fix which had such side effect. If it was a 
bug fix, it was one of backward incompatible bugfixes which are not backported 
to older versions.

--
nosy: +serhiy.storchaka
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36543] Remove old-deprecated ElementTree features (part 2)

[Christian Heimes]

Christian Heimes  added the comment:

Stefan wrote:
> Christian, I understand your complaint, but I've actually never seen code in 
> the wild that didn't provide a fallback for the import, usually something 
> like what Serhiy spelled out

My package defusedxml broke with the change. Python's documention recommends my 
package for parsing XML from untrusted sources. defusedxml 0.7.0rc1 addresses 
the problem by removing defusedxml.cElementTree in 3.9, too. The fix is going 
to break other packages, too.

Serhiy wrote:
> xml.etree.cElementTree should be treated in the same way as cStringIO and 
> cPickle -- they where separate modules in the past, but now the acceleration 
> is used by default in io.StringIO and pickle. It looks an oversign that it 
> was not removed in 3.0.

I completely agree with you. It should have been removed with 3.0. But this 
fact doesn't grant a carte blanche to remove the module without a deprecation 
phase with a proper deadline.

Stefan wrote:
> So, how do we deal with this? We can't warn about the deprecation without 
> annoying everyone. We can't just remove the empty module without breaking 
> lots of code. We also can't really let the module stick around forever and 
> let people write more code that uselessly imports it.

I prefer annoyance over getting broking by surprise. If the current deprecation 
warning system doesn't work for us, then we need to come up with a better plan. 
In the mean time please give users one release head star to adjust their code. 
I propose that feature shall not be removed in 3.x unless the documentation of 
3.(x-1).0 release has explicitly stated the removal of a feature (unless other 
special circumstances and general consense require it).

* Add deprecation and removal warning for xml.etree.cElementTree to 
whatsnew/3.9.rst
* Remove xml.etree.cElementTree in 3.10

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36543] Remove old-deprecated ElementTree features (part 2)

[Christian Heimes]

Christian Heimes  added the comment:

IMHO we don't have to spend extra work on an import hook. It might be a good 
idea to raise the problem with https://github.com/PyCQA/, though. Large 
projects tend to use code checkers and linters.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36543] Remove old-deprecated ElementTree features (part 2)

[Stefan Behnel]

Stefan Behnel  added the comment:

It's not like the deprecation was hidden – the ET docs have this line right at 
the top:

"Changed in version 3.3: This module will use a fast implementation whenever 
available. The xml.etree.cElementTree module is deprecated."

https://docs.python.org/3/library/xml.etree.elementtree.html

But as I said, people don't read docs.

I agree that it's best to keep the module "for now", and maybe leave it to IDEs 
and code checkers to spread the deprecation notice for us. As Serhiy said, it 
doesn't hurt us much to have it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40028] Math module method to find prime factors for non-negative int n

[Rémi Lapeyre]

Change by Rémi Lapeyre :


--
nosy: +remi.lapeyre
nosy_count: 7.0 -> 8.0
pull_requests: +19232
pull_request: https://github.com/python/cpython/pull/19918

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36543] Remove old-deprecated ElementTree features (part 2)

[Christian Heimes]

Christian Heimes  added the comment:

3.3.0 was released in 2012. That's almost a lifetime ago. I assume that 
majority of users either forgot, missed the note, or no longer took the 
deprecation serious. After seven, eight years we should do our users a service 
and point out the deprecation with a concrete deadline.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40028] Math module method to find prime factors for non-negative int n

[Rémi Lapeyre]

Rémi Lapeyre  added the comment:

>> Regardless, how can we best move forward with this idea?

> Provide a pull request.

Hi, I looked into what scientific programs where doing. Most of them uses a 
form of the Baillie–PSW primality test which is a probabilistic primality test 
that's never wrong up to 2**64 and for which their is currently no known 
pseudoprime.

This first version I wrote uses a deterministic variant of the Miller-Rabin 
test for n < 3317044064679887385961981. For larger n, a probabilistic 
Miller-Rabin test is used with s=25 random bases. The probabilistic 
Miller-Rabin test is never wrong when it concludes that n is composite and has 
a probability of error of 4^(-s) when it concludes that n is prime.

The implementations of next_prime() and previous_prime() are straightforward 
and factorise() uses the Phollard's rho heuristic which gives satisfactory 
results for numbers with small factors. It's a generator as it may hang when n 
has very large factors e.g. 2**(2**8)+1.

I implemented all functions Steven D'Aprano suggested but did not bother with 
the sieve as Tim Peters already provided one in the python-ideas thread. The 
code is in imath for now but I can move it.

Hopefully this is enough to bikeshed the API, if this proposal is accepted I 
will write more tests and fix any bug found.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40028] Math module method to find prime factors for non-negative int n

[Rémi Lapeyre]

Rémi Lapeyre  added the comment:

I can't mark the issue as open thought, can someone do this?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40509] In argparse, allow REMAINDER(...) arguments in a mutually exclusive group

[Shani Armon]

New submission from Shani Armon :

Options specified with the REMAINDER nargs should be treated as optional.
And an empty list should be treated as default.

--
components: Library (Lib)
messages: 368126
nosy: Shani Armon, rhettinger
priority: normal
severity: normal
status: open
title: In argparse, allow REMAINDER(...) arguments in a mutually exclusive group
type: behavior
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36543] Remove old-deprecated ElementTree features (part 2)

[Stefan Behnel]

Stefan Behnel  added the comment:

>> But as I said, people don't read docs.

> After seven, eight years we should do our users a service and point out the 
> deprecation with a concrete deadline.

¯\_(ツ)_/¯

Honestly, let's just keep it. Maybe we can add an invisible 
PendingDeprecationWarning and wait if some static analysers pick it up. I 
wouldn't want to invest any more work into it than that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40028] Math module method to find prime factors for non-negative int n

[Steven D'Aprano]

Steven D'Aprano  added the comment:

Miller-Rabin is known to be deterministic for all N < 2**64 too.

To be pedantic, M-R is known to be deterministic if you check every 
value up to sqrt(N), or possibly 2*log(N) if the generalized Riemann 
hypothesis is true. The question is whether there is a smaller set of 
values that is sufficient to make M-R deterministic, and that has been 
proven for N up to 2**64.

Wikipedia has more details including some small sets of bases which are 
deterministic. There may be even smaller sets, but for N < 2**64, just 
12 M-R tests with the following set of bases is sufficient to give a 
deterministic result:

2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37.

https://en.wikipedia.org/wiki/Miller%E2%80%93Rabin_primality_test#Testing_against_small_sets_of_bases

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40509] In argparse, allow REMAINDER(...) arguments in a mutually exclusive group

[Shani Armon]

Change by Shani Armon :


--
keywords: +patch
pull_requests: +19233
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19919

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40028] Math module method to find prime factors for non-negative int n

[Christian Heimes]

Christian Heimes  added the comment:

I'm still not convinced that it's a good idea to add a general prime factor 
function to Python's standard library. IMO the feature is better suited for an 
external math library or crypto library.

If we are going to add a prime factor function, then we should consider 
existing implementations. OpenSSL has BN_generate_prime_ex() [1] API. It's 
based on MR probabilistic prime test. The API can also generate primes with 
additional properties, e.g. Sophie Germain primes or primes suitable for finite 
field Diffie-Hellman.

[1] https://www.openssl.org/docs/man1.1.1/man3/BN_generate_prime_ex.html

--
nosy: +christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40510] [regression] ZipFile fails to round trip on some files

[David Naylor]

New submission from David Naylor :

With commit 18ee29d0b8 [1] a change was introduced that prevents a round-trip 
of some zip files (i.e. files generated by Microsoft Excel) due to the 
clobbering of `ZipInfo.flag_bits`[2] and `external_attr`[3].  

For example:
```python[name=zip-round-trip.py]
#!/usr/bin/env python3
import io
import sys
import zipfile

compression = zipfile.ZIP_STORED

source = sys.stdin
dest = sys.stdout

with io.BytesIO(source.buffer.read()) as source, io.BytesIO() as buffer:
with zipfile.ZipFile(source, "r") as source_zip, zipfile.ZipFile(buffer, 
"w") as dest_zip:
dest_zip.comment = source_zip.comment
for info in source_zip.infolist():
content = source_zip.read(info)
dest_zip.writestr(info, content, compression)

buffer.seek(0)
dest.buffer.write(buffer.read())
```

```shell
> python3.5 zip-round-trip.py < Book1.zip > Python.zip
> diff Book1.zip Python.zip
> python3.6 zip-round-trip.py < Book1.zip > Python.zip
> diff Book1.zip Python.zip
Binary files Book1.zip and Python.zip differ
```

[1] 
https://github.com/python/cpython/commit/18ee29d0b870caddc0806916ca2c823254f1a1f9
[2] 
https://github.com/python/cpython/blob/f01d1be97d740ea0369379ca305646a26694236e/Lib/zipfile.py#L1579
[3] 
https://github.com/python/cpython/blob/f01d1be97d740ea0369379ca305646a26694236e/Lib/zipfile.py#L1586

--
components: Library (Lib)
files: Book1.zip
messages: 368130
nosy: DragonSA
priority: normal
severity: normal
status: open
title: [regression] ZipFile fails to round trip on some files
type: behavior
versions: Python 3.6
Added file: https://bugs.python.org/file49121/Book1.zip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36543] Remove old-deprecated ElementTree features (part 2)

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +19235
stage: resolved -> patch review
pull_request: https://github.com/python/cpython/pull/19920

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36453] pkgutil.get_importer only return the first valid path_hook(importer)

[Christian Heimes]

Change by Christian Heimes :


--
keywords: +patch
nosy: +christian.heimes
nosy_count: 1.0 -> 2.0
pull_requests: +19236
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19921

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40028] Math module method to find prime factors for non-negative int n

[Mark Dickinson]

Mark Dickinson  added the comment:

Reopening as requested.

Adding a new module to the standard library is a fairly big change. I don't 
think we can do that through just this issue and PR. I think we're going to 
need a PEP sooner or later. (In any case, a PEP is standard procedure for a new 
standard library module.)

--
resolution: postponed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40028] Math module method to find prime factors for non-negative int n

[Mark Dickinson]

Mark Dickinson  added the comment:

Here's the relevant part of the dev. guide:

   https://devguide.python.org/stdlibchanges/#proposal-process

The PEP would also need a core dev sponsor.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36543] Remove old-deprecated ElementTree features (part 2)

[Christian Heimes]

Change by Christian Heimes :


--
pull_requests: +19237
pull_request: https://github.com/python/cpython/pull/19921

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40511] Yellow hint frame blinks when entering () in strings

[wyz23x2]

New submission from wyz23x2 :

The yellow frame blinks as shown in mp4. It's annoying and isn't good to the 
eyes.

--
assignee: terry.reedy
components: IDLE
messages: 368133
nosy: terry.reedy, wyz23x2
priority: normal
severity: normal
status: open
title: Yellow hint frame blinks when entering () in strings
type: behavior
versions: Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40511] Yellow hint frame blinks when entering () in strings

[wyz23x2]

wyz23x2  added the comment:

OMG, request too large. I can't upload mp4 /(ㄒoㄒ)/~~

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40511] IDLE yellow hint frame blinks when entering () in strings

[wyz23x2]

Change by wyz23x2 :


--
title: Yellow hint frame blinks when entering () in strings -> IDLE yellow hint 
frame blinks when entering () in strings

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40511] IDLE yellow hint frame blinks when entering () in strings in functions/classes

[wyz23x2]

Change by wyz23x2 :


--
title: IDLE yellow hint frame blinks when entering () in strings -> IDLE yellow 
hint frame blinks when entering () in strings in functions/classes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36543] Remove old-deprecated ElementTree features (part 2)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I do not think that it is right to emit a warning at import time. We should not 
punish people which do things right: fallback to xml.etree.ElementTree and test 
with -We.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1635741] Py_Finalize() doesn't clear all Python objects at exit

[Dong-hee Na]

Change by Dong-hee Na :


--
pull_requests: +19238
pull_request: https://github.com/python/cpython/pull/19923

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40512] Meta issue: per-interpreter GIL

[STINNER Victor]

New submission from STINNER Victor :

To be able to run multiple (sub)interpreters in parallel, the unique global 
interpreter lock aka "GIL" should be replace with multiple GILs: one "GIL" per 
interpreter. The scope of such per-interpreter GIL would be a single 
interpreter.

The current CPython code base is not fully read to have one GIL per 
interpreter. TODO:

* Move signals pending and gil_drop_request from _PyRuntimeState.ceval to 
PyInterpreterState.ceval: 
https://github.com/ericsnowcurrently/multi-core-python/issues/34
* Add a lock to pymalloc, or disable pymalloc when subinterpreters are used: 
https://github.com/ericsnowcurrently/multi-core-python/issues/30
* Make free lists per interpreters: tuple, dict, frame, etc.
* Make Unicode interned strings per interpreter
* Make Unicode latin1 single character string singletons per interpreter
* None, True, False, ... singletons: make them per-interpreter (bpo-39511) or 
immortal (bpo-40255)
* etc.

Until we can ensure that no Python object is shared between two interpreters, 
we might make PyObject.ob_refcnt, PyGC_Head (_gc_next and _gc_prev) and 
_dictkeysobject.dk_refcnt atomic.

C extension modules should be modified as well:

* Convert to PEP 489 multi-phase initialization
* Replace globals ("static" variables) with a module state, or design a new 
"per-interpreter" local storage similar to Thread Local Storage (TLS). There is 
already PyInterpreterState.dict which is convenient to use in "Python" code, 
but it's not convenient to use in "C" code (code currently using "static int 
..." for example).

I'm not sure how to handle C extensions which are binding for a C library which 
has a state and so should not be used multiple times in parallel. Some C 
extensions use a "global lock" for that. The question is how to get 

Most of these tasks are already tracked in Eric Snow's "Multi Core Python" 
project:
https://github.com/ericsnowcurrently/multi-core-python/issues

This issue is related to PEP 554 "Multiple Interpreters in the Stdlib", but not 
required by this PEP.

This issue is a tracker for sub-issues related to the goal "have one GIL per 
interpreter".

--

Some changes have a negative impact on "single threaded" Python application. 
Even if the overhead is low, one option to be able to move faster on this issue 
may be to add a new temporary configure option to have an opt-in build mode to 
better isolate subinterpreters. Examples:

* disable pymalloc
* atomic reference counters
* disable free lists

That would be a temporary solution to "unblock" the development on this list. 
For the long term, free lists should be made per-interpreter, pymalloc should 
support multiple interpreters, no Python object must be shared by two 
interpreters, etc.

--

One idea to detect if a Python object is shared by two interpreters *in debug 
mode* would be to store a reference to the interpreter which created it, and 
then check if the current interpreter is the same. If not, fail with a Python 
Fatal Error.

--

During Python 3.9 development cycle, many states moved from the global 
_PyRuntimeState to per-interpreter PyInterpreterState:

* GC state (bpo-36854)
* warnings state (bpo-36737)
* small integer singletons (bpo-38858)
* parser state (bpo-36876)
* ceval pending calls and "eval breaker" (bpo-39984)
* etc.

Many corner cases related to daemon threads have also been fixed:

* https://vstinner.github.io/daemon-threads-python-finalization-python32.html
* https://vstinner.github.io/threading-shutdown-race-condition.html
* https://vstinner.github.io/gil-bugfixes-daemon-threads-python39.html

And more code is now shared for the initialization and finalization of the main 
interpreter and subinterpreters (ex: see bpo-38858).

Subinterpreters builtins and sys are now really isolated from the main 
interpreter (bpo-38858).

--

Obviously, there are likely tons of other issues which are not known at this 
stage. Again, this issue is a placeholder to track them all. It may be more 
efficient to create one sub-issue per sub-task, rather than discussing all 
tasks at the same place.

--
components: Interpreter Core
messages: 368136
nosy: vstinner
priority: normal
severity: normal
status: open
title: Meta issue: per-interpreter GIL
type: enhancement
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

New submission from STINNER Victor :

To get one GIL per interpreter (bpo-40512), remaining _PyRuntimeState.ceval 
members should be moved to PyInterpreterState.ceval. The work started in 
bpo-39984 with ceval "pending calls" and ceval "eval breaker".

There are 4 remaining fields:

struct _ceval_runtime_state {
int recursion_limit;
/* Request for dropping the GIL */
_Py_atomic_int gil_drop_request;
/* Request for checking signals. */
_Py_atomic_int signals_pending;
struct _gil_runtime_state gil;
};

The most complex part will be to move the "gil" member: bpo-40512 lists all 
sub-tasks which must be completed before being able to have one GIL per 
interpreter without crashing.

--
components: Interpreter Core
messages: 368137
nosy: vstinner
priority: normal
severity: normal
status: open
title: Move _PyRuntimeState.ceval to PyInterpreterState
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40512] Meta issue: per-interpreter GIL

[STINNER Victor]

STINNER Victor  added the comment:

> Move signals pending and gil_drop_request from _PyRuntimeState.ceval to 
> PyInterpreterState.ceval: 
> https://github.com/ericsnowcurrently/multi-core-python/issues/34

I created bpo-40513: "Move _PyRuntimeState.ceval to PyInterpreterState".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +19239
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19924

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

STINNER Victor  added the comment:

See also: https://github.com/ericsnowcurrently/multi-core-python/issues/34

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40028] Math module method to find prime factors for non-negative int n

[Steven D'Aprano]

Steven D'Aprano  added the comment:

Speaking of OpenSSL, a few years ago this paper came out about OpenSSL's 
vulnerability to adversarial composites. Quote:

"As examples of our findings, weare able to construct 2048-bit 
composites that are declared prime with probability 1/16 byOpenSSL’s 
primality testing in its default configuration; the advertised 
performance is 2^−80. We can also construct 1024-bit composites 
that always pass the primality testing routine in GNU GMP when 
configured with the recommended minimum number of rounds."

https://eprint.iacr.org/2018/749.pdf

The paper discusses various languages, libraries, crypto toolkits, 
computer algebra systems, etc, including some Python libraries, and 
shows how many of them are vulnerable to adversarial composites. With 
some of them, the authors were able to defeat the isprime function 100% 
of the time.

My take on this is as follows:

For 64-bit ints, a deterministic set of M-R bases is sufficient (since 
it's deterministic there's no way to fool it into passing a composite as 
prime).

For ints with more than 64-bits, the authors suggest either:

- a minimum of one M-R test with base 2, followed by 1 Lucas test (this 
  is equivalent to a Baillie-PSW test; there are currently no known 
  Baillie-PSW pseudoprimes and no known adversarily attacks against it;

  (unless the NSA has some, but if so, they aren't saying)

- a default of 64 rounds with randomly choosen Miller-Rabin bases.

Presumably doing trial division on larger numbers to weed out the easy 
cases is acceptable too :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40514] Add --experimental-isolated-subinterpreters build option

[STINNER Victor]

New submission from STINNER Victor :

Modifying Python internals to have one GIL per interpreter (bpo-40512) is a 
large project which requires to modify many small things (again, see bpo-40512).

I propose to add a temporary build --experimental-isolated-subinterpreters 
build option to configure for developers who want to hack on isolated 
subinterpreters. The intent is to speedup bpo-40512 development by making some 
practice compromises, to have more time to properly design the real fixes.

For example, tuple, dict and frame have free lists which are shared by 
subinterpreters. A practical solution is to simply disable them at build time 
to avoid the need to have per-interpreter free lists.

Another example is pymalloc which is shared by all subinterpreters and rely on 
the unique global interpreter lock (GIL) to protect its internal states. A 
practical solution is to disable it and force the usage of libc malloc() 
function instead.

Some compromosises cannot be the default since they have a significant negative 
impact on performances. So I propose to add a temporary build option until all 
these small issues will be fixed.

--
components: Build
messages: 368141
nosy: vstinner
priority: normal
severity: normal
status: open
title: Add --experimental-isolated-subinterpreters build option
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40512] Meta issue: per-interpreter GIL

[STINNER Victor]

STINNER Victor  added the comment:

> Some changes have a negative impact on "single threaded" Python application. 
> Even if the overhead is low, one option to be able to move faster on this 
> issue may be to add a new temporary configure option to have an opt-in build 
> mode to better isolate subinterpreters. (...)

I created bpo-40514: "Add --experimental-isolated-subinterpreters build option".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40515] test_ssl.py hangs with SSL 1.1 built with no threads

[Miguel]

New submission from Miguel :

Hello, this is my first python bug report!

I've been running builds of Python 3.7.x on CentOS Linux release 7.7 
(64bit/Intel Core
2 Duo) 
and I ran into hangs with test_ssl.py when using latest SSL 1.1.1d sources.

I've done a full compilation from source for Python 3.7.7 and SSL 1.1 in my
workspaces.

>From what I can tell the problem is when SSL 1.1 is built with no threading
there is no locking enabled by python.

This one line change will make the hangs in test_ssl.py  go away:

Index: Modules/_ssl.c

--- Modules/_ssl.c  (revision 70)
+++ Modules/_ssl.c  (working copy)
@@ -5875,7 +5875,7 @@
 if (!_setup_ssl_threads()) {
 return NULL;
 }
-#elif OPENSSL_VERSION_1_1 && defined(OPENSSL_THREADS)
+#elif OPENSSL_VERSION_1_1
 / OpenSSL 1.1.0 builtin thread support is enabled /
 _ssl_locks_count++;

# endif

There appears to be an assumption in _ssl.c and test_ssl.y that SSL 1.1 will be
threaded
but this may not be true (as in my case).

Appreciate any feedback. Thanks!

--
assignee: christian.heimes
components: SSL
messages: 368143
nosy: christian.heimes, mig28suarez
priority: normal
severity: normal
status: open
title: test_ssl.py hangs with SSL 1.1 built with no threads
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40028] Math module method to find prime factors for non-negative int n

[Christian Heimes]

Christian Heimes  added the comment:

It seems we agree that prime functions only appear to be easy until you realize 
that they are far from trivial. :)

+1 to require a PEP. I'm happy to give my feedback on crypto and 
security-related part of the feature.

OpenSSL now uses 64 MR rounds for small and 128 for larger primes 
(https://github.com/openssl/openssl/blob/278260bfa238aefef5a1abe2043d2f812c3a4bd5/crypto/bn/bn_prime.c#L87-L99)
 and trial divisions to update to 2048 tests 
(https://github.com/openssl/openssl/blob/278260bfa238aefef5a1abe2043d2f812c3a4bd5/crypto/bn/bn_prime.c#L70-L85).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40028] Math module method to find prime factors for non-negative int n

[Christian Heimes]

Change by Christian Heimes :


--
stage: resolved -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40515] test_ssl.py hangs with SSL 1.1 built with no threads

[Christian Heimes]

Christian Heimes  added the comment:

How did you build OpenSSL 1.1.1d? Python assumes default build of OpenSSL. 
Since Python is now always multi-threaded we no longer support non-threaded 
OpenSSL builds.

--
versions:  -Python 3.5, Python 3.6, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40516] GCC 9 compiler warnings on MacOS Catalina

[Rémi Lapeyre]

New submission from Rémi Lapeyre :

Building master with GCC 9.3.0 gives some warnings on posix and nis:

./Modules/posixmodule.c: In function 'os_chown_impl':
./Modules/posixmodule.c:3397:39: warning: the comparison will always evaluate 
as 'false' for the address of 'lchown' will never be NULL [-Waddress]
 3397 | if ((!follow_symlinks) && (lchown == NULL)) {
  |   ^~
./Modules/posixmodule.c: In function 'PyInit_posix':
./Modules/posixmodule.c:14939:16: warning: the comparison will always evaluate 
as 'false' for the address of 'lchown' will never be NULL [-Waddress]
14939 | if (lchown == NULL) {
  |^~
/Users/remi/src/cpython/Modules/nismodule.c: In function 'nis_cat':
/Users/remi/src/cpython/Modules/nismodule.c:216:18: warning: cast between 
incompatible function types from 'int (*)(int,  char *, int,  char *, int,  
struct ypcallback_data *)' to 'int (*)(long unsigned int,  char *, int,  char 
*, int,  void *)' [-Wcast-function-type]
  216 | cb.foreach = (foreachfunc)nis_foreach;
  |  ^
/Users/remi/src/cpython/Modules/nismodule.c: In function 'nis_xdr_ypmaplist':
/Users/remi/src/cpython/Modules/nismodule.c:302:42: warning: cast between 
incompatible function types from 'int (*)(XDR *, nismaplist *)' {aka 'int 
(*)(struct __rpc_xdr *, struct nismaplist *)'} to 'int (*)(XDR *, void *, 
unsigned int)' {aka 'int (*)(struct __rpc_xdr *, void *, unsigned int)'} 
[-Wcast-function-type]
  302 |  sizeof(nismaplist), (xdrproc_t)nis_xdr_ypmaplist))
  |  ^
/Users/remi/src/cpython/Modules/nismodule.c: In function 
'nis_xdr_ypresp_maplist':
/Users/remi/src/cpython/Modules/nismodule.c:328:42: warning: cast between 
incompatible function types from 'int (*)(XDR *, nismaplist *)' {aka 'int 
(*)(struct __rpc_xdr *, struct nismaplist *)'} to 'int (*)(XDR *, void *, 
unsigned int)' {aka 'int (*)(struct __rpc_xdr *, void *, unsigned int)'} 
[-Wcast-function-type]
  328 |  sizeof(nismaplist), (xdrproc_t)nis_xdr_ypmaplist))
  |  ^
In file included from 
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/rpc/rpc.h:76,
 from /Users/remi/src/cpython/Modules/nismodule.c:17:
/Users/remi/src/cpython/Modules/nismodule.c: In function 'nisproc_maplist_2':
/Users/remi/src/cpython/Modules/nismodule.c:344:19: warning: cast between 
incompatible function types from 'int (*)(XDR *, char **)' {aka 'int (*)(struct 
__rpc_xdr *, char **)'} to 'int (*)(XDR *, void *, unsigned int)' {aka 'int 
(*)(struct __rpc_xdr *, void *, unsigned int)'} [-Wcast-function-type]
  344 |   (xdrproc_t)nis_xdr_domainname, (caddr_t)argp,
  |   ^
/Users/remi/src/cpython/Modules/nismodule.c:345:19: warning: cast between 
incompatible function types from 'int (*)(XDR *, nisresp_maplist *)' {aka 'int 
(*)(struct __rpc_xdr *, struct nisresp_maplist *)'} to 'int (*)(XDR *, void *, 
unsigned int)' {aka 'int (*)(struct __rpc_xdr *, void *, unsigned int)'} 
[-Wcast-function-type]
  345 |   (xdrproc_t)nis_xdr_ypresp_maplist, (caddr_t)&res,
  |   ^


I'm not sure about the fix, especially for nis but I will propose a PR. There 
is also some warnings concerning libffi_osx but this looks like a vendored 
dependency so I left it untouched.

--
components: Build
messages: 368146
nosy: remi.lapeyre
priority: normal
severity: normal
status: open
title: GCC 9 compiler warnings on MacOS Catalina
type: behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40516] GCC 9 compiler warnings on MacOS Catalina

[Rémi Lapeyre]

Change by Rémi Lapeyre :


--
keywords: +patch
pull_requests: +19240
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19925

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40514] Add --experimental-isolated-subinterpreters build option

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +19241
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19926

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 4e01946cafca0cf49f796c3118e0d65237bcad69 by Victor Stinner in 
branch 'master':
bpo-40513: Per-interpreter signals pending (GH-19924)
https://github.com/python/cpython/commit/4e01946cafca0cf49f796c3118e0d65237bcad69


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40517] Syntax highlighting for ASDL

[Batuhan Taskaya]

New submission from Batuhan Taskaya :

ASDL is around here for a long time, and it was showed as raw text on 
documentation (under library/ast), IMHO it would be great to highlight it.

--
assignee: docs@python
components: Documentation
messages: 368148
nosy: BTaskaya, docs@python, eric.araujo, ezio.melotti, mdk, willingc
priority: normal
severity: normal
status: open
title: Syntax highlighting for ASDL
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +19242
pull_request: https://github.com/python/cpython/pull/19927

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40517] Syntax highlighting for ASDL

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
keywords: +patch
pull_requests: +19243
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19928

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40518] ValueError when using resource.setrlimit on macOS Catalina

[Shota Iwamoto]

New submission from Shota Iwamoto :

Hello, I'm having trouble when using `resource.setrlimit` on macOS.

```
import resource
resource.setrlimit(resource.RLIMIT_STACK, 
resource.getrlimit(resource.RLIMIT_STACK))
```

Running this code gives the following error:

```
Traceback (most recent call last):
  File "main.py", line 2, in 
resource.setrlimit(resource.RLIMIT_STACK, 
resource.getrlimit(resource.RLIMIT_STACK))
ValueError: current limit exceeds maximum limit
```

I ran the same code in my linux environment and it worked.

* macOS Catalina 10.15.4 (19E287)
* `python --version` : Python 3.7.7

Thanks.

--
components: macOS
messages: 368149
nosy: morioprog, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: ValueError when using resource.setrlimit on macOS Catalina
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

STINNER Victor  added the comment:

Python/ceval.c also has "int _Py_CheckRecursionLimit = 
Py_DEFAULT_RECURSION_LIMIT;". _Py_CheckRecursiveCall() has this comment:

/* Needed for ABI backwards-compatibility (see bpo-31857) */
_Py_CheckRecursionLimit = recursion_limit;

I converted Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() macros into 
opaque functions in bpo-38644: commit f4b1e3d7c64985f5d5b00f6cc9a1c146bbbfd613.

I don't think that the "ABI backwards-compatibility" rationale is correct. 
These macros never worked in the limited C API: they accessed 
PyThreadState.recursion_depth and PyThreadState.overflowed members, whereas the 
PyThreadState structure is opaque in the limited C API (on purpose, see also 
bpo-39947).

I think that it is now safe to simply remove _Py_CheckRecursionLimit.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40518] ValueError when using resource.setrlimit on macOS Catalina

[Dong-hee Na]

Change by Dong-hee Na :


--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 0b1e3307e24b0af45787ab6456535b8346e0239a by Victor Stinner in 
branch 'master':
bpo-40513: Per-interpreter gil_drop_request (GH-19927)
https://github.com/python/cpython/commit/0b1e3307e24b0af45787ab6456535b8346e0239a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40514] Add --experimental-isolated-subinterpreters build option

[Dong-hee Na]

Change by Dong-hee Na :


--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[hai shi]

Change by hai shi :


--
nosy: +shihai1991

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +19244
pull_request: https://github.com/python/cpython/pull/19929

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40519] Preserve AttributeError exception context in __getattr__

[Arusekk]

New submission from Arusekk :

This is another attempt at issue 39865, but with a different attitude. Please 
see that issue for some extra motivation for this change.

My suggestion is to change getattr logic, which now is (in this case):

def getattr(obj, attr):
try:
return normal_getattr(obj, attr)
except AttributeError:
pass
return obj.__getattr__(attr)

to be more like:

def getattr(obj, attr):
try:
return normal_getattr(obj, attr)
except AttributeError:
return obj.__getattr__(attr)

A __getattr__ function would then be able to know the exception context
(through sys.exc_info()) and to have the context automatically attached to all 
the exceptions raised (still respecting raise ... from None).

This particular issue only lies in Objects/typeobject.c, but is probably valid 
for other uses of PyErr_Clear() in the interpreter.
I checked some using a simple shell pipeline:

$ grep -r -A5 PyErr_ExceptionMatches |grep -C5 PyErr_Clear

And found some interesting examples of what be worth looking into:
Python/sysmodule.c:708
Parser/tokenizer.c:1110
Objects/memoryobject.c:fix_error_int

I prepared two patches for this (please forgive if I violated code style 
somewhere, I am not saying that they are a final version ready to be merged): a 
simple one, addressing just this very issue, and a robust one, allowing other 
places (e.g. from the list above) to reuse it.

--
components: Interpreter Core
files: typeobject.patch
keywords: patch
messages: 368152
nosy: Arusekk, ammar2, pasenor
priority: normal
severity: normal
status: open
title: Preserve AttributeError exception context in __getattr__
type: behavior
versions: Python 3.9
Added file: https://bugs.python.org/file49122/typeobject.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

STINNER Victor  added the comment:

For signals_pending, see also: bpo-40010 "Inefficient signal handling in 
multithreaded applications" which added _Py_ThreadCanHandleSignals() and 
_Py_ThreadCanHandlePendingCalls() functions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40519] Preserve AttributeError exception context in __getattr__

[Arusekk]

Arusekk  added the comment:

Feel free to reuse the patches if you have better ideas

--
Added file: https://bugs.python.org/file49123/robust.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40515] test_ssl.py hangs with SSL 1.1 built with no threads

[Miguel]

Miguel  added the comment:

I built with 'no-threads' option.
I understand if you no longer wish to support non-threaded SSL. 
But it just seemed to me that you could if you protected the SSL API calls with 
locking since I can get all your ssl tests to work with this minor change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40514] Add --experimental-isolated-subinterpreters build option

[STINNER Victor]

STINNER Victor  added the comment:


New changeset c5fa364f4ea836f25dd07cfb328152d40a568371 by Victor Stinner in 
branch 'master':
bpo-40514: Add --with-experimental-isolated-subinterpreters (GH-19926)
https://github.com/python/cpython/commit/c5fa364f4ea836f25dd07cfb328152d40a568371


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40462] [easy] undefined name in Lib/test/mock_socket.py

[STINNER Victor]

STINNER Victor  added the comment:

> IMHO we should create backport patch for 3.8 and 3.7

It's deadcode, I don't think that it's worth it. But it's up to you. If you 
consider that it's worth it, go ahead and backport the fix.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40520] port the declartions in pydebug.h to initconfig.h

[hai shi]

New submission from hai shi :

Nick left a comment in `pydebug.h`
/* These global variable are defined in pylifecycle.c */
/* XXX (ncoghlan): move these declarations to pylifecycle.h? */

I have checked those global variables are not only debug variables and they are 
defined in `Python/initconfig.c` now.
If I understand clearly, those declarations should moved to 
`Include/cpython/initconfig.h` and `pydebug.h` could be deleted.

ref: 
https://github.com/python/cpython/commit/d600951748d7a19cdb3e58a376c51ed804b630e6

--
components: Interpreter Core
messages: 368158
nosy: ncoghlan, shihai1991, vstinner
priority: normal
severity: normal
status: open
title: port the declartions in pydebug.h to initconfig.h
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40514] Add --experimental-isolated-subinterpreters build option

[STINNER Victor]

STINNER Victor  added the comment:

--with-experimental-isolated-subinterpreters option name is very long on 
purpose: to advertize that you must not use it, unless you fully understand its 
purpose :-)

I didn't document the build option in Misc/SpecialBuilds.txt or 
Doc/whatsnew/3.9.rst on purpose: you should not use this special build :-) 
Basically, this option is only for Eric Snow and me :-D

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40520] port the declartions in pydebug.h to initconfig.h

[hai shi]

Change by hai shi :


--
keywords: +patch
pull_requests: +19245
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19930

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40520] port the declartions in pydebug.h to initconfig.h

[STINNER Victor]

STINNER Victor  added the comment:

Global configuration variables are part of the C API. They are now documented:
https://docs.python.org/dev/c-api/init.html#global-configuration-variables

They are used to populate PyConfig (PEP 587). At startup, the PyConfig is 
copied into these global configuration variables. After the Python 
initilization, they are no longer read, but PyConfig (tstate->interp->config) 
is read instead.

You can simply remove the two lines comment from pydebug.h.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32117] Tuple unpacking in return and yield statements

[miss-islington]

miss-islington  added the comment:


New changeset 627f7012353411590434a7d5777ddcbcc8d97fcd by Javier Buzzi in 
branch 'master':
bpo-32117: Updated Simpsons names in docs (GH-19737)
https://github.com/python/cpython/commit/627f7012353411590434a7d5777ddcbcc8d97fcd


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 4e30ed3af06ae655f4cb8aad8cba21f341384250 by Victor Stinner in 
branch 'master':
bpo-40513: Per-interpreter recursion_limit (GH-19929)
https://github.com/python/cpython/commit/4e30ed3af06ae655f4cb8aad8cba21f341384250


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40520] port the declartions in pydebug.h to initconfig.h

[hai shi]

Change by hai shi :


--
pull_requests: +19246
pull_request: https://github.com/python/cpython/pull/19931

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40520] port the declartions in pydebug.h to initconfig.h

[hai shi]

hai shi  added the comment:

Got it, thanks for your explanation, victor ;)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +19247
pull_request: https://github.com/python/cpython/pull/19932

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29587] Generator/coroutine 'throw' discards exc_info state, which is bad

[STINNER Victor]

STINNER Victor  added the comment:


New changeset b0be6b3b94fbdf31b796adc19dc86a04a52b03e1 by Victor Stinner in 
branch 'master':
bpo-29587: _PyErr_ChainExceptions() checks exception (GH-19902)
https://github.com/python/cpython/commit/b0be6b3b94fbdf31b796adc19dc86a04a52b03e1


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[Eric Snow]

Eric Snow  added the comment:

FWIW, I think it would make sense to keep "signals_pending" under 
_PyRuntimeState rather than moving it to PyInterpreterState.

Signals are only handled by the main interpreter (in its main thread).  Even 
though "signals_pending" is useful to only one interpreter in the runtime, 
making it per-interpreter sends the wrong message that it is significant at 
that level.  This may be confusing to readers of the code.

At the very least there should be a clear comment with the field in 
Include/internal/pycore_interp.h explaining how it is only used by the main 
interpreter and why we made it per-interpreter anyway.

--
nosy: +eric.snow

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40514] Add --experimental-isolated-subinterpreters build option

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +eric.snow

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40512] Meta issue: per-interpreter GIL

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +eric.snow

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40520] port the declartions in pydebug.h to initconfig.h

[miss-islington]

miss-islington  added the comment:


New changeset 6351d9e4400a77fe1fcbe4f03e5fb6620cca236d by Hai Shi in branch 
'master':
bpo-40520: Remove redundant comment in pydebug.h (GH-19931)
https://github.com/python/cpython/commit/6351d9e4400a77fe1fcbe4f03e5fb6620cca236d


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40520] Remove redundant comment in pydebug.h

[Dong-hee Na]

Change by Dong-hee Na :


--
title: port the declartions in pydebug.h to initconfig.h -> Remove redundant 
comment in pydebug.h

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40520] Remove redundant comment in pydebug.h

[Dong-hee Na]

Dong-hee Na  added the comment:

Thanks hai shi :)

--
nosy: +corona10
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40512] Meta issue: per-interpreter GIL

[Dong-hee Na]

Change by Dong-hee Na :


--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[Eric Snow]

Eric Snow  added the comment:

>From a user perspective, does it make sense to have a different 
>recursion_limit per interpreter?  I don't see a problem with it.  However, 
>does it make sense to also keep a global value that we default to when a 
>per-interpreter value is not set?  I think it might.

I suppose a bigger question is what users will expect the recursion limit (AKA 
"sys.getrecursionlimit()") to be for a newly created subinterpreter.  Will it 
be some global default?  Will it be the value from the parent interpreter?  I'd 
go with a global default, which would imply that the default value should be 
stored under _PyRuntimeState like we had it (but still keep the actual 
per-interpreter field for the actual value).

FWIW, the existing docs don't really block either approach.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[Eric Snow]

Eric Snow  added the comment:

If this issue covers the GIL (which it seems to) then I'd expect 
_PyRuntimeState.gilstate to be handled here too.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40520] Remove redundant comment in pydebug.h

[hai shi]

hai shi  added the comment:

Thanks, Dong-hee Na ;)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40010] Inefficient signal handling in multithreaded applications

[Eric Snow]

Eric Snow  added the comment:

Good catch on this, Victor.  Thanks for doing it.

--
nosy: +eric.snow

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40514] Add --experimental-isolated-subinterpreters build option

[Eric Snow]

Eric Snow  added the comment:

It would probably make sense to remove the build option in the 3.9 release.  We 
can leave it in master, but remove it in the 3.9 branch once it has been 
created.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40379] multiprocessing's default start method of fork()-without-exec() is broken

[Itamar Turner-Trauring]

Itamar Turner-Trauring  added the comment:

Just got an email from someone for whom switching to "spawn" fixed a problem. 
Earlier this week someone tweeted about this fixing things. This keeps hitting 
people in the real world.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40513] Move _PyRuntimeState.ceval to PyInterpreterState

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 299b8c61e9d1a42b929b8deb1b05067876e191e6 by Victor Stinner in 
branch 'master':
Revert "bpo-40513: Per-interpreter signals pending (GH-19924)" (GH-19932)
https://github.com/python/cpython/commit/299b8c61e9d1a42b929b8deb1b05067876e191e6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40521] Make tuple, dict, frame free lists, unicode interned strings, unicode latin1 singletons per-interpreter

[STINNER Victor]

New submission from STINNER Victor :

tuple, dict and frame use free lists to optimize the creation of objects.

Unicode uses "interned" strings to reduce the Python memory footprint and 
speedup dictionary lookups.

Unicode also uses singletons for single letter Latin1 characters ([U+; 
U+00FF] range).

All these optimizations are incompatible with isolated subinterpreters, since 
caches are currently shared by all inteprepreters. These caches should be made 
per-intepreter. See bpo-40512 "Meta issue: per-interpreter GIL" for the 
rationale.

I already made small integer singletons per interpreter in bpo-38858:

* commit 5dcc06f6e0d7b5d6589085692b86c63e35e2325e
* commit 630c8df5cf126594f8c1c4579c1888ca80a29d59.

--
components: Interpreter Core
messages: 368175
nosy: vstinner
priority: normal
severity: normal
status: open
title: Make tuple, dict, frame free lists, unicode interned strings, unicode 
latin1 singletons per-interpreter
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40515] test_ssl.py hangs with SSL 1.1 built with no threads

[Christian Heimes]

Christian Heimes  added the comment:

It not about what I wish or wish not to do.

Python requires thread-safe libraries. A library without proper locking and 
thread safety is no longer safe to use without great effort and careful locking 
in the glue code. A non-threaded OpenSSL build will lead to memory corruption 
and eventually crash the interpreter.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40521] Make tuple, dict, frame free lists, unicode interned strings, unicode latin1 singletons per-interpreter

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +19248
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19933

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40417] PyImport_ReloadModule emits deprecation warning

[Robert Rouhani]

Change by Robert Rouhani :


--
pull_requests: +19249
pull_request: https://github.com/python/cpython/pull/19934

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40417] PyImport_ReloadModule emits deprecation warning

[Robert Rouhani]

Change by Robert Rouhani :


--
pull_requests: +19250
pull_request: https://github.com/python/cpython/pull/19935

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38787] PEP 573: Module State Access from C Extension Methods

[Petr Viktorin]

Change by Petr Viktorin :


--
pull_requests: +19251
pull_request: https://github.com/python/cpython/pull/19936

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com