[issue18813] Speed up slice object processing

[Stefan Behnel]

Stefan Behnel added the comment:

Here's another idea. There could be two implementations of "slice", one that 
uses Python object indices (as currently) and one that has Py_ssize_t indices  
(and properties for the start/stop/step attributes). Similar to what Unicode 
strings do, the slice type would decide which to use at instantiation time.

The internals of PySliceObject are not part of the stable ABI, and the normal 
way to figure out the indices is PySlice_GetIndicesEx(), which would be adapted 
accordingly.

This breaks compatibility with some C extensions that access the slice 
attributes directly, but that should be rare, given how complex index 
calculations are.

This change is certainly more invasive than adding Py_ssize_t fields to the 
existing object, though.

--
versions: +Python 3.5 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22880] hmac.new docs show optional args incorrectly

[Berker Peksag]

Berker Peksag added the comment:

Thanks for the report, but the signature is correct.

>From https://docs.python.org/devguide/documenting.html#information-units

"The signature should include the parameters, enclosing optional parameters 
in brackets."

See also http://sphinx-doc.org/domains.html#python-signatures

--
nosy: +berker.peksag
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18813] Speed up slice object processing

[Stefan Behnel]

Stefan Behnel added the comment:

Thanks for the review, Serhiy. There isn't really a reason not to 'normalise' 
the Python step object to None when 1 is passed from C code, so I updated the 
patch to do that.

Regarding on-demand creation of the Python values, the problem is that C code 
that accesses the struct fields directly would not be prepared to find NULL 
values in them, so this would be a visible change and potential source of 
crashes. However, my guess is that this would rarely be a problem (see my last 
comment on changing the slice type internally).

--
Added file: http://bugs.python.org/file37205/cache_slice_indices.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18813] Speed up slice object processing

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

> There could be two implementations of "slice", one that
> uses Python object indices (as currently) and one that has Py_ssize_t
> indices  (and properties for the start/stop/step attributes).

Agree, this idea LGTM. Single boolean flag should be enough to switch between 
implementations. This shouldn't break well written code. The PySliceObject 
structure is not a part of stable API.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22881] show median in benchmark results

[Stefan Behnel]

New submission from Stefan Behnel:

The median tends to give a better idea about benchmark results than an average 
as it inherently ignores outliers.

--
components: Benchmarks
files: show_median.patch
keywords: patch
messages: 231239
nosy: pitrou, scoder
priority: normal
severity: normal
status: open
title: show median in benchmark results
type: enhancement
Added file: http://bugs.python.org/file37206/show_median.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22881] show median in benchmark results

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

In case of even number of samples the median value is calculated as arithmetic 
mean of two middle samples.

med_base = (base_times[len(base_times)//2] + 
base_times[(len(base_times)+1)//2]) / 2

--
nosy: +serhiy.storchaka
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22881] show median in benchmark results

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
Removed message: http://bugs.python.org/msg231240

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22881] show median in benchmark results

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

In case of even number of samples the median value is calculated as arithmetic 
mean of two middle samples.

med_base = (base_times[len(base_times)//2] + 
base_times[(len(base_times)-1)//2]) / 2

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17293] uuid.getnode() MAC address on AIX

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Oh, socket.gaierror was not made an alias of OSError, only subclass of it. 
Well, we can apply the same patch to all releases. Does it fix tests?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22881] show median in benchmark results

[Stefan Behnel]

Stefan Behnel added the comment:

Fair enough, patch updated.

--
Added file: http://bugs.python.org/file37207/show_median.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18813] Speed up slice object processing

[Stefan Behnel]

Stefan Behnel added the comment:

I reran the benchmarks with the fast path in _PyEval_SliceIndex() and the 
results are not conclusive. There is no clear indication that it improves the 
performance.

The problem is that most slices have no step, many slices are open ended on at 
least one side (i.e. usually have one or two fields set to None) and they are 
only used once, so integer index optimisations can only have a limited impact 
compared to instantiating the slice object in the first place.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22882] [patch] Linux packages you need to compile Python with all dependencies

[Ludovic Gasc]

New submission from Ludovic Gasc:

Hi,

To install easily Python 3.4.2 on Linux, I use Pythonz: 
http://saghul.github.io/pythonz/

I've discovered that, depends on the packages already installed on Linux, I 
don't have the same Python each time after compilation on servers.
Some features could missing, like pip, if you don't install some packages.

I've made a pull request in Pythonz with the list of packages you need to avoid 
that: https://github.com/saghul/pythonz#before-installing-pythons-via-pythonz

But, for me, this packages list should be in official Python documentation 
directly.

I've tested this packages list on Debian (Squeeze, Wheezy, Jessie, Sid), Ubuntu 
(12.04, 14.04) and CentOS 6.

Thanks for the merge.

Regards.

--
assignee: docs@python
components: Documentation
files: build_dep_python.patch
keywords: patch
messages: 231245
nosy: Ludovic.Gasc, docs@python
priority: normal
severity: normal
status: open
title: [patch] Linux packages you need to compile Python with all dependencies
type: enhancement
versions: Python 3.4, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file37208/build_dep_python.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22883] Get rid of references to PyInt in Py3 sources

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

There are several references to PyInt in outdated comments in .c and .h files:

$ find * -name '*.[ch]' -exec egrep -n --color -- 'PyInt\b' '{}' +
Include/longobject.h:34:/* It may be useful in the future. I've added it in the 
PyInt -> PyLong
Modules/fcntlmodule.c:157:   into either a large positive number (PyLong or 
PyInt on 64-bit
Modules/fcntlmodule.c:158:   platforms) or a negative number on others 
(32-bit PyInt)
Modules/_json.c:813:PyInt, PyLong, or PyFloat.
Modules/itertoolsmodule.c:3869:assert(cnt != PY_SSIZE_T_MAX && long_cnt == 
NULL && long_step==PyInt(1));
Python/ceval.c:4606:/* Extract a slice index from a PyInt or PyLong or an 
object with the

--
components: Extension Modules, Interpreter Core
messages: 231246
nosy: mark.dickinson, serhiy.storchaka
priority: normal
severity: normal
stage: needs patch
status: open
title: Get rid of references to PyInt in Py3 sources
versions: Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18813] Speed up slice object processing

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Indeed, PySequence_GetSlice() is used only in few performance non-critical 
places in the stdlib, PySequence_Set/DelSlice() are not used at all. So looks 
as this way doesn't speed up any code.

As for faster_PyEval_SliceIndex.patch see issue17576. May be such optimization 
is not always correct (only for exact PyLong). And why you pass through fast 
path only if -1<= Py_SIZE(v) <=1?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22599] traceback: errors in the linecache module at exit

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
assignee:  -> haypo
dependencies: +Add a function to know about interpreter shutdown
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22696] Add a function to know about interpreter shutdown

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I'm +0 for is_finalizing.

There is a typo in the docstring.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22884] argparse.FileType.__call__ returns unwrapped sys.stdin and stdout

[Kevin Orr]

New submission from Kevin Orr:

When one uses a file object returned by `FileType.__call__` as a context 
manager, `sys.stdin`'s or `sys.stdout`'s `__exit__` will be triggered upon exit 
of the context, in turn calling their `close` method.

Perhaps the issue is that `sys.stdin` and `sys.stdout` have poor `__exit__` 
methods, but my proposal (and it's not a particularly clean one) is to override 
the file object's `__exit__` if it happens to be either `sys.stdin` or 
`sys.stdout` to simply return True when called.

--
components: Library (Lib)
messages: 231249
nosy: keviv
priority: normal
severity: normal
status: open
title: argparse.FileType.__call__ returns unwrapped sys.stdin and stdout
type: behavior
versions: Python 2.7, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17611] Move unwinding of stack for "pseudo exceptions" from interpreter to compiler.

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I like the idea. I have not make faultfinding review yet, but one thing is 
questionable to me.

Is it necessary to get rid of the END_FINALLY opcode?

Currently Python code

try:
stmt1
finally:
stmt2

is compiled to:

SETUP_FINALLY L1
// stmt1
POP_BLOCK
LOAD_CONST None
L1: // stmt2
END_FINALLY

With the patch it is compiled to:

SETUP_FINALLY L1
// stmt1
POP_BLOCK
// stmt2
JUMP_ABSOLUTE L2
L1: // stmt2
RERAISE
L2:

Finally body is duplicated. In case of large finally body this increases the 
size of the bytecode. Line numbers are duplicated in disassembly listing, this 
is confused. And I afraid that due to increasing the size of the bytecode, some 
relative jumps can became too long.

If it is absolutely necessary to remove the END_FINALLY opcode, I suggest to 
generate following bytecode:

SETUP_FINALLY L1
// stmt1
POP_BLOCK
LOAD_CONST None
L1: // stmt2
POP_JUMP_IF_FALSE L2
RERAISE
L2:

But it looks to me that special END_FINALLY opcode which combines 
POP_JUMP_IF_FALSE/RERAISE would be better for performance and compiled code 
size.

--
components: +Interpreter Core
stage:  -> patch review
type:  -> enhancement
versions: +Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22884] argparse.FileType.__call__ returns unwrapped sys.stdin and stdout

[Kevin Orr]

Kevin Orr added the comment:

Hmm, not sure why I thought I needed `inline code formatting`. It's all 
monospace anyway!

Anyway, I'm thinking that adding this somewhere in argparse.py:

class _WrappedIO(object):
def __init__(self, fileobj):
self._file = fileobj

def __exit__(tp, val, tb):
return True

def __getattr__(self, name):
return self._file.__gettattr__(name)

and then applying the attached patch *may* fix this.

--
keywords: +patch
Added file: http://bugs.python.org/file37209/argparse.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22880] hmac.new docs show optional args incorrectly

[R. David Murray]

R. David Murray added the comment:

To be clear: this was/is the python2 documentation style.  In python3 we've 
switched to using keyword argument notation in functions that support 
specifying optional arguments via keyword (which is many more of the C 
functions than in python2).

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22882] [patch] Linux packages you need to compile Python with all dependencies

[R. David Murray]

R. David Murray added the comment:

I don't think it is a good idea for us to try to track which packages are 
needed and what they are named by the various downstreams.  We could document 
the optional modules by project name, though.  Note also that the devguide does 
point to the "umbrella" downstream packages that pull in the dependencies 
needed to build python, where those exist.

It is likely that that section should cross link to the devguide.  The devguide 
didn't exist when it was written.

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22884] argparse.FileType.__call__ returns unwrapped sys.stdin and stdout

[R. David Murray]

R. David Murray added the comment:

Perhaps instead it could return an io object built from the file descriptor 
with closefd=False.

--
nosy: +r.david.murray
versions: +Python 3.4, Python 3.5 -Python 2.7, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22885] Arbitrary code execution vulnerability due to unchecked eval() call in dumbdbm module

[Stephen Farris]

New submission from Stephen Farris:

The dumbdbm module uses an unchecked call to eval() in the _update method, 
which is called in response to a call to dumbdbm.open(), and is used to load 
the index from the directory file.  This poses a security vulnerability because 
it allows an attacker to execute arbitrary code on the victim's machine by 
inserting python code into the DBM directory file.  This vulnerability could 
allow an attacker to execute arbitrary commands on the victim machine, 
potentially allowing them to deploy malware, gain system access, destroy files 
and data, expose sensitive information, etc.

--
components: Library (Lib)
messages: 231255
nosy: Guido.van.Rossum, lemburg, stephen.farris
priority: normal
severity: normal
status: open
title: Arbitrary code execution vulnerability due to unchecked eval() call in 
dumbdbm module
type: security
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18813] Speed up slice object processing

[Stefan Behnel]

Stefan Behnel added the comment:

> why you pass through fast path only if -1<= Py_SIZE(v) <=1?

It's usually enough, it's the fastest fast path, and it doesn't even need
error handling.

Any slicing where the slice calculation time matters is going to be of
fairly limited size, often involving the values (!) 1 or -1 in the slice
object fields, but definitely small integers.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22882] [patch] Linux packages you need to compile Python with all dependencies

[Ludovic Gasc]

Ludovic Gasc added the comment:

I'm not agree with you: I've lost a lot of time to find all packages I need for 
that, I've merged information from several blog posts.
I imagine that I'm not alone with this problem.

If you move this information in devguide, most persons don't find this.

Moreover, the names of packages are pretty stable across Linux versions, I 
don't think we will have a lot of exceptions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22882] [patch] Linux packages you need to compile Python with all dependencies

[Andrew Svetlov]

Andrew Svetlov added the comment:

For Ubuntu/debian you can just run:
$ sudo apt-get build-dep python3

--
nosy: +asvetlov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22882] [patch] Linux packages you need to compile Python with all dependencies

[Ludovic Gasc]

Ludovic Gasc added the comment:

> $ sudo apt-get build-dep python3

Good to know, I will test for my next deployment.
The only problem with that, if you use a old Debian with a Python3 that need 
less dependencies that actual version.
But you can add a line for that in documentation if we have the case one day.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22876] ip_interface can't be broadcast or number net

[Nick Coghlan]

Nick Coghlan added the comment:

The Interface classes are actually designed to cover any association of an IP 
address with a specific network, not just host interface addresses. We knew it 
was a slight misnomer when we chose it, but network and broadcast addresses 
weren't considered special enough to be worth separating out (beyond the 
distinction present in iterating over the whole network range vs just the host 
addresses).

It's relatively straightforward to write a helper function that only allows the 
creation of host interfaces:

def host_interface(address):
ifaddr = ip_interface(address)
if ifaddr.ip == ifaddr.network.network_address:
raise ValueError("'{}' is a network address".format(ifaddr)
if ifaddr.ip == ifaddr.network.broadcast_address:
raise ValueError("'{}' is a broadcast address".format(ifaddr)
return ifaddr

I'm not sure if it's worthwhile to add such a helper function to the module 
itself.

One argument in favour of adding it is that it may help to emphasise that the 
normal ip_interface factory function and the *Interface class constructors are 
*not* restricted purely to host interfaces, and thus allow network and 
broadcast addresses to be associated with their corresponding network 
definition.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22886] TestProgram leaves defaultTestLoader.errors dirty

[Robert Collins]

New submission from Robert Collins:

TestProgram leaves defaultTestLoader.errors dirty. This primarily affects tests 
but it would be good hygiene to clear it at the end of TestProgram.

--
components: Library (Lib)
messages: 231261
nosy: rbcollins
priority: normal
severity: normal
status: open
title: TestProgram leaves defaultTestLoader.errors dirty
type: behavior
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22888] ensurepip and distutils' build_scripts fails on Windows when path to Python contains accented characters

[José Alberto Goncalves]

New submission from José Alberto Goncalves:

Summary:

Python 3.4's venv works fine in Windows, and pip works fine when installing 
both pure Python libraries and extension modules. However, when the virtual 
environment is under a path with non-ASCII characters, attempting to install a 
package that specifies console_scripts or scripts (like pip or mutagen, 
respectivelly), it fails with encoding errors.

I looked around the Internet for a solution but the best I could find was Issue 
#10419, which is over 3 years old and is marked as resolved, and couldn't find 
any other open issue about this.

Details of my case:

I created a Python 3.4 (32-bit) virtualenv via Python Tools for Visual Studio, 
on windows 8.1 (64-bit), in a folder that is under my home directory 
(C:\Users\José Alberto\), which happens to contain an accented character, using 
the latest Python you can download from the homepage.

Via Powershell I activated the virtualenv and tried to execute pip install 
mutagen (https://pypi.python.org/pypi/mutagen, it is relevant because it 
specifies scripts in its setup.py). The installation failed with the following 
error:

Downloading/unpacking mutagen
  Running setup.py (path:C:\Users\José 
Alberto\Documents\podtimizer\env_podtimizer\build\mutagen\setup.py) egg_info for
package mutagen

Installing collected packages: mutagen
  Running setup.py install for mutagen
Traceback (most recent call last):
  File "C:\Python34\lib\distutils\command\build_scripts.py", line 114, in 
copy_scripts
shebang.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 14: 
invalid continuation byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "", line 1, in 
  File "C:\Users\JosÚ 
Alberto\Documents\podtimizer\env_podtimizer\build\mutagen\setup.py", line 277, 
in 
"""
  File "C:\Python34\lib\distutils\core.py", line 148, in setup
dist.run_commands()
  File "C:\Python34\lib\distutils\dist.py", line 955, in run_commands
self.run_command(cmd)
  File "C:\Python34\lib\distutils\dist.py", line 974, in run_command
cmd_obj.run()
  File "C:\Users\JosÚ 
Alberto\Documents\podtimizer\env_podtimizer\lib\site-packages\setuptools-6.0.2-py3.4.egg\setuptools\command\install.py",
 line 61, in run
  File "C:\Python34\lib\distutils\command\install.py", line 539, in run
self.run_command('build')
  File "C:\Python34\lib\distutils\cmd.py", line 313, in run_command
self.distribution.run_command(command)
  File "C:\Python34\lib\distutils\dist.py", line 974, in run_command
cmd_obj.run()
  File "C:\Python34\lib\distutils\command\build.py", line 126, in run
self.run_command(cmd_name)
  File "C:\Python34\lib\distutils\cmd.py", line 313, in run_command
self.distribution.run_command(command)
  File "C:\Python34\lib\distutils\dist.py", line 974, in run_command
cmd_obj.run()
  File "C:\Python34\lib\distutils\command\build_scripts.py", line 50, in run
self.copy_scripts()
  File "C:\Python34\lib\distutils\command\build_scripts.py", line 118, in 
copy_scripts
"from utf-8".format(shebang))
ValueError: The shebang (b'#!C:\\Users\\Jos\xe9 
Alberto\\Documents\\podtimizer\\env_podtimizer\\Scripts\\python.exe\n') is not 
decodable from utf-8

I looked around the Internet for a solution, but the best I could find was the 
Issue #10419, which is over 3 years old and is marked as closed and resolved. 
The last comment mentions a fix that was commited to Distribute around that 
time, with the caveat that entry points script creation would fail if the path 
contained unencodeable characters (which sounds exactly like the problem I'm 
having). I Couldn't find an open issue to follow up on this.

I went to the source of the error, around 
Lib/distutils/command/build_scripts.py:106. Since this is Windows, the result 
of os.fsencode() uses the encoding 'mbcs' (as reported by Python), then it 
tries to decode it back using utf-8, and it blows up:

>>> import os
>>> os.fsencode('C:\\Users\\José Alberto\\')
b'C:\\Users\\Jos\xe9 Alberto\\'
>>> 'C:\\Users\\José Alberto\\'.encode('utf-8')
b'C:\\Users\\Jos\xc3\xa9 Alberto\\'

I commented both try..except after the os.fsencode and it worked, but 
commenting random code whose purpose I don't fully understand doesn't seem like 
a good strategy.

While testing for the above, I found I couldn't finish installing pip 
successfully on a virtualenv using just the Python installed from python.org.

On Powershell I created several virtualenvs using C:\Python34\python.exe -m 
venv. The envs were created successfully, but the pip's console_scripts 
installation failed silently. I could still run python -m pip and install 
packages, but the pip.exe files were not created.

I removed pip from the environment's site-packages directory and tried to 
reinstall it via

[issue22887] ensurepip and distutils' build_scripts fails on Windows when path to Python contains accented characters

[José Alberto Goncalves]

New submission from José Alberto Goncalves:

Summary:

Python 3.4's venv works fine in Windows, and pip works fine when installing 
both pure Python libraries and extension modules. However, when the virtual 
environment is under a path with non-ASCII characters, attempting to install a 
package that specifies console_scripts or scripts (like pip or mutagen, 
respectivelly), it fails with encoding errors.

I looked around the Internet for a solution but the best I could find was Issue 
#10419, which is over 3 years old and is marked as resolved, and couldn't find 
any other open issue about this.

Details of my case:

I created a Python 3.4 (32-bit) virtualenv via Python Tools for Visual Studio, 
on windows 8.1 (64-bit), in a folder that is under my home directory 
(C:\Users\José Alberto\), which happens to contain an accented character, using 
the latest Python you can download from the homepage.

Via Powershell I activated the virtualenv and tried to execute pip install 
mutagen (https://pypi.python.org/pypi/mutagen, it is relevant because it 
specifies scripts in its setup.py). The installation failed with the following 
error:

Downloading/unpacking mutagen
  Running setup.py (path:C:\Users\José 
Alberto\Documents\podtimizer\env_podtimizer\build\mutagen\setup.py) egg_info for
package mutagen

Installing collected packages: mutagen
  Running setup.py install for mutagen
Traceback (most recent call last):
  File "C:\Python34\lib\distutils\command\build_scripts.py", line 114, in 
copy_scripts
shebang.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 14: 
invalid continuation byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "", line 1, in 
  File "C:\Users\JosÚ 
Alberto\Documents\podtimizer\env_podtimizer\build\mutagen\setup.py", line 277, 
in 
"""
  File "C:\Python34\lib\distutils\core.py", line 148, in setup
dist.run_commands()
  File "C:\Python34\lib\distutils\dist.py", line 955, in run_commands
self.run_command(cmd)
  File "C:\Python34\lib\distutils\dist.py", line 974, in run_command
cmd_obj.run()
  File "C:\Users\JosÚ 
Alberto\Documents\podtimizer\env_podtimizer\lib\site-packages\setuptools-6.0.2-py3.4.egg\setuptools\command\install.py",
 line 61, in run
  File "C:\Python34\lib\distutils\command\install.py", line 539, in run
self.run_command('build')
  File "C:\Python34\lib\distutils\cmd.py", line 313, in run_command
self.distribution.run_command(command)
  File "C:\Python34\lib\distutils\dist.py", line 974, in run_command
cmd_obj.run()
  File "C:\Python34\lib\distutils\command\build.py", line 126, in run
self.run_command(cmd_name)
  File "C:\Python34\lib\distutils\cmd.py", line 313, in run_command
self.distribution.run_command(command)
  File "C:\Python34\lib\distutils\dist.py", line 974, in run_command
cmd_obj.run()
  File "C:\Python34\lib\distutils\command\build_scripts.py", line 50, in run
self.copy_scripts()
  File "C:\Python34\lib\distutils\command\build_scripts.py", line 118, in 
copy_scripts
"from utf-8".format(shebang))
ValueError: The shebang (b'#!C:\\Users\\Jos\xe9 
Alberto\\Documents\\podtimizer\\env_podtimizer\\Scripts\\python.exe\n') is not 
decodable from utf-8

I looked around the Internet for a solution, but the best I could find was the 
Issue #10419, which is over 3 years old and is marked as closed and resolved. 
The last comment mentions a fix that was commited to Distribute around that 
time, with the caveat that entry points script creation would fail if the path 
contained unencodeable characters (which sounds exactly like the problem I'm 
having). I Couldn't find an open issue to follow up on this.

I went to the source of the error, around 
Lib/distutils/command/build_scripts.py:106. Since this is Windows, the result 
of os.fsencode() uses the encoding 'mbcs' (as reported by Python), then it 
tries to decode it back using utf-8, and it blows up:

>>> import os
>>> os.fsencode('C:\\Users\\José Alberto\\')
b'C:\\Users\\Jos\xe9 Alberto\\'
>>> 'C:\\Users\\José Alberto\\'.encode('utf-8')
b'C:\\Users\\Jos\xc3\xa9 Alberto\\'

I commented both try..except after the os.fsencode and it worked, but 
commenting random code whose purpose I don't fully understand doesn't seem like 
a good strategy.

While testing for the above, I found I couldn't finish installing pip 
successfully on a virtualenv using just the Python installed from python.org.

On Powershell I created several virtualenvs using C:\Python34\python.exe -m 
venv. The envs were created successfully, but the pip's console_scripts 
installation failed silently. I could still run python -m pip and install 
packages, but the pip.exe files were not created.

I removed pip from the environment's site-packages directory and tried to 
reinstall it via

[issue22887] ensurepip and distutils' build_scripts fails on Windows when path to Python contains accented characters

[Ezio Melotti]

Changes by Ezio Melotti :


--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> ensurepip and distutils' build_scripts fails on Windows when 
path to Python contains accented characters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17293] uuid.getnode() MAC address on AIX

[koobs]

koobs added the comment:

I don't have the environment to test here. Can you run a custom build on the 
buildbots?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22882] [patch] Linux packages you need to compile Python with all dependencies

[R. David Murray]

R. David Murray added the comment:

The fact that it cost you a lot of time buttresses my point.  Any documentation 
we write is likely to get stale; it is only the downstream that knows when 
these things change and could maintain such a document.  And the easy answer is 
as Andrew indicated, which is maintained by downstream.  That said, if the 
various downstreams are using a unified set of names, then the equation would 
change.  The fact that you had separate sections for separate distributions 
argues that that is not (yet?) the case, though.

The build instructions in the devguide are more likely to be kept up to date, 
which is why I suggested we should cross link to them from the main build 
documentation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22882] [patch] Linux packages you need to compile Python with all dependencies

[R. David Murray]

Changes by R. David Murray :


--
versions:  -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22885] Arbitrary code execution vulnerability due to unchecked eval() call in dumbdbm module

[R. David Murray]

Changes by R. David Murray :


--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22882] Document Linux packages you need to compile Python with all dependencies

[R. David Murray]

Changes by R. David Murray :


--
title: [patch] Linux packages you need to compile Python with all dependencies 
-> Document Linux packages you need to compile Python with all dependencies

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22889] set a timeout for DNS lookups

[Kevin Burke]

New submission from Kevin Burke:

It would be nice to be able to set a timeout for DNS lookups in the event of a 
DNS server failure.

1. Set your DNS to something like 123.123.123.123 or any other host that is not 
listening for DNS queries on port 53.

2. Run requests.get('http://jsonip.com', timeout=3), or similar using 
urllib/http.client/httplib

3. Observe that the wall clock time is 2 minutes or higher.

It's known that a timeout value does not correspond to wall clock time, but 
this can be an unexpected cause of latency in http client requests, and is 
completely uncontrollable from client code (unless the user resolves DNS 
themselves). 

For a comparison, Go provides this functionality, see for example 
https://code.google.com/p/go/source/browse/src/pkg/net/lookup.go?name=release#55

--
components: Library (Lib)
messages: 231266
nosy: kevinburke
priority: normal
severity: normal
status: open
title: set a timeout for DNS lookups
type: enhancement
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22888] ensurepip and distutils' build_scripts fails on Windows when path to Python contains accented characters

[eryksun]

eryksun added the comment:

On Windows, shouldn't copy_scripts use UTF-8 instead of os.fsencode (MBCS)? The 
Python launcher executes the shebang line on Windows, and it defaults to UTF-8 
if a script doesn't have a BOM. See line 1105 in maybe_handle_shebang:

https://hg.python.org/cpython/file/ab2c023a9432/PC/launcher.c#l1064

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22890] StringIO.StringIO pickled in 2.7 is not unpickleable on 3.x

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

StringIO.StringIO is pickleable and unpickleable on 2.7 but is not unpickleable 
on 3.x.

Python 2.7:
>>> import pickle, StringIO
>>> pickle.dumps(StringIO.StringIO('abc'), 2)
'\x80\x02(cStringIO\nStringIO\nq\x00oq\x01}q\x02(U\tsoftspaceq\x03K\x00U\x07buflistq\x04]q\x05U\x03posq\x06K\x00U\x03lenq\x07K\x03U\x06closedq\x08\x89U\x03bufq\tU\x03abcq\nub.'
>>> pickle.loads(b'\x80\x02(cStringIO\nStringIO\nq\x00oq\x01}q\x02(U\tsoftspaceq\x03K\x00U\x07buflistq\x04]q\x05U\x03posq\x06K\x00U\x03lenq\x07K\x03U\x06closedq\x08\x89U\x03bufq\tU\x03abcq\nub.')


On 3.x I got an error:
Traceback (most recent call last):
  File "", line 1, in 
TypeError: _io.StringIO.__setstate__ argument should be 4-tuple, got dict

--
components: Library (Lib)
messages: 231268
nosy: alexandre.vassalotti, pitrou, serhiy.storchaka
priority: normal
severity: normal
status: open
title: StringIO.StringIO pickled in 2.7 is not unpickleable on 3.x
type: behavior
versions: Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18473] some objects pickled by Python 3.x are not unpicklable in Python 2.x because of incorrect REVERSE_IMPORT_MAPPING

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Here is a patch which fixes multiple reversible mappings.

Mappings for StringIO and cStringIO is removed at all because 
cStringIO.StringIO is not pickleable on 2.7 at all and pickled 
StringIO.StringIO is not compatible with 3.x (issue22890).

--
Added file: http://bugs.python.org/file37210/pickle_fix_import.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18473] some objects pickled by Python 3.x are not unpicklable in Python 2.x because of incorrect REVERSE_IMPORT_MAPPING

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com