[issue23742] expandvars removes single quotes ( ' )

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Here is a patch.

--
assignee:  -> serhiy.storchaka
components: +Library (Lib)
keywords: +patch
stage: needs patch -> patch review
versions: +Python 2.7
Added file: http://bugs.python.org/file38646/issue23742.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23062] test_argparse --version test cases

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I should agree with you. Then the patch LGTM.

--
assignee:  -> berker.peksag
stage: patch review -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22977] Unformatted “Windows Error 0x%X” exception message on Wine

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

LGTM.

--
stage: patch review -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23745] Exception when parsing an email using email.parser.BytesParser

[Elmer]

New submission from Elmer:

I am working with a large dataset of emails and loading one of them resulted in 
an exception: "TypeError: unorderable types: ValueTerminal() < CFWSList()"

I have attached the (anonymised and minimised) email source of the email that 
triggered the exception.

$ python
Python 3.4.2 (default, Nov 12 2014, 18:23:59) 
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.54)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import email
>>> from email import parser, policy
>>> 
>>> f = open("testmail.eml",'rb')
>>> src = f.read()
>>> f.close()
>>> 
>>> msg = email.parser.BytesParser(_class=email.message.EmailMessage, 
>>> policy=email.policy.default).parsebytes(src)
Traceback (most recent call last):
  File "", line 1, in 
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/email/parser.py",
 line 124, in parsebytes
return self.parser.parsestr(text, headersonly)
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/email/parser.py",
 line 68, in parsestr
return self.parse(StringIO(text), headersonly=headersonly)
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/email/parser.py",
 line 57, in parse
feedparser.feed(data)
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/email/feedparser.py",
 line 178, in feed
self._call_parse()
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/email/feedparser.py",
 line 182, in _call_parse
self._parse()
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/email/feedparser.py",
 line 384, in _parsegen
for retval in self._parsegen():
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/email/feedparser.py",
 line 255, in _parsegen
if self._cur.get_content_type() == 'message/delivery-status':
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/email/message.py",
 line 579, in get_content_type
value = self.get('content-type', missing)
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/email/message.py",
 line 472, in get
return self.policy.header_fetch_parse(k, v)
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/email/policy.py",
 line 145, in header_fetch_parse
return self.header_factory(name, ''.join(value.splitlines()))
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/email/headerregistry.py",
 line 583, in __call__
return self[name](name, value)
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/email/headerregistry.py",
 line 194, in __new__
cls.parse(value, kwds)
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/email/headerregistry.py",
 line 441, in parse
kwds['decoded'] = str(parse_tree)
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/email/_header_value_parser.py",
 line 195, in __str__
return ''.join(str(x) for x in self)
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/email/_header_value_parser.py",
 line 195, in 
return ''.join(str(x) for x in self)
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/email/_header_value_parser.py",
 line 1136, in __str__
for name, value in self.params:
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/email/_header_value_parser.py",
 line 1101, in params
parts = sorted(parts)
TypeError: unorderable types: ValueTerminal() < CFWSList()

--
components: email
files: testmail.eml
messages: 238987
nosy: Elmer, barry, r.david.murray
priority: normal
severity: normal
status: open
title: Exception when parsing an email using email.parser.BytesParser
type: behavior
versions: Python 3.4
Added file: http://bugs.python.org/file38647/testmail.eml

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22516] Windows Installer won't - even when using "just for me"option

[Nick Coghlan]

Nick Coghlan added the comment:

I think we learned our lesson re making decisions at the language summit 
(rather than focusing on bringing folks up to speed on a problem area) after 
needing to later reverse engineer the rationale for some of the decisions made 
at the first couple :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22977] Unformatted “Windows Error 0x%X” exception message on Wine

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

There are other occurrences of %X in the code. Do you want provide a patch for 
them Martin?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21526] Support new booleans in Tkinter

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23252] Add support of writing to unseekable file in zipfile

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23647] imaplib.py MAXLINE value is too low for gmail

[Arnt Gulbrandsen]

Arnt Gulbrandsen added the comment:

You're entirely right. I should've reread 2683 too (a decade has passed since I 
read that).

The danger with accepting the unlimited line length is that batch scripts might 
accept an infinitely large batch. Which is a matter of python culture, really.

(This has been a very positive experience.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16314] Support xz compression in distutils

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Addressed Berker's comments. Added versionchanged directives and a whatsnews 
entry.

--
Added file: http://bugs.python.org/file38648/distutils-lzma_3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23722] During metaclass.__init__, super() of the constructed class does not work

[Martin Teichmann]

Martin Teichmann added the comment:

A note on the implementation:

The compiler leaves a __cell__ entry in the class' namespace, which
is then filled by type.__new__, and removed from the namespace by
the latter. This is the same way it is done for __qualname__.

As the patch tampers with the compiler, when testing the patch
don't forget to remove old .pyc files, otherwise strange things will
happen.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23746] sysconfg.is_python_build() is buggy

[Anand B Pillai]

New submission from Anand B Pillai:

On Python 3.5.0a1+ built from source,

>>> import sysconfig
>>> sysconfig.is_python_build()
False
>>> sysconfig.is_python_build(True)
False
>>> sysconfig._PROJECT_BASE
'/opt/bin'
>>> import sys
>>> sys._home
>>>

The problem is, when sys._home is None, this function uses 
_is_python_source_dir(_PROJECT_BASE) . In this case the _PROJECT_BASE is 
clearly passed wrongly as '/opt/bin'. That is the INSTALL_PREFIX, not 
_PROJECT_BASE .

Let us do a small hack and set _PROJECT_BASE to the folder where I build this 
Python version.

# Of course this can't be reproduced but you get the idea.

>>> sysconfig._PROJECT_BASE='/home/anand/code/cpython/'
>>> sysconfig.is_python_build()
True

The documentation says,

"

sysconfig.is_python_build()

Return True if the current Python installation was built from source.

"

which is clearly in conflict with what it is doing. 

>From a quick look at sysconfig.py it looks like it is calculating 
>_PROJECT_BASE wrongly. 

I can give a patch for this, but first I am more interested in finding out what 
this function is supposed to do - why have this function if you are not able to 
get the details of the build environment from the built interpreter ? Clearly 
it is not doing that here.



The conclusions are part of the attached file in comments.

--
components: Library (Lib)
messages: 238993
nosy: pythonhacker
priority: normal
severity: normal
status: open
title: sysconfg.is_python_build() is buggy
versions: Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23746] sysconfg.is_python_build() is buggy

[Anand B Pillai]

Changes by Anand B Pillai :


Added file: http://bugs.python.org/file38649/sysconfig_test.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23746] sysconfg.is_python_build() is buggy

[STINNER Victor]

STINNER Victor added the comment:

IMO it's more a documentation issue than a bug.

sysconfig.is_python_build() returns True when Python is run from its source 
code directory. For example, I compiled Python in ~/prog/python/default.

~/prog/python/default/python returns True.

I installed python to /opt/py35/bin/python: it returns False.

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23747] platform module exposes win32_ver function on posix systems

[Anand B Pillai]

New submission from Anand B Pillai:

>>> import platform
>>> platform.system()
'Linux'
>>> platform.win32_ver()
('', '', '', '')

Why is this function even exposed on Linux ? It should be conditionally exposed 
only on win32 platforms. 

Funny that this is coming from the module named "platform" itself :)

--
components: Library (Lib)
messages: 238995
nosy: pythonhacker
priority: normal
severity: normal
status: open
title: platform module exposes win32_ver function on posix systems

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23747] platform module exposes win32_ver function on posix systems

[STINNER Victor]

STINNER Victor added the comment:

> Why is this function even exposed on Linux?

It is a deliberate design choice of the platform module. mac_ver(), win32_ver() 
and linux_distribution() are available on all platforms.

mac_ver() says "entries which cannot be determined are set to ''. All tuple 
entries are strings."

Maybe it should be better explained in the doc.

--
assignee:  -> docs@python
components: +Documentation
nosy: +docs@python, haypo
versions: +Python 2.7, Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23747] platform module exposes win32_ver function on posix systems

[Anand B Pillai]

Anand B Pillai added the comment:

Similarly for mac_ver, java_ver etc.

>>> platform.mac_ver()
('', ('', '', ''), '')
>>> platform.java_ver()
('', '', ('', '', ''), ('', '', ''))

Maybe it is okay if these functions are present, but can't they raise an 
exception or return None instead of returning these funny tuples when empty 
strings ?

I am surprised at Python's inconsistency in such things. For example,

>>> import winsound
Traceback (most recent call last):
  File "", line 1, in 
ImportError: No module named 'winsound'

Works as expected on Linux. In the same vein, these functions shouldn't be 
present as well IMHO - I agree this is debatable of course.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23746] sysconfg.is_python_build() is buggy

[Anand B Pillai]

Anand B Pillai added the comment:

Thanks. From current documentation it isn't clear. Agree to make this a doc bug 
then. 

It should also be made clear this would work only for the executable built in 
the source directory. As in,

(Running from the source folder using source Python executable).

anand@toshiba-laptop:~/code/cpython$ ./python 
Python 3.5.0a1+ (default:656543a2ad75, Mar  3 2015, 22:56:27) 
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.is_python_build()
True

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15836] unittest assertRaises should verify excClass is actually a BaseException class

[Michael Foord]

Michael Foord added the comment:

I like the first variant suggested by Ezio as more concise. I'll try and look 
at the substance of the patch today.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23746] sysconfg.is_python_build() is buggy

[Anand B Pillai]

Anand B Pillai added the comment:

On second thoughts, why have such a function if it works correctly only when 
executed from source Cpython folder using source CPython executable  and not 
otherwise ?

Cuz if I am calling it like that I already KNOW that I am in such an 
environment.

Possibly this is an internal function to be used by developers to find out if 
they are in such an environment.

IMHO Then -> make it private (not callable outside) or drop it altogether.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23747] platform module exposes win32_ver function on posix systems

[STINNER Victor]

STINNER Victor added the comment:

> Maybe it is okay if these functions are present, but can't they raise an 
> exception or return None instead of returning these funny tuples when empty 
> strings ?

It would break the backward compatibility. Again, it's a delibarate choice.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23746] sysconfg.is_python_build() is buggy

[STINNER Victor]

STINNER Victor added the comment:

> On second thoughts, why have such a function if it works correctly only when 
> executed from source Cpython folder using source CPython executable  and not 
> otherwise ?

It's useful to know such information. Python behaves a little bit
differently when run from the source code (not installed).

Example of usage of this function:

Lib/test/test_asdl_parser.py:12:if not sysconfig.is_python_build():
Lib/test/test_gdb.py:38:if not sysconfig.is_python_build():
Lib/test/test_pyexpat.py:448:if sysconfig.is_python_build():
Lib/test/test_subprocess.py:415:
@unittest.skipIf(sysconfig.is_python_build(),

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23746] sysconfg.is_python_build() is buggy

[Anand B Pillai]

Anand B Pillai added the comment:

> Lib/test/test_asdl_parser.py:12:if not sysconfig.is_python_build():

As I guessed - it is used internally by unit tests and possibly has no other 
significan audience.

It is best to document this clearly.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21859] Add Python implementation of FileIO

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Updated to the tip (added the closefd attribute in repr). os.fstat() is now 
called only once in the constructor.

--
Added file: http://bugs.python.org/file38651/pyio_fileio_7.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write?

[Wolfgang Maier]

Changes by Wolfgang Maier :


Added file: http://bugs.python.org/file38650/write_bytes_like_objects_v3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23748] platform._uname_cache is writeable

[Anand B Pillai]

New submission from Anand B Pillai:

>> import platform
>>> print 'Actual =>',platform.uname()
Actual => ('Linux', 'toshiba-laptop', '3.13.0-24-generic', '#47-Ubuntu SMP Fri 
May 2 23:30:00 UTC 2014', 'x86_64', 'x86_64')
>>> import hack_uname
# Someone imports my module unaware of the hack (see attached file)
>>> platform.uname()
('Limux', 'hacker-laptop', '11.15.0-28000-absurd', '#1 - FunkyDistro SMMP 
Fry Feb 30 2015 23:59:00 UTC 2015', 'x866_64', 'x866_64')

Fix - Make the global _uname_cache inaccessible via the module and hence 
unwriteable. I can provide a patch - it is kind of easy fix.

I think this might also be a security issue since if someone is writing a 
significant piece of code based on the platform it can screw up the system - or 
his web application if a piece of code like this is introduced in a module via 
his chain of imports by a malicious hacker.

--
components: Library (Lib)
files: hack_uname.py
messages: 239005
nosy: pythonhacker
priority: normal
severity: normal
status: open
title: platform._uname_cache is writeable
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5
Added file: http://bugs.python.org/file38652/hack_uname.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write?

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

In general the patch LGTM.

--
assignee:  -> serhiy.storchaka
stage:  -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23748] platform._uname_cache is writeable

[Anand B Pillai]

Anand B Pillai added the comment:

I am changing the type to security as I dont think this is a behaviour issue.

--
type: behavior -> security

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23748] platform._uname_cache is writeable

[STINNER Victor]

STINNER Victor added the comment:

we are all consenting adults here. Why do you modify a private attribute?

> I am changing the type to security as I dont think this is a behaviour issue.

I don't understand why do you consider that it is a security vulnerability?

>>> import hack_uname
# Someone imports my module unaware of the hack (see attached file)

Your exploit starts by running untrusted Python code. Never do that. The 
vulnerability is the ability to load unstrusted Python code, not to modify the 
platform module.

I close the issue as not a bug.

--
nosy: +haypo
resolution:  -> not a bug
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23748] platform._uname_cache is writeable

[Anand B Pillai]

Anand B Pillai added the comment:

Hmmm... dear sir -  what prevents you from adding an __all__ to the module with 
these cache variables excluded ?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23748] platform._uname_cache is writeable

[Anand B Pillai]

Anand B Pillai added the comment:

Closing is an easy fix, bet on my word this would appear as a different bug, 
thanks for your quickie fix!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23748] platform._uname_cache is writeable

[STINNER Victor]

STINNER Victor added the comment:

> Hmmm... dear sir -  what prevents you from adding an __all__ to the module 
> with these cache variables excluded ?

You didn't understand the general philosophy, we are not hidding internals. For 
the specific case of platform._uname_cache, it uses the "_" prefix which is an 
indicator saying that "you are not supposed to modify it except if you really 
understand what do you". It's useful to have this variable modifiable, for unit 
tests for example.

I don't see how adding __all__ variable to the platform would change anything. 
Variables prefixed by "_" are already excluded when using "from platform import 
*".

If you don't understand the Python philosophy, please open a thread on 
python-ideas or even python-dev mailing list, to get a longer explanation.

Read also the PEP 20.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21859] Add Python implementation of FileIO

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Restored first os.fstat() (however it now is redundant) and addressed most 
other Victor's comments.

In general I prefer EAFP over BDFL, and often "except AttributeError" looks 
better to me than getattr().

--
Added file: http://bugs.python.org/file38653/pyio_fileio_8.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23310] Mock constructor configuration fails for magic methods

[Anand Reddy Pandikunta]

Anand Reddy Pandikunta added the comment:

mock itself fails

--
nosy: +Anand Reddy Pandikunta
title: MagicMock constructor configuration fails for magic methods -> Mock 
constructor configuration fails for magic methods
Added file: http://bugs.python.org/file38654/test_mock.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19610] setup.py does not allow a tuple for classifiers

[Berker Peksag]

Berker Peksag added the comment:

Here is a new patch. I didn't touched provides, requires and obsoletes fields 
since they are not used much in the setuptools era.

Distribution.finalize_options() already converts string types to lists for 
platforms and keywords fields. I didn't changed that behavior.

--
Added file: http://bugs.python.org/file38655/issue19610_v3.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21560] gzip.write changes trailer ISIZE field before type checking - corrupted gz file after trying to write string

[Roundup Robot]

Roundup Robot added the comment:

New changeset 4dfe0634d11a by Serhiy Storchaka in branch '2.7':
Issue #21560: An attempt to write a data of wrong type no longer cause
https://hg.python.org/cpython/rev/4dfe0634d11a

New changeset 6eb48b22ff5c by Serhiy Storchaka in branch '3.4':
Issue #21560: An attempt to write a data of wrong type no longer cause
https://hg.python.org/cpython/rev/6eb48b22ff5c

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write?

[Roundup Robot]

Roundup Robot added the comment:

New changeset 4dc69e5124f8 by Serhiy Storchaka in branch 'default':
Issue #23688: Added support of arbitrary bytes-like objects and avoided
https://hg.python.org/cpython/rev/4dc69e5124f8

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21560] gzip.write changes trailer ISIZE field before type checking - corrupted gz file after trying to write string

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Tests are taken from issue23688. Thanks for your contribution Wolfgang.

--
assignee:  -> serhiy.storchaka
nosy: +serhiy.storchaka
resolution:  -> fixed
stage: test needed -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write?

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23676] Add support of UnicodeTranslateError in standard error handlers

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

No, currently UnicodeTranslateError is not used in the stdlib in 3.x. But it is 
documented and supported by some error handlers. I think it should be wider 
used in text-to-text translations similar to proposed in issue18814.

--
assignee:  -> serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write?

[Stefan Krah]

Stefan Krah added the comment:

I think there's a behavior change: Before you could gzip non-contiguous
views directly, now that operation raises BufferError.

--
nosy: +skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write?

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Could you provide an example?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket

[Giovanni Cannata]

New submission from Giovanni Cannata:

It's not possible to wrap a socket in tls. The StreamWriter object should have 
an option to start a tls negotiation using the SSLContext of the server.

This is needed for protocols the have a "start_tls" feature, for example the 
ldap protocol.

In a non async program it's very easy: 
   wrapped_socket = ssl_context.wrap_socket(connection.socket, 
server_side=True, do_handshake_on_connect=True)

there should be something similar in the StreamWriter interface:
   yield from writer.wrap_socket()

Bye,
Giovanni

--
components: asyncio
messages: 239021
nosy: gc, gvanrossum, haypo, yselivanov
priority: normal
severity: normal
status: open
title: asyncio missing wrap_socket
type: enhancement
versions: Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket

[STINNER Victor]

STINNER Victor added the comment:

Yes, it's not supported yet. It was already requested in this issue:
https://code.google.com/p/tulip/issues/detail?id=79

asyncio got a new SSL implementation which makes possible to implement 
STARTTLS. Are you interested to implement it?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23740] http.client request and send method have some datatype issues

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Note that for file-like objects we have also the same issue as issue22468. 
Content-Length is not determined correctly for GzipFile and like.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write?

[Stefan Krah]

Stefan Krah added the comment:

Sure:

import gzip
x = memoryview(b'x' * 10)
y = x[::-1]
with gzip.GzipFile("x", 'w') as f:
f.write(y)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23738] Clarify documentation of positional-only default values

[Brett Cannon]

Brett Cannon added the comment:

I say switch to what Argument Clinic uses, else there's a disconnect 
syntactically between help() and the docs.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21062] Evalute all import-related modules for best practices

[Brett Cannon]

Changes by Brett Cannon :


--
assignee:  -> brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23749] asyncio missing wrap_socket

[Giovanni Cannata]

Giovanni Cannata added the comment:

Thanks, I will look to the new implementation of ssl in 3.5, and try to adapt 
it for my project (sldap3). I'd like to help, but I'm not skilled in 
asynchronous programming so I'm not sure if I succeed.

Bye,
Giovanni

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write?

[Wolfgang Maier]

Wolfgang Maier added the comment:

ouch. haven't thought of this.

OTOH, just plain io with your example:

with open('xy', 'wb') as f:
f.write(y)

Traceback (most recent call last):
  File "", line 2, in 
f.write(y)
BufferError: memoryview: underlying buffer is not C-contiguous

fails too and after all that's not too surprising.

In a sense, the old behavior was an artefact of silently copying the memoryview 
to bytes. You never used it *directly*.
But, yes, it is a change in (undocumented) behavior :(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23750] Clarify difference between os.system/subprocess.call in section "Replacing os.system()"

[Andreas Sommer]

New submission from Andreas Sommer:

Reading over the section "Replacing os.system()" 
(https://docs.python.org/2/library/subprocess.html#replacing-os-system), one 
might assume that the return value of os.system and subprocess.call are 
equivalent.

status = os.system("mycmd" + " myarg")
# becomes
status = subprocess.call("mycmd" + " myarg", shell=True)

However, they are not. Example:

import sys
import os
import subprocess

print subprocess.call("false")
print os.system("false")

gives 1 and 256, respectively. Maybe this could be rephrased for clarity, or a 
hint added.

--
assignee: docs@python
components: Documentation
messages: 239028
nosy: Andreas Sommer, docs@python
priority: normal
severity: normal
status: open
title: Clarify difference between os.system/subprocess.call in section 
"Replacing os.system()"
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write?

[Wolfgang Maier]

Wolfgang Maier added the comment:

to preserve compatibility:

there is the memoryview.c_contiguous flag. Maybe we should just check it and if 
it is False fall back to the old copying behavior ?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write?

[Wolfgang Maier]

Wolfgang Maier added the comment:

something like:


def write(self,data):
self._check_closed()
if self.mode != WRITE:
import errno
raise OSError(errno.EBADF, "write() on read-only GzipFile object")

if self.fileobj is None:
raise ValueError("write() on closed GzipFile object")

if isinstance(data, bytes):
length = len(data)
elif isinstance(data, memoryview) and not data.c_contiguous:
data = data.tobytes()
length = len(data)
else:
# accept any data that supports the buffer protocol
data = memoryview(data)
length = data.nbytes

if length > 0:
self.fileobj.write(self.compress.compress(data))
self.size += length
self.crc = zlib.crc32(data, self.crc) & 0x
self.offset += length

return length

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write?

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Here is a patch that restores support on non-contiguous memoryviews.

It would be better to drop support of non-contiguous data, because it worked 
only by accident. Needed support of only bytes-like memoryviews written by 
BufferedWriter.

--
Added file: http://bugs.python.org/file38656/gzip_write_noncontiguous.patch

___
Python tracker 

___diff -r 7e179ee91af0 Lib/gzip.py
--- a/Lib/gzip.py   Mon Mar 23 15:26:49 2015 +0200
+++ b/Lib/gzip.py   Mon Mar 23 16:27:08 2015 +0200
@@ -340,6 +340,8 @@ class GzipFile(io.BufferedIOBase):
 # accept any data that supports the buffer protocol
 data = memoryview(data)
 length = data.nbytes
+if not data.contiguous:
+data = bytes(data)
 
 if length > 0:
 self.fileobj.write(self.compress.compress(data))
diff -r 7e179ee91af0 Lib/test/test_gzip.py
--- a/Lib/test/test_gzip.py Mon Mar 23 15:26:49 2015 +0200
+++ b/Lib/test/test_gzip.py Mon Mar 23 16:27:08 2015 +0200
@@ -74,6 +74,7 @@ class TestGzip(BaseTest):
 m = memoryview(bytes(range(256)))
 data = m.cast('B', shape=[8,8,4])
 self.write_and_read_back(data)
+self.write_and_read_back(memoryview(data1 * 50)[::-1])
 
 def test_write_bytearray(self):
 self.write_and_read_back(bytearray(data1 * 50))
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write?

[Stefan Krah]

Stefan Krah added the comment:

> In a sense, the old behavior was an artefact of silently copying the 
> memoryview to bytes.

It likely wasn't intentional, but tobytes() *is* used to serialize
weird arrays to their C-contiguous representation (following the
logical structure of the array rather than the physical one).

Since the gzip docs don't help much, I guess the new behavior is
probably okay.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write?

[Stefan Krah]

Stefan Krah added the comment:

I just see that non-contiguous arrays didn't work in 2.7 either,
so that was probably the original intention.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write?

[Wolfgang Maier]

Wolfgang Maier added the comment:

Serhiy:

I think I saw that you committed this also to the 2.7 branch, but that would 
not work since memoryviews do not have the nbytes attribute (they do not seem 
to have cast either). One would have to calculate the length instead from other 
properties.
Tests would fail too I think.

If I'm mistaken, then sorry for the noise.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write?

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

OK, so left it as is if nobody complains.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21859] Add Python implementation of FileIO

[Hynek Schlawack]

Changes by Hynek Schlawack :


--
nosy:  -hynek

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write?

[Wolfgang Maier]

Wolfgang Maier added the comment:

I see now that it is just issue21560 that went into 2.7 and that's fine.
As I said: sorry for the noise

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23688] unnecessary copying of memoryview in gzip.GzipFile.write?

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

> I think I saw that you committed this also to the 2.7 branch,

I committed only working tests and a fix from issue21560.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22560] New SSL implementation based on ssl.MemoryBIO

[Berker Peksag]

Changes by Berker Peksag :


--
stage: patch review -> resolved

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23751] Allow reverse lookup (value-to-member) for Enum sublcasses decorated with @enum.unique

[Mohith]

New submission from Mohith:

Addendum to issue 18042.

I've had a use case where I wanted to allow reverse lookups (i.e. 
value-to-member lookup) for my enum.Enum that was decorated via @enum.unique. 
In such a case, I would add my own class method that performs the logic. E.g. 
an unoptimized copy-and-paste-able example for use via python3 interactive 
shell:


import datetime
import enum

@enum.unique
class DayOfWeek(enum.Enum):
  SUNDAY = 0  # Sunday is '0' according to strftime('%w')
  MONDAY = 1
  TUESDAY = 2
  WEDNESDAY = 3
  THURSDAY = 4
  FRIDAY = 5
  SATURDAY = 6  # Saturday is '6' according to strftime('%w')
  @classmethod
  def reverse_lookup(cls, value):
for _, member in cls.__members__.items():
  if member.value == value:
return member
raise LookupError

today_value = int(datetime.date.today().strftime('%w'))
today = DayOfWeek.reverse_lookup(today_value)
print('Today is', today.name.title())


---

(Aside: it seems like an optimized version that uses a cached lookup dictionary 
is inconvenient to implement in that it involves storing the cache in a global 
variable or a closure or a descriptor attribute or etc. [unless there's a 
simple recommended approach of which I am just unaware].)

I think use cases for reverse lookup would not be uncommon. For example, an 
IntEnum subclass decorated via @enum.unique is a great candidate (as is any 
simple hashable/equatable value type). I am unsure of the proper interface, but 
I am thinking that passing a boolean argument to enum.unique should enable the 
usage of a special __lookup__ metaclass property for optimized reverse lookups. 
I am thinking a possible (untested) enhancement to enum.unique along the lines 
of:


class EnumMeta(type):

  ...

  _reverse_lookup_ = None

  @property
  def __lookup__(cls):
if cls._reverse_lookup_ is None:
  raise AttributeError('Reverse lookup unsupported.')
return MappingProxyType(cls._reverse_lookup_)


def unique(enumeration, reverse_lookup=False):
...
reverse_lookup = {} if reverse_lookup else None
for name, member in enumeration.__members__.items():
  ...
  if reverse_lookup is not None:
reverse_lookup[member.value] = member
...
if reverse_lookup is not None:
  enumeration._reverse_lookup_ = reverse_lookup
return enumeration

--
components: Library (Lib)
messages: 239038
nosy: barry, eli.bendersky, ethan.furman, mmuddasani
priority: normal
severity: normal
status: open
title: Allow reverse lookup (value-to-member) for Enum sublcasses decorated 
with @enum.unique
type: enhancement
versions: Python 3.4, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23751] Allow reverse lookup (value-to-member) for Enum sublcasses decorated with @enum.unique

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Use DayOfWeek(today_value).

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23751] Allow reverse lookup (value-to-member) for Enum sublcasses decorated with @enum.unique

[Mohith]

Mohith added the comment:

Doh! That works. And it was in the documentation too. Silly me.

Apologies for opening an issue, and thanks for showing me the way!

--
resolution:  -> works for me
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23751] Allow reverse lookup (value-to-member) for Enum sublcasses decorated with @enum.unique

[Ezio Melotti]

Changes by Ezio Melotti :


--
stage:  -> resolved

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14260] re.groupindex is available for modification and continues to work, having incorrect data inside it

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

What approach looks better, a copy or a read-only proxy?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23699] Add a macro to ease writing rich comparisons

[Petr Viktorin]

Petr Viktorin added the comment:

Changed the macro to Py_RETURN_RICHCOMPARE. This is not an expression, allowing 
the use of a switch statement. On the other hand, it's even larger macro than 
before now.

>From the discussion it seems that doing this correctly is tricky to do this 
>correctly - another point for standardizing this.

I put everything in a single macro to ease review in Rietveld.

--
Added file: 
http://bugs.python.org/file38657/Use-a-macro-to-reduce-boilerplate-in-rich-comparison.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16328] win_add2path.py sets wrong user path

[Steve Dower]

Steve Dower added the comment:

It looks like the script hasn't been updated for the moved user scripts 
directory (under %APPDATA%), but otherwise this is not a bug and can be closed.

--
nosy: +paul.moore

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16328] win_add2path.py sets wrong user path

[Paul Moore]

Paul Moore added the comment:

Pip and/or setuptools will add the Scripts directory when needed, so it's not a 
big deal that the installer doesn't create it. In 2.7.9 and later, and 3.4 
onwards, the Scripts directory is created as part of the ensurepip step of the 
install, so it's pretty much a moot point anyway.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23752] Cleanup io.FileIO

[STINNER Victor]

New submission from STINNER Victor:

While reviewing a Python implementation of io.FileIO (_pyio.FileIO) in the 
issue #21859, many issues were found in the C implementation. I open an issue 
to fix them.

We may fix them before or after the Python implementation is merged.

--
components: Library (Lib)
messages: 239045
nosy: haypo
priority: normal
severity: normal
status: open
title: Cleanup io.FileIO
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21859] Add Python implementation of FileIO

[STINNER Victor]

STINNER Victor added the comment:

I opened the issue #23752: "Cleanup io.FileIO".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23753] Drop HAVE_FSTAT: require fstat() to compile/use Python

[STINNER Victor]

New submission from STINNER Victor:

Topic previously discussed at:
https://mail.python.org/pipermail/python-dev/2013-May/126285.html

Related issue:
http://bugs.python.org/issue12082

Antoine Pitrou wrote in the issue:
"I would personally like to remove HAVE_FSTAT and make Python unconditionally 
use fstat(). It will make the code quite simpler in some places."

I agree. I'm quite sure that Python doesn't work on such platform, and it would 
require much more changes than just making fstat optional.

So I'm in favor of dropping the check on fstat() and expect it to be always 
available.

Examples of Python modules of the standard library using os.fstat:

- fileinput
- genericpath
- netrc
- os which contains "set.add(stat) # fstat always works"
- _pyio (the call is optional, it catchs AttributeError)
- shutil
- socket
- tarfile
- asyncio
- http.client (optional, catch AttributeError)
- http.server
- logging
- io
- etc.

--
messages: 239047
nosy: haypo, neologix, pitrou
priority: normal
severity: normal
status: open
title: Drop HAVE_FSTAT: require fstat() to compile/use Python
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23602] Implement __format__ for Fraction

[Mark Dickinson]

Mark Dickinson added the comment:

[Eric]

> I'm not sure it needs fixing...

Hmm.  We've gone to some lengths to make sure that we get correctly-rounded 
results for formatting of Decimal and float types, as well as to make sure that 
operations like converting a Fraction to a float are correctly rounded.  It 
would be disappointing if the result of formatting a Fraction wasn't correctly 
rounded, and I'd personally consider it a bug.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23654] infinite loop in faulthandler._stack_overflow

[Matt Frank]

Matt Frank added the comment:

This is a patch that turns off the Intel Compiler's optimization for the 
stack_overflow() function.  It turns out that icc doesn't support gcc's 
__attribute__((optimize("no-optimize-sibling-calls"))).  Instead I used an 
ifdef'd intel-specific pragma that turns off optimization completely for just 
this function, and just for the intel compiler.

This particular pragma has the benefit that it should also work with the Intel 
compiler on Windows without needing further ifdefs.

--
Added file: http://bugs.python.org/file38658/icc-stackoverflow.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16328] win_add2path.py sets wrong user path

[Steve Dower]

Steve Dower added the comment:

This script adds them to PATH. As I said, the original issue is not a bug, but 
it has drawn attention to something we apparently missed a while ago.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16328] win_add2path.py sets wrong user path

[Paul Moore]

Paul Moore added the comment:

Steve - is it the fact that it's not using the versioned user-site directory 
that you're referring to?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16328] win_add2path.py sets wrong user path

[Paul Moore]

Paul Moore added the comment:

... because I think it does do that (see 
http://bugs.python.org/file38085/userscripts.patch which updates this file)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16328] win_add2path.py sets wrong user path

[Paul Moore]

Paul Moore added the comment:

I think the problem here is that as the OP mentioned, win_add2path.py doesn't 
add the directory if it doesn't exist, so if you run it immediately after 
install, it won't add the directories that don't exist yet. And because it's 
setting the registry entries to make the path changes persistent, you only run 
it once and don't rerun it after those directories are created. Although if you 
did, that would fix the problem...

Maybe win_add2path.py should add the directories even if they don't exist? It 
only means removing the isdir check from the line

if path and path not in envpath and os.path.isdir(path):

I don't really have a feel for whether that would be a good idea. I guess the 
only harm would be a bit of clutter on %PATH%, and probably no-one cares about 
that...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16328] win_add2path.py sets wrong user path

[Paul Moore]

Paul Moore added the comment:

One further thought - the installer can't create the user scripts directory if 
you're doing an all-users install, that's sort of the point of "all users". It 
could do so for a per-user install, but then there would be an inconsistency 
that I don't think is a good idea.

So maybe skipping the "does the directory exist" check is the only option.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16328] win_add2path.py sets wrong user path

[Paul Moore]

Paul Moore added the comment:

Here is a patch to fix the issue.

--
keywords: +patch
Added file: http://bugs.python.org/file38659/addpath.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23602] Implement __format__ for Fraction

[Stefan Behnel]

Stefan Behnel added the comment:

Absolutely. Fractions are all about exact calculations, much more so than 
Decimals. So the formatting output should be as accurate as requested or 
possible (well, excluding infinity).

--
nosy: +scoder

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16328] win_add2path.py sets wrong user path

[Steve Dower]

Steve Dower added the comment:

Ah, you're right, it was fixed. Guess I misread my local copy (or I was looking 
at 3.4).

That patch looks good to me. It'll help cover the case where someone does 
something to create the directory.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16328] win_add2path.py sets wrong user path

[Paul Moore]

Paul Moore added the comment:

Cool. I'll try to set up a test (it's not covered by the testsuite AFAICT) on a 
VM or something, just to confirm I haven't broken anything.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23654] infinite loop in faulthandler._stack_overflow

[Roundup Robot]

Roundup Robot added the comment:

New changeset d6003de8ecc8 by Victor Stinner in branch '3.4':
Issue #23654: Fix faulthandler._stack_overflow() for the Intel C Compiler (ICC)
https://hg.python.org/cpython/rev/d6003de8ecc8

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23654] infinite loop in faulthandler._stack_overflow

[STINNER Victor]

STINNER Victor added the comment:

Thanks for your contribution Matt!

--
resolution:  -> fixed
status: open -> closed
versions:  -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23753] Drop HAVE_FSTAT: require fstat() to compile/use Python

[STINNER Victor]

STINNER Victor added the comment:

stat.patch: Stop pretending that Python works without stat() nor fstat(), 
consider that these functions are always available.

--
keywords: +patch
Added file: http://bugs.python.org/file38660/stat.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23753] Drop HAVE_FSTAT: require fstat() to compile/use Python

[STINNER Victor]

Changes by STINNER Victor :


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16328] win_add2path.py sets wrong user path

[Paul Moore]

Paul Moore added the comment:

One further thought. This is a change in behaviour (albeit to a script in 
Tools), so I'm inclined to say it's a new feature for 3.5, rather than a bugfix 
to be backported to 2.7 and 3.4.

For users of 2.7/3.4, the workaround is simply to rerun the script - it doesn't 
add directories it has already added, so it's safe to rerun to add newly 
created directories.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23752] Cleanup io.FileIO

[STINNER Victor]

STINNER Victor added the comment:

fileio_drop_dup_fstat.patch: Remove the duplicate call to fstat() in fileio.c. 
Tests pass, invalid FD are still detected since there is a second call to 
fstat() which also raises OSError(EBADF) if fstat(fd) fails.

--
keywords: +patch
Added file: http://bugs.python.org/file38661/fileio_drop_dup_fstat.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23753] Drop HAVE_FSTAT: require fstat() to compile/use Python

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

See issue22623 for moving in opposite direction.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22623] Missing guards for some POSIX functions

[Antoine Pitrou]

Antoine Pitrou added the comment:

Personally, I'd rather answer that it's not our problem if some systems are not 
POSIX-compliant. Maintainers of such systems can always maintain a patch to 
disable the missing functionality. Adding conditionals everywhere has a 
non-trivial maintenance cost.

--
nosy: +loewis, pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23752] Cleanup io.FileIO

[STINNER Victor]

STINNER Victor added the comment:

The previous change related to fstat() is the changeset 3b5279b5bfd1 from the 
issue #21679. The changeset introduces the private _blksize attribute. The 
strange thing is that the changelog is:
"Issue #21679: Prevent extraneous fstat() calls during open().  Patch by 
Bohuslav Kabrda."

fstat() was called twice and it is still called twice. Maybe I missed something.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22623] Missing guards for some POSIX functions

[STINNER Victor]

STINNER Victor added the comment:

> Many POSIX functions aren’t available on every system, especially embedded 
> ones.

What do you call "embedded systems"? I worked on set top boxes (the box to 
watch television) between 2011 and 2013 and we had a regular Linux kernel with 
all POSIX functions. Most of the time, the CPU was a slow MIPS. The tool chain 
was based on GCC. For some hardware, we used the µlibc but this C library 
provides all functions required by Python.

> Personally, I'd rather answer that it's not our problem if some systems are 
> not POSIX-compliant. Maintainers of such systems can always maintain a patch 
> to disable the missing functionality. Adding conditionals everywhere has a 
> non-trivial maintenance cost.

I agree. Anyway, in the embedded world, softwares are usually old and heavily 
patched. For example, in 2013 we still used Python 2.5.2 with a lot of patches. 
For example, patches for cross compilation. But also backported features like 
HTTP Keep-Alive or HTTPS checks.

Technically, you can easily fork Python repository and keep your patches up to 
date using rebase.

We defined better rules to support officially a platform. A requirement is for 
example to have a buildbot running tests daily on the platform. I'm not sure 
that you are able to host a public buildbot for each custom embedded platform...

Android looks a more important target and it was already discussed to setup a 
Android buildbot.

I'm in favor of *dropping* all these annoying #ifdef because it's harder to 
review, maintain and debug such code. Here is my contribution: issue #23753 
"Drop HAVE_FSTAT: require fstat() to compile/use Python". A concrete example: 
we are working on a Python implementation of io.FileIO and I don't want to see 
hasattr(os, 'fstat') in the Python code because it has a performance cost *at 
runtime* (see issue #21859).

See also the new Mobile-SIG mailing list which is maybe more appropriate to 
discuss such changes:
https://mail.python.org/mailman/listinfo/mobile-sig

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23753] Drop HAVE_FSTAT: require fstat() to compile/use Python

[STINNER Victor]

STINNER Victor added the comment:

My changes only *removes* code and so make it simpler ;-)

$ diffstat stat.patch 
 Include/fileutils.h  |6 --
 Include/pyport.h |   22 --
 Modules/_io/fileio.c |   20 
 Modules/mmapmodule.c |4 
 Python/fileutils.c   |   16 
 Python/marshal.c |4 
 6 files changed, 72 deletions(-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23754] Add a new os.read_into() function to avoid memory copies

[STINNER Victor]

New submission from STINNER Victor:

Sockets have a recv_into() method, io.IOBase has a readinto() method, but there 
is no os.read_into() function. It would avoid memory copies. It would benefit 
to the Python implementation FileIO (readall() and readinto() methods), see the 
issue #21859.

--
components: Library (Lib)
messages: 239069
nosy: haypo
priority: normal
severity: normal
status: open
title: Add a new os.read_into() function to avoid memory copies
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23752] Cleanup io.FileIO

[STINNER Victor]

STINNER Victor added the comment:

See also issue #21861: "io class name are hardcoded in reprs".

In my review of _pyio.FileIO, I asked if it would be possible to use 
__qualname__ in __getstate__().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21859] Add Python implementation of FileIO

[STINNER Victor]

STINNER Victor added the comment:

New issue #23754: "Add a new os.read_into() function to avoid memory copies".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23754] Add a new os.read_into() function to avoid memory copies

[STINNER Victor]

STINNER Victor added the comment:

os.read_into() may be used by the following functions.

subprocess.Popen._execute_child():

# Wait for exec to fail or succeed; possibly raising an
# exception (limited in size)
errpipe_data = bytearray()
while True:
part = os.read(errpipe_read, 5)
errpipe_data += part
if not part or len(errpipe_data) > 5:
break

subprocess.Popen.communicate():

self._fileobj2output = {}
if self.stdout:
self._fileobj2output[self.stdout] = []
...
data = os.read(key.fd, 32768)
if not data:
...
self._fileobj2output[key.fileobj].append(data)
...
stdout = b''.join(...)

multiprocessing.Connection._recv():

def _recv(self, size, read=_read):
buf = io.BytesIO()
handle = self._handle
remaining = size
while remaining > 0:
chunk = read(handle, remaining)
n = len(chunk)
if n == 0:
if remaining == size:
raise EOFError
else:
raise OSError("got end of file during message")
buf.write(chunk)
remaining -= n
return buf

multiprocessing.read_unsigned():

def read_unsigned(fd):
data = b''
length = UNSIGNED_STRUCT.size
while len(data) < length:
s = os.read(fd, length - len(data))
if not s:
raise EOFError('unexpected EOF')
data += s
return UNSIGNED_STRUCT.unpack(data)[0]

The problem is that some functions still require to return a bytes, not a 
bytearray or something else. Converting a bytearray to a bytes still require a 
memory copy...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23648] PEP 475 meta issue

[STINNER Victor]

STINNER Victor added the comment:

> Well, all the syscalls which can blocki can fail with EINTR, so all
> I:O related one.

This is also what I expect, but how do you explain that I'm unable to see 
os.stat() failing with EINTR? I'm testing on ext4 file system. May it occur 
with other file systems (ex: NFS)? Or maybe not on Linux, but on other 
platforms?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com