[issue19829] _pyio.BufferedReader and _pyio.TextIOWrapper destructor don't emit ResourceWarning if the file is not closed

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

> Serhiy: why did you add 2.7 to this bug? For 2.7, I don’t think anything 
> should be done. There is no ResourceWarning in 2.7.

Just for the case.

Not calling close() in __del__() is one option, and it looks attractive. But 
there are possible hidden catches. There is no guarantee that flush() writes 
all buffered data, some stateful encoder or compressor can left some data in 
the buffer unless explicitly closed. close() can write required footer or close 
connection. If not call close() in __del__(), GzipFile will produce incorrect 
file.

May be this is appropriate. This is why explicit close() should be called or 
context manager should be used.

Other option is to use some API through all closable objects. _dealloc_warn() 
is not the only option.

1. Add an optional parameter to close() methods to denote that it is called 
from __del__. This is backward incompatible option, it is too late to do this.

2. __del__() calls special method instead of close() if exists.

3. The _dealloc_warn() option (the name should be changed). The wrapper 
delegates emitting a warning to wrapped object by calling the _dealloc_warn() 
method if exists. Unlike to previous option this method is called in addition 
to close(), not instead.

4. The wrapper checks special attribute (or calls a method) of wrapped object. 
If it exists and is true, the wrapper emits a warning.

--
nosy: +benjamin.peterson, stutzbach
versions:  -Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26644] SSLSocket.recv(-1) triggers SystemError

[Martin Panter]

New submission from Martin Panter:

SystemError indicates an internal error that is not supposed to be triggerable 
from Python code. We should probably raise ValueError like plain sockets 
instead.

>>> s = create_connection(("python.org", 443))
>>> s.recv(-1)
Traceback (most recent call last):
  File "", line 1, in 
ValueError: negative buffersize in recv
>>> ss = ssl.wrap_socket(s)
>>> ss.recv(-1)
Traceback (most recent call last):
  File "", line 1, in 
  File "/home/proj/python/cpython/Lib/ssl.py", line 910, in recv
return self.read(buflen)
  File "/home/proj/python/cpython/Lib/ssl.py", line 787, in read
return self._sslobj.read(len, buffer)
  File "/home/proj/python/cpython/Lib/ssl.py", line 573, in read
v = self._sslobj.read(len or 1024)
SystemError: Negative size passed to PyBytes_FromStringAndSize

--
components: Extension Modules
messages: 262488
nosy: martin.panter
priority: normal
severity: normal
status: open
title: SSLSocket.recv(-1) triggers SystemError
type: behavior
versions: Python 2.7, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26644] SSLSocket.recv(-1) triggers SystemError

[Martin Panter]

Martin Panter added the comment:

Proposed patch

--
keywords: +patch
stage:  -> patch review
Added file: http://bugs.python.org/file42295/ssl-negative.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25987] collections.abc.Reversible

[Ivan Levkivskyi]

Changes by Ivan Levkivskyi :


--
nosy: +levkivskyi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26643] regrtest: rework libregrtest.save_env submodule

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

> I replaced get_sys_path()/restore_sys_path() (2 methods) with 
> ModuleAttr.get()+ModuleAttrList.encode_value() / 
> ModuleAttr.restore()+ModuleAttrList.restore_attr() (4 methods).

I need to examine all other methods scattered around just to conclude that they 
are overloaded and aren't used indirectly.

Imagine that you are reading the code first time and need to know only about 
one resource. Three lines in two methods are much simpler then hundreds lines 
scattered in tens of methods.

> I'm not sure that it's a good thing to have dependencies between "resources". 
> I see that Files depends on Cwd, but maybe the Files check can be integrated 
> into Cwd?

I'm not sure that there are only two depended resources. In any case 
deterministic behavior is better. It is very unhappy to to research random test 
failures.

> Ok so the order matters, in this case, I can use a metaclass to use the order 
> of class definition. It will make get_resources() simpler, it will be build 
> directly in the metaclass.

No-no-no. Please don't complicate the code just because you can. This will make 
reading and modifying the code yet harder.

> How do you want to implement that? Currently, a resource has a "get" method 
> which returns a blackbox. It's not easy to display it. There is not standard 
> format.

There is a number of ways.

1. Just write smart general diff reporter as in unittest. It's Python, use 
force^W data reflection!

2. Add third method for resources that need special diff reporting. Use simple 
general diff reporter by default.

3. Split the "sys.path" resource on two simple resources: one test that 
sys.path identity is not changed, other tests it's content.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11245] Implementation of IMAP IDLE in imaplib?

[Rob la Lau]

Changes by Rob la Lau :


--
nosy: +ohreally

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26645] argparse prints help messages to stdout instead of stderr by default

[Alexey Muranov]

New submission from Alexey Muranov:

I believe that printing help and usage messages to stdout is a design error.

In stdout i expect to find the output of my program, not help or diagnostic 
messages.  It is strange to see nothing printed on the screen (where stderr 
usually goes), and then to find help, usage, or *error* messages in the file 
where stdout was sent.  (Yes, argparse prints even error messages to stdout by 
default).

This issue has been discussed before because the implementation did not agree 
with the documentation: http://bugs.python.org/issue10728
I believe that the conclusion to adjust the documentation to the implementation 
was a mistake.

P.S. Compare with the direction of the output of `grep -h` or `git -h`.

--
components: Library (Lib)
messages: 262491
nosy: Alexey Muranov
priority: normal
severity: normal
status: open
title: argparse prints help messages to stdout instead of stderr by default
type: behavior
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26645] argparse prints help messages to stdout instead of stderr by default

[SilentGhost]

Changes by SilentGhost :


--
nosy: +bethard
versions: +Python 3.6 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26645] argparse prints help messages to stdout instead of stderr by default

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

`grep --help` outputs to stdout. `grep -h` outputs to stderr, because this is 
error, there is no the `-h` flag. The same for git and most other command line 
tools.

argparse follows this behavior. There is no bug.

--
nosy: +serhiy.storchaka
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26587] Possible duplicate entries in sys.path if .pth files are used with zip's

[Wolfgang Langner]

Wolfgang Langner added the comment:

Ok, I implemented point 3.
Check if it is a dir or file and makepath only in this case.
All other entries are added unmodified to the set.

Added a test case also for an URL path.

I think duplicate detection is now improved and it should break nothing.

--
Added file: http://bugs.python.org/file42296/site2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22898] segfault during shutdown attempting to log ResourceWarning

[Xavier de Gaye]

Xavier de Gaye added the comment:

Victor,

With warn_5.patch *filename is not set when globals is NULL: setup_context() 
returns 0, and so do_warn() returns NULL without calling warn_explicit().

This is different from your initial warn.patch where setup_context() returns 1 
in that case and an attempt is made to issue the warning.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26587] Possible duplicate entries in sys.path if .pth files are used with zip's

[Brett Cannon]

Changes by Brett Cannon :


--
assignee:  -> brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26646] Allow built-in module in package

[Daniel Shaulov]

New submission from Daniel Shaulov:

Hi,

I was trying to build a python interpreter that has the cpp part of msgpack as 
a built-in module. I thought that I could just copy the msgpack folder to the 
Modules folder and add this 2 lines to Modules/Setup.local:

msgpack._packer msgpack/_packer.cpp
msgpack._unpacker msgpack/_unpacker.cpp

I had a few obstacles, the attached patch fixes them all.

The first - makesetup has a list of regexes to match and it has *.* after going 
through all known extensions to throw a "bad word" error.
I removed the check. All those things will now be assumed to be a module with a 
package.
Now the actual init function name is PyInit__packer and doesn't have msgpack in 
it, so I also added code in makesetup to use the full name as the name and only 
the module name for the PyInit function.


The second is that in Lib/importlib/_bootsrap.py in BuiltinImporter.find_spec 
there is a specific case to ignore modules that are part of a package.
Is there a reason to forbid it? I removed that check.
There were also unit tests that checked this behavior which I deleted.
I added tests that check module in package instead of them.
Changing _bootsrap.py also changes Python/importlib.h (it is the frozen 
importlib), I added a separate patch with that change, to not clutter the main 
patch.

The third - the __name__ didn't have the package prefix.
Digging around I found a comment in PyModule_Create2 that says that the shared 
library loader stores the full name _Py_PackageContext before loading the 
module, so I did the same in _imp_create_builtin that is done in 
_PyImport_LoadDynamicModuleWithSpec to set _Py_PackageContext and the __name__ 
was fixed too.

(If anyone tries to do this with msgpack and wants to see that it works - you 
also need to copy the msgpack directory to the Lib directory, and in 
__init__.py, where it catches the import error and goes to the fallback, just 
reraise the exception instead of letting it go to the fallback)

Do note that this does not allow for built-in packages, only build-it module in 
package. If we want to allow built-in packages, we will need to decide on some 
syntax to distinguish it in the Setup files and some way to distinguish them in 
PyImport_Inittab (for example - an asterix before the name of the package?)

Thanks, Daniel.

--
components: Build, Interpreter Core
files: builtin_package.patch
keywords: patch
messages: 262495
nosy: Daniel Shaulov, brett.cannon, eric.snow, ncoghlan
priority: normal
severity: normal
status: open
title: Allow built-in module in package
type: enhancement
versions: Python 3.6
Added file: http://bugs.python.org/file42297/builtin_package.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26646] Allow built-in module in package

[Daniel Shaulov]

Changes by Daniel Shaulov :


Added file: http://bugs.python.org/file42298/importlib_h.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26645] argparse prints help messages to stdout instead of stderr by default

[Alexey Muranov]

Alexey Muranov added the comment:

Thanks for the explanation, this makes sense.  I did not notice that argparse 
outputs to stderr if command line arguments are wrong, i was probably wrong 
when said it prints error messages to stdout.  I did not notice indeed that 
there were no `-h` option in git.

However, my grep version 2.5.1-FreeBSD outputs help to stderr even with 
`--help` option.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26644] SSLSocket.recv(-1) triggers SystemError

[Benjamin Peterson]

Benjamin Peterson added the comment:

Is this what other file-like objects do with negatives sizes?

--
nosy: +benjamin.peterson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26644] SSLSocket.recv(-1) triggers SystemError

[Martin Panter]

Martin Panter added the comment:

Socket objects aren’t exactly file-like. Plain non-SSL sockets don’t even have 
read() methods.

I think giving a meaning to recv(-1) would be an (unwanted) new feature, rather 
than a bug fix. If you want a file-like object linked to a socket, I would 
suggest using something like the makefile() method instead of adding to the 
low-level socket object API.

But to answer your question: no, most file methods treat a negative size as a 
special request to read until EOF, e.g. read(-1), readline(-1) and 
readlines(-1) of RawIOBase, BufferedIOBase and TextIOBase. On the other hand, 
BufferedIOBase.read1(-1) is poorly defined and supported (Issue 23214), but may 
end up meaning something like “read an arbitrary non-zero chunk with a minimum 
amount of low-level calls and processing”.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26644] SSLSocket.recv(-1) triggers SystemError

[Benjamin Peterson]

Benjamin Peterson added the comment:

Thanks for the explanation. Your patch lgtm.

On Sat, Mar 26, 2016, at 15:01, Martin Panter wrote:
> 
> Martin Panter added the comment:
> 
> Socket objects aren’t exactly file-like. Plain non-SSL sockets don’t even
> have read() methods.
> 
> I think giving a meaning to recv(-1) would be an (unwanted) new feature,
> rather than a bug fix. If you want a file-like object linked to a socket,
> I would suggest using something like the makefile() method instead of
> adding to the low-level socket object API.
> 
> But to answer your question: no, most file methods treat a negative size
> as a special request to read until EOF, e.g. read(-1), readline(-1) and
> readlines(-1) of RawIOBase, BufferedIOBase and TextIOBase. On the other
> hand, BufferedIOBase.read1(-1) is poorly defined and supported (Issue
> 23214), but may end up meaning something like “read an arbitrary non-zero
> chunk with a minimum amount of low-level calls and processing”.
> 
> --
> 
> ___
> Python tracker 
> 
> ___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26644] SSLSocket.recv(-1) triggers SystemError

[Martin Panter]

Martin Panter added the comment:

Thanks for the reviews. Here is a patch that avoids breaking read(-1, buffer).

--
Added file: http://bugs.python.org/file42299/ssl-negative.v2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26647] Wordcode

[Demur Rumed]

New submission from Demur Rumed:

Originally started @ https://github.com/abarnert/cpython/tree/wpy

This patch is based off of https://github.com/serprex/cpython/tree/wpy

It omits importlib.h & importlib_external.h as those are generated

It omits https://github.com/serprex/cpython/blob/wpy/Python/wordcode.md

I got around to benchmarking against building on master rather than using my 
repo's packaged version, it's currently a 1% speed improvement (every bit 
counts). I'm testing on an Intel Atom 330 with Linux.  Besides the minor perf 
increase, it generates smaller bytecode & is simpler (peephole now handles 
EXTENDED_ARG since it isn't too hard to track & while loops become for loops in 
dis)

Previous discussion: 
https://mail.python.org/pipermail/python-dev/2016-February/143357.html

pdb works without changes. coverage.py doesn't seem to rely on anything this 
changes

I modified byteplay to target this change mostly over the course of half an 
hour before work: https://github.com/serprex/byteplay/blob/master/wbyteplay.py

I'd be interested to hear if this encoding simplifies things for FAT python & 
the recent work to cache attribute/global lookup

Remaining code issues: peepholer could allocate half the space as it does now 
for basic block tracking, compile.c & peephole.c repeat themselves on computing 
instruction size given an argument & how to spit out an instruction given an 
argument

Breaking change in dis: I've removed HAVE_ARGUMENT. This is to help code fail 
fast. It could be replaced with IGNORES_ARGUMENT or, as abarnert suggested, a 
range(90,256) named after the other hasXXXs 'hasarg'

--
components: Interpreter Core
files: wpy.patch
keywords: patch
messages: 262501
nosy: Demur Rumed
priority: normal
severity: normal
status: open
title: Wordcode
type: performance
versions: Python 3.6
Added file: http://bugs.python.org/file42300/wpy.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26648] csv.reader Error message indicates to use deprecated

[Philip Martin]

New submission from Philip Martin:

Currently, the error message:

_csv.Error: new-line character seen in unquoted field - do you need to open the 
file in universal-newline mode?

is cryptic in that universal line mode has been deprecated, and will not run in 
Python 3.5., i.e.:
open(escape_path, "rU", encoding=ENCODING)
>>> DeprecationWarning: 'U' mode is deprecated

I think a message indicating a suggestion to open the file with newline='' to 
enable universal line mode is more insightful.

--
messages: 262502
nosy: Philip Martin
priority: normal
severity: normal
status: open
title: csv.reader Error message indicates to use deprecated
type: enhancement
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22854] Documentation/implementation out of sync for IO

[Martin Panter]

Martin Panter added the comment:

Okay let’s document fileno(), read, write and seek operations in the base 
classes as raising OSError then. This effectively rejects the OP (Stanislaw’s) 
view that the exception should be more specific.

In patch v3, I changed everything over to say OSError is raised. I also added a 
background thread to drain the pipe writer. And I removed a test_no_fileno() 
method which was having an “existential crisis”. :)

--
Added file: http://bugs.python.org/file42301/UnsupportedOperation.v3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25940] SSL tests failed due to expired svn.python.org SSL certificate

[Roundup Robot]

Roundup Robot added the comment:

New changeset 3da812602881 by Martin Panter in branch 'default':
Issue #25940: Use internal local server more in test_ssl
https://hg.python.org/cpython/rev/3da812602881

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26644] SSLSocket.recv(-1) triggers SystemError

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

LGTM.

--
assignee:  -> martin.panter
nosy: +serhiy.storchaka
stage: patch review -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22854] Documentation/implementation out of sync for IO

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

LGTM.

--
assignee:  -> martin.panter
stage: patch review -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25940] SSL tests failed due to expired svn.python.org SSL certificate

[Martin Panter]

Martin Panter added the comment:

Okay so for the record, the maintainence branches were changed over from 
svn.python.org to self-signed.pythontest.net, and just now I changed the 3.6 
branch to use a local server for most tests.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23735] Readline not adjusting width after resize with 6.3

[Eric Price]

Eric Price added the comment:

Well, I could piggyback on the existing flags if I just wanted to support 
readline -- there are already two flags for readline 4.0 -- but if our real 
goal is to be compatible with editline, we probably need another flag.

I think you're right that it should reinstall itself, in case it's installed 
using signal().

I've made a new patch that includes a config flag and puts everything inside 
the ifdefs.

--
Added file: http://bugs.python.org/file42302/rl_sigwinch_version_3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26645] argparse prints help messages to stdout instead of stderr by default

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Does FreeBSD grep support the `--help` option? Long options is GNU extension, 
classical UNIX commands supported only short options.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26649] Fail update installation: 'utf-8' codec can't decode

[q20611...@163.com]

New submission from q20611...@163.com:

This error  occureed on the gns3 ;
give you the errorlog files .

--
files: gns3-errorlog
messages: 262510
nosy: shanzhengcheng
priority: normal
severity: normal
status: open
title: Fail update installation: 'utf-8' codec can't decode
versions: Python 3.5
Added file: http://bugs.python.org/file42303/gns3-errorlog

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com