[issue35972] _xxsubinterpreters: channel_send() may truncate ints on 32-bit platforms

2019-02-12 Thread Alexey Izbyshev


Alexey Izbyshev  added the comment:

"long long" is mandated to be at least 64-bit by C99 (5.2.4.2.1 Sizes of 
integer types). If it were 32-bit, no warnings would have been issued.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue21109] tarfile: Traversal attack vulnerability

2019-02-12 Thread Tal Einat


Tal Einat  added the comment:

> Is there any update on this? Will this be fixed in the next release?

There was progress made as described on this issue, but there is yet work to be 
done, and no-one seems to be taking this upon themselves at the moment.

I agree that it would be great to have this in 3.8.

--
versions:  -Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

2019-02-12 Thread Emmanuel Arias


Emmanuel Arias  added the comment:

Hi,

The reason of the aborted(core dump) of the interpreter is because when the 
init process was wrong, the 

```_Py_NO_RETURN fatal_error()```

call and abort()

So, I change it for a exit(status) to avoid break the interpreter (I attach a 
patch)

What do you think about it? If all it's ok I can prepare the PR

--
keywords: +patch
nosy: +eamanu
Added file: 
https://bugs.python.org/file48134/0001-avoid-abort-when-initialize-python-fail.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

Last year, I reworked Python initialization to introduce a new _PyInitError 
structure which allows to report a failure to the caller of a function instead 
of calling directly Py_FatalError() which always call abort() immediately.

_PyInitError allows to distinguish "user error" and "internal error". User 
error is caused by an user mistake, whereas internal errors are all other 
errors. User errors don't call abort() to avoid dumping a core dump.

It's not always easy to decide if a bug is caused by the user or not.

For example, a memory allocation failure is now considered as an "user error":

#define _Py_INIT_NO_MEMORY() _Py_INIT_USER_ERR("memory allocation failed")

See Include/coreconfig.h for the definition of _PyInitError and related macros.

One enhancement of this new API is that it now reports the name of the C 
function which causes the error. The initial bug report says "Fatal Python 
error: init_sys_streams: can't initialize sys standard 
streams": init_sys_streams() function raised the fatal error.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

A workaround is the usage of -I option:

vstinner@apu$ PYTHONPATH=: python3 -c ""
Fatal Python error: init_sys_streams: can't initialize sys standard streams
Traceback (most recent call last):
  File "/usr/lib64/python3.7/io.py", line 52, in 
  File "/tmp/demo/abc.py", line 1, in 
Exception: Hi
Aborted (core dumped)

vstinner@apu$ PYTHONPATH=: python3 -I -c ""


A long term solution would to enable -I behavior by default: don't add the 
current directory to sys.path when using -c CMD.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35960] dataclasses.field does not preserve empty metadata object

2019-02-12 Thread Eric V. Smith


Eric V. Smith  added the comment:


New changeset b01786c8812c4cc24dd561b5941025bdd6f444c0 by Eric V. Smith 
(Christopher Hunt) in branch 'master':
 bpo-35960: Fix dataclasses.field throwing away empty metadata.  (GH-11815)
https://github.com/python/cpython/commit/b01786c8812c4cc24dd561b5941025bdd6f444c0


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35960] dataclasses.field does not preserve empty metadata object

2019-02-12 Thread miss-islington


Change by miss-islington :


--
pull_requests: +11857

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35960] dataclasses.field does not preserve empty metadata object

2019-02-12 Thread Eric V. Smith


Eric V. Smith  added the comment:


New changeset 0a834c18cdafa5447d4436878e64aa7a4e4dc817 by Eric V. Smith (Miss 
Islington (bot)) in branch '3.7':
bpo-35960: Fix dataclasses.field throwing away empty metadata.  (GH-11815) 
(GH-11826)
https://github.com/python/cpython/commit/0a834c18cdafa5447d4436878e64aa7a4e4dc817


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35960] dataclasses.field does not preserve empty metadata object

2019-02-12 Thread Eric V. Smith


Eric V. Smith  added the comment:

Thanks for the patch!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35972] _xxsubinterpreters: channel_send() may truncate ints on 32-bit platforms

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

What matters here is that _PyCrossInterpreterData.data type is void*. So the 
largest integer that you can store is intptr_t (signed) or uintptr_t 
(unsigned). In practice, sizeof(void*) == sizeof(intptr_t) == sizeof(uintptr_t) 
== sizeof(size_t) == sizeof(ssize_t). IMHO using size_t or ssize_t is a good 
solution, since it's well supported in Python.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35957] Indentation explanation is unclear

2019-02-12 Thread Jérôme LAURENS

Jérôme LAURENS  added the comment:

To be more precise, consider code

def f(x):
   \tx=0 # 7 spaces + one tab
return x # 8 spaces

In cpython, both indentation levels are 8 and no indentation error is reported 
(this is the case where both tab size and alt tab size are equal)

If instead of 8 the tab would count for 6 spaces, then we would have 12 and 8 
as indentation level, resulting in a mismatch and an indentation error being 
reported, according to the documentation. This is inconsistent.
Then either the documentation is faulty or cpython is.

Actually, cpython accepts a mix of space and tabs only when tabs are in 8, 16, 
24... positions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25461] Unclear language (the word ineffective) in the documentation for os.walk

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

> What is the status on this? From this discussion, it looks like @vstinner 
> pushed changes to resolve this. Do we close this? If this still needs a 
> patch, then one of the patches can be reviewed in a PR on GitHub.

It seems like os_walk_doc.patch and issue25461.patch have been written after I 
pushed my changes. Would you be interested to convert these patches into a 
proper PR?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35971] Documentation should warn about code injection from current working directory

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

Documentation is one thing. But I'm interested to discuss again (in 2019) the 
idea of changing the default behavior in Python 3.8 to enhance the "default" 
security.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35927] Intra-package References Documentation Incomplete

2019-02-12 Thread Nick Coghlan


Nick Coghlan  added the comment:

As discussed in 
http://python-notes.curiousefficiency.org/en/latest/python_concepts/import_traps.html#executing-the-main-module-twice
 and https://www.python.org/dev/peps/pep-0395/#why-are-my-imports-broken the 
cryptic error message here is due to the relative import being resolved based 
on the module name "__main__", since the submodule was executed directly, 
rather than the intended "sound.filter.vocoder" name that it would have when 
imported or executed via the "-m" switch.

It's spectacularly unobvious what's going on if you're not deeply familiar with 
the intricacies of main module execution, but I haven't had any great ideas for 
practical ways to improve the error message.

I'm not sure we currently have access to the required info at the point this 
error is raised, but perhaps it would be enough to add the name of the module 
doing the import to the error message as a reminder that directly executing 
scripts inside packages does weird things to import resolution?

Something like:

  "Attempted relative import beyond top-level package '__main__'"

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

2019-02-12 Thread Paul Ganssle


Paul Ganssle  added the comment:

> One enhancement of this new API is that it now reports the name of the C 
> function which causes the error. The initial bug report says "Fatal Python 
> error: init_sys_streams: can't initialize sys standard 
streams": init_sys_streams() function raised the fatal error.

I think that's a welcome change, the main question is whether it's necessary to 
hard-crash the interpreter and dump the core, or if, in most cases, the error 
message + non-zero return code would be sufficient.

It's not clear to me how useful the traceback itself is, but of course I'm 
giving a toy example where the Python exception is sufficient to debug what's 
failing. I would think that *in general* if the interpreter is failing to 
initialize because of a problem in the Python stack, printing the Python 
traceback should be sufficient. If there's good reason to think otherwise, then 
I'm fine with closing this bug. Otherwise I'd say it would be nice to refactor 
in such a way that avoids the core dump when the problem is on the Python side 
(leaving in place the error message you added that shows where it failed on the 
C side).

> A workaround is the usage of -I option:

In this case I don't think we need a workaround, because the issue is really 
"should it be possible for a pure python crash to cause the interpreter to 
crash rather than exit with an error code?" Shadowing a standard library module 
from your PYTHONPATH is just the easiest way to manifest this. (Though probably 
more people should be using -I, so it's good to bring it up).


> A long term solution would to enable -I behavior by default: don't add the 
> current directory to sys.path when using -c CMD.

If you've been following the recent issues with the PEP 517 rollout in 
setuptools, you'll see that I'm deeply sympathetic to this view (though in this 
case I'm not amazingly optimistic about its feasibility). Probably we shouldn't 
derail this issue too much with a discussion of our revolutionary views on the 
right default Python path, though.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

See also bpo-35971: "Documentation should warn about code injection from 
current working directory" which discuss changing the default behavior.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue14465] xml.etree.ElementTree: add feature to prettify XML output

2019-02-12 Thread Clayton Olney


Clayton Olney  added the comment:

I have a use case where the receiving application is expecting the indentation, 
and I need to run my code in Lambda. So, lxml is out of the question.

--
nosy: +Clayton Olney

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35971] Documentation should warn about code injection from current working directory

2019-02-12 Thread Ronald Oussoren


Ronald Oussoren  added the comment:

I don't know a good reason for including $PWD in sys.path for "python -m", I 
expect only scripts that run other scripts (such as coverage.py) might want the 
current behaviour and those can adjust to new behaviour.

For "python -c CMD" the current behaviour is easier to defend, this matches the 
behaviour w.r.t. sys.path of typing the command in an interactive session.


The behaviour for scripts (add the directory where the script is located to 
sys.path) is also easier to defend: that makes it easier to use scripts without 
installing stuf and without explicitly managing sys.path. 

I've used this in the past to split scripts into modules without installing 
modules in a global directory, but these days I tend to use virtual environment 
for that. I guess the usefulness of this behaviour depends on the type of user: 
the current behaviour is useful for more casual use of Python (which includes 
people learning python)

--
nosy: +ronaldoussoren

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35951] os.renames() creates directories if original name doesn't exist

2019-02-12 Thread Joannah Nanjekye


Change by Joannah Nanjekye :


--
keywords: +patch
pull_requests: +11858
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35951] os.renames() creates directories if original name doesn't exist

2019-02-12 Thread STINNER Victor


Change by STINNER Victor :


--
versions: +Python 3.8 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35972] _xxsubinterpreters: channel_send() may truncate ints on 32-bit platforms

2019-02-12 Thread Eric Snow


Eric Snow  added the comment:

Having slept on it and given the extra info you've provided I'm more 
comfortable with the proposed solution now. :)

So the result will be that on 32-bit architectures only 32-bit (signed) ints 
will be allowed through channels.  On 64-bit platforms (and higher) it will 
still be 64-bit ints.  That's a livable outcome, particularly since 32-bit 
platforms already have to deal with such variation already. :)  Plus, the 
consistency with sys.maxsize makes sense.

The alternative (on 32-bit arch) would be to send through a struct containing 
the long long, similar to what we do for str and bytes.  I don't think that's 
worth it.  If it matters we can do something like that later (since it's an 
implementation detail).

Consequently, I've approved the PR.  I'll merge it in a little while.

Thanks for working on this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35972] _xxsubinterpreters: channel_send() may truncate ints on 32-bit platforms

2019-02-12 Thread Eric Snow


Change by Eric Snow :


--
components: +Interpreter Core -Extension Modules
stage: patch review -> commit review
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35972] _xxsubinterpreters: channel_send() may truncate ints on 32-bit platforms

2019-02-12 Thread Eric Snow


Eric Snow  added the comment:

FTR, issue #34569 covered similar ground.

--
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35971] Documentation should warn about code injection from current working directory

2019-02-12 Thread Eric Snow


Eric Snow  added the comment:

+Nick (who may have some insight here)

--
nosy: +eric.snow, ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

2019-02-12 Thread Eric Snow


Eric Snow  added the comment:

On Tue, Feb 12, 2019 at 3:57 AM STINNER Victor  wrote:
> A long term solution would to enable -I behavior by default: don't add the 
> current directory to sys.path when using -c CMD.

>From what I can recall, it was a conscious decision to include CWD in 
>sys.path.  Doing so benefits some scripting use cases.  The downside is the 
>effect when running an app.

--
nosy: +eric.snow

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35971] Documentation should warn about code injection from current working directory

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

If someone wants to change the Python default behavior, a PEP will be required 
since it would be a major backward incompatible changes. A lot of details and 
use cases must be discussed and documented.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

2019-02-12 Thread Eric Snow


Eric Snow  added the comment:

related: issue #13475

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

2019-02-12 Thread Eric Snow


Eric Snow  added the comment:

@Guido, I recall a while back you explained the value of adding CWD to 
sys.path.  Would you mind providing some insight here?

--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35971] Documentation should warn about code injection from current working directory

2019-02-12 Thread Eric Snow


Eric Snow  added the comment:

related: issue #13475

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35971] Documentation should warn about code injection from current working directory

2019-02-12 Thread Eric Snow


Eric Snow  added the comment:

Note that I've asked Guido to comment about CWD in sys.path on issue #35969.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35972] _xxsubinterpreters: channel_send() may truncate ints on 32-bit platforms

2019-02-12 Thread Eric Snow


Eric Snow  added the comment:


New changeset 16f842da3c862d76a1177bd8ef9579703c24fa5a by Eric Snow (Alexey 
Izbyshev) in branch 'master':
bpo-35972: _xxsubinterpreters: Fix potential integer truncation on 32-bit in 
channel_send() (gh-11822)
https://github.com/python/cpython/commit/16f842da3c862d76a1177bd8ef9579703c24fa5a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35972] _xxsubinterpreters: channel_send() may truncate ints on 32-bit platforms

2019-02-12 Thread Eric Snow


Eric Snow  added the comment:

Thanks, Alexey!

--
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

2019-02-12 Thread Paul Ganssle


Paul Ganssle  added the comment:

> @Guido, I recall a while back you explained the value of adding CWD to 
> sys.path.  Would you mind providing some insight here

I think bpo-35971 or bpo-13475 are better places to discuss that, because this 
issue doesn't require CWD to be in the path. You can trigger this by any 
mechanism that would raise an exception in the Python code used to initialize 
the interpreter.

Even if we restrict it to the most common case - shadowing a part of the 
standard library - you would still be able to trigger this as long as you have 
*some* mechanism to manipulate the python path in the environment that allows 
you to put paths on there with higher precedence than the standard library.

I think the core question in this issue is whether errors in the Python code 
run during the interpreter initialization should crash the interpreter or exit 
gracefully with an error code.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35969] Interpreter crashes with "can't initialize init_sys_streams" when abc fails to import

2019-02-12 Thread Guido van Rossum


Change by Guido van Rossum :


--
nosy:  -gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35977] test_slice crashed on s390x Debian 3.x: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

When the test has been ran one more time in verbose mode, it passed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35977] test_slice crashed on s390x Debian 3.x: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread STINNER Victor


New submission from STINNER Victor :

s390x Debian 3.x:
https://buildbot.python.org/all/#/builders/13/builds/2344

0:06:26 load avg: 0.92 [291/420/1] test_slice crashed (Exit code -6)
Modules/gcmodule.c:110: gc_decref: Assertion "gc_get_refs(g) > 0" failed: 
refcount is too small
Enable tracemalloc to get the memory block allocation traceback

object  : .BadCmp object at 
0x3ff93967e90>
type: BadCmp
refcount: 1
address : 0x3ff93967e90
Fatal Python error: _PyObject_AssertFailed

Current thread 0x03ff95272700 (most recent call first):
  File 
"/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/test/test_slice.py",
 line 107 in 
  File 
"/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/unittest/case.py", 
line 197 in handle
  File 
"/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/unittest/case.py", 
line 782 in assertRaises
  File 
"/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/test/test_slice.py",
 line 107 in test_cmp
  File 
"/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/unittest/case.py", 
line 642 in run
  File 
"/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/unittest/case.py", 
line 702 in __call__
  ...

--
components: Interpreter Core
messages: 335320
nosy: pablogsal, vstinner
priority: normal
severity: normal
status: open
title: test_slice crashed on s390x Debian 3.x: gc_decref: Assertion 
"gc_get_refs(g) > 0" failed: refcount is too small
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35977] test_slice crashed on s390x Debian 3.x: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

I'm unable to reproduce the issue on my Fedora 29 (x86-64) laptop. I used two 
terminals:

* ./python -m test -F -j0 test_slice test_slice test_slice test_slice 
test_slice test_slice test_slice test_slice
* ./python -m test -j0 -u all,-gui -r # stress the system to randomize timings 
in test_slice

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35977] test_slice crashed on s390x Debian 3.x: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread Karthikeyan Singaravelan


Karthikeyan Singaravelan  added the comment:

Duplicate of issue35961 ? It occurs in Travis too

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35977] test_slice crashed on s390x Debian 3.x: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread Karthikeyan Singaravelan


Karthikeyan Singaravelan  added the comment:

Sorry, it occurs in Travis too randomly not consistently.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35976] Enable Windows projects to build with platform ARM32

2019-02-12 Thread Steve Dower


New submission from Steve Dower :

Just updating the title to reflect what the patch actually does in isolation - 
you probably should update the PR and NEWS items to match.

It's fine (probably preferred) to mention in the issue and PR text that it's 
been split out from another PR. But we have to account for the (very small) 
chance that this part is merged and the rest is not - in that case, having a 
standalone description of the change is much more helpful.

--
title: PCBuild file changes for arm32 should be separated from code changes for 
review -> Enable Windows projects to build with platform ARM32

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35977] test_slice crashed on s390x Debian 3.x: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread STINNER Victor


Change by STINNER Victor :


--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is 
too small

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35961] gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

bpo-35977 has been marked as a duplicate, copy of the first message:

s390x Debian 3.x:
https://buildbot.python.org/all/#/builders/13/builds/2344

0:06:26 load avg: 0.92 [291/420/1] test_slice crashed (Exit code -6)
Modules/gcmodule.c:110: gc_decref: Assertion "gc_get_refs(g) > 0" failed: 
refcount is too small
Enable tracemalloc to get the memory block allocation traceback

object  : .BadCmp object at 
0x3ff93967e90>
type: BadCmp
refcount: 1
address : 0x3ff93967e90
Fatal Python error: _PyObject_AssertFailed

Current thread 0x03ff95272700 (most recent call first):
  File 
"/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/test/test_slice.py",
 line 107 in 
  File 
"/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/unittest/case.py", 
line 197 in handle
  File 
"/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/unittest/case.py", 
line 782 in assertRaises
  File 
"/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/test/test_slice.py",
 line 107 in test_cmp
  File 
"/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/unittest/case.py", 
line 642 in run
  File 
"/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/unittest/case.py", 
line 702 in __call__
  ...

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35961] test_slice: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread STINNER Victor


Change by STINNER Victor :


--
title: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small 
-> test_slice: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is 
too small

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35961] test_slice: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

> I am seeing some failures in travis and some buildbots:

Do you have an url to an example of crash on Travis?

Which buildbots are affected? I saw the crash on s390x Debian 3.x.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35961] test_slice: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

I'm unable to reproduce the issue on my Fedora 29 (x86-64) laptop. I used two 
terminals:

* ./python -m test -F -j0 test_slice test_slice test_slice test_slice 
test_slice test_slice test_slice test_slice
* ./python -m test -j0 -u all,-gui -r # stress the system to randomize timings 
in test_slice

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35961] test_slice: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

Extract of the crashing test:

def test_cmp(self):
...

class Exc(Exception):
pass

class BadCmp(object):
def __eq__(self, other):
raise Exc

s1 = slice(BadCmp())
s2 = slice(BadCmp())
self.assertEqual(s1, s1)
self.assertRaises(Exc, lambda: s1 == s2)   # < CRASH HERE

...

The latest change in test_slice.py has been made in 2016: 3 years ago.

Latest changes in Objects/sliceobject.c were refactoring related to header 
files (november 2018). The latest "significant" change was done in April 2018: 
"bpo-33012: Fix invalid function cast warnings with gcc 8 for METH_NOARGS".

I bet more on a recent change which is not directly related to slices... like 
the implementation of the PEP 572? It would help to know when the crash started 
to occur exactly (date + time).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35961] test_slice: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

Lukasz: I'm not sure what is happening here, but such random bugs are always 
scrary. I raise the priority to "release blocker" until we know more about the 
issue.

--
nosy: +lukasz.langa
priority: normal -> release blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35961] test_slice: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

On s390x Debian 3.x, it seems like the bug started to show up near:
https://buildbot.python.org/all/#/builders/13/builds/2328

This build contains 2 changes, one doc change and this namedtuple change which 
mentions reference count:

https://github.com/python/cpython/commit/64360ada0f6123a051e9dc6cd04f030ec1322e46

"Complete and neaten-up namedtuple's replacement of builtin function lookups 
with derefs (GH-11794)"

Ok, but I don't see the relationship between namedtupled and this specific 
test!?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35961] test_slice: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread Karthikeyan Singaravelan


Karthikeyan Singaravelan  added the comment:

> Do you have an url to an example of crash on Travis?

Travis crash that occurred few days back : 
https://travis-ci.org/python/cpython/jobs/491701299 . I first noticed it on an 
IDLE lib PR where this failure was unrelated but I can remember occurring at 
least few weeks back. I will try get the older one. Sorry I forgot to file an 
issue during that time.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35961] test_slice: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

Ah! I found a way to reproduce the crash:

diff --git a/Lib/test/test_slice.py b/Lib/test/test_slice.py
index 4ae4142c60..b18f8f1c8c 100644
--- a/Lib/test/test_slice.py
+++ b/Lib/test/test_slice.py
@@ -1,4 +1,8 @@
 # tests for slice objects; in particular the indices method.
+import gc, random
+n = random.randint(1, 700)
+print("set_threshold(%s)" % n)
+gc.set_threshold(n)
 
 import itertools
 import operator


$ ./python -m test -F -j0 test_slice test_slice test_slice test_slice 
test_slice test_slice test_slice test_slice


I went up to January 1st, 2018 (commit 
e8ed96550c6aa9a1e39c36e67e892994e25e2c41): the test already crashed with my 
patch, so it's not a regression. I reset the priority.

--
priority: release blocker -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35961] test_slice: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

See also this idea of changing gc.set_threshold() when using -X dev:
https://mail.python.org/pipermail/python-dev/2018-June/153857.html

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35961] test_slice: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread Karthikeyan Singaravelan


Karthikeyan Singaravelan  added the comment:

It has been crashing for almost all the last 5 PRs merged to master : 

https://travis-ci.org/python/cpython/jobs/492241988 - Feb 12, 21:51
https://travis-ci.org/python/cpython/jobs/492123432 - Feb 12, 17:33
https://travis-ci.org/python/cpython/jobs/491917875 - Feb 12, 06:20
https://travis-ci.org/python/cpython/jobs/491898690 - Feb 12, 05:15
https://travis-ci.org/python/cpython/jobs/491798997 - Feb 12, 01:19

It's easy to overlook this since PR passes on verbose mode . There is another 
test these days that fails very frequently but passes on verbose mode : 
issue35809 . It's little hard to track their first failure since Travis 
provides no search capability for logs.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue5996] abstract class instantiable when subclassing built-in types

2019-02-12 Thread Josh Rosenberg


Josh Rosenberg  added the comment:

Closed #35958 as a duplicate of this issue (and updated the title, since 
clearly the problem is not specific to dict).

Patch probably needs to be rebased/rewritten against latest trunk (given it 
dates from Mercurial days).

--
nosy: +Jon McMahon, josh.r
stage:  -> patch review
title: abstract class instantiable when subclassing dict -> abstract class 
instantiable when subclassing built-in types
versions: +Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue5996] abstract class instantiable when subclassing built-in types

2019-02-12 Thread Rémi Lapeyre

Change by Rémi Lapeyre :


--
nosy: +remi.lapeyre

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35918] multiprocessing's SyncManager.dict.has_key() method is broken

2019-02-12 Thread Josh Rosenberg


Change by Josh Rosenberg :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35978] test_venv fails in Travis with GCC

2019-02-12 Thread Karthikeyan Singaravelan


New submission from Karthikeyan Singaravelan :

I noticed this while checking issue35961. test_venv is always failing on GCC 
which is marked as optional in Travis.

Log : https://travis-ci.org/python/cpython/jobs/492123436#L1909

0:39:35 load avg: 1.00 [390/416] test_venv
test test_venv failed -- Traceback (most recent call last):
  File "/home/travis/build/python/cpython/Lib/test/test_venv.py", line 309, in 
test_multiprocessing
out, err = check_output([envpy, '-c',
  File "/home/travis/build/python/cpython/Lib/test/test_venv.py", line 37, in 
check_output
raise subprocess.CalledProcessError(
subprocess.CalledProcessError: Command '['/tmp/tmpg8ubeyfn/bin/python', '-c', 
'from multiprocessing import Pool; 
print(Pool(1).apply_async("Python".lower).get(3))']' died with 
.

Also GCC test have been timing out for at least past 4 months : 
https://python.zulipchat.com/#narrow/stream/116742-core.2Fhelp/topic/GCC.20build.20in.20Travis.20always.20times.20out

--
messages: 335338
nosy: pablogsal, vstinner, xtreak
priority: normal
severity: normal
status: open
title: test_venv fails in Travis with GCC
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35961] test_slice: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

I removed Lukasz from the nosy list since it's not a regression. No need to 
spam our 3.8 release manager ;-)

--
nosy:  -lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue5996] abstract class instantiable when subclassing built-in types

2019-02-12 Thread Guido van Rossum


Change by Guido van Rossum :


--
nosy:  -gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35973] `growable_int_array type_ignores` in parsetok.c is not always freed.

2019-02-12 Thread Brennan Vincent


Brennan Vincent  added the comment:

Hi Guido,

I have tried applying your patch. It seems to fix the issue (Valgrind no longer 
reports memory definitely lost).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35961] test_slice: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread STINNER Victor


Change by STINNER Victor :


--
keywords: +patch
pull_requests: +11859
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35961] test_slice: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

Extract of slice_richcompare():

t1 = PyTuple_New(3);
if (t1 == NULL)
return NULL;
t2 = PyTuple_New(3);
if (t2 == NULL) {
Py_DECREF(t1);
return NULL;
}

PyTuple_SET_ITEM(t1, 0, ((PySliceObject *)v)->start);
PyTuple_SET_ITEM(t1, 1, ((PySliceObject *)v)->stop);
PyTuple_SET_ITEM(t1, 2, ((PySliceObject *)v)->step);
PyTuple_SET_ITEM(t2, 0, ((PySliceObject *)w)->start);
PyTuple_SET_ITEM(t2, 1, ((PySliceObject *)w)->stop);
PyTuple_SET_ITEM(t2, 2, ((PySliceObject *)w)->step);

res = PyObject_RichCompare(t1, t2, op);

PyTuple_SET_ITEM(t1, 0, NULL);
PyTuple_SET_ITEM(t1, 1, NULL);
PyTuple_SET_ITEM(t1, 2, NULL);
PyTuple_SET_ITEM(t2, 0, NULL);
PyTuple_SET_ITEM(t2, 1, NULL);
PyTuple_SET_ITEM(t2, 2, NULL);

Py_DECREF(t1);
Py_DECREF(t2);

return res;

t1 and t2 tuples are tracked by the GC, but t1 and t2 items are *borrowed* 
references.

gc.collect() calls subtract_refs() which decreases the reference counter of all 
objects tracked by the GC... but v, w, t1 and t2 contain the same references 
which causes the assertion error.

The code is quite old:

commit 47b9ff6ba11fab4c90556357c437cb4feec1e853
Author: Guido van Rossum 
Date:   Thu Aug 24 00:41:19 2006 +

Restructure comparison dramatically.  There is no longer a default
*ordering* between objects; there is only a default equality test
(defined by an object being equal to itself only).  Read the comment
in object.c.  The current implementation never uses a three-way
comparison to compute a rich comparison, but it does use a rich
comparison to compute a three-way comparison.  I'm not quite done
ripping out all the calls to PyObject_Compare/Cmp, or replacing
tp_compare implementations with tp_richcompare implementations;
but much of that has happened (to make most unit tests pass).
(...)

+static PyObject *
+slice_richcompare(PyObject *v, PyObject *w, int op)
...
+   t1 = PyTuple_New(3);
+   t2 = PyTuple_New(3);
+   if (t1 == NULL || t2 == NULL)
+   return NULL;
+
+   PyTuple_SET_ITEM(t1, 0, ((PySliceObject *)v)->start);
+   PyTuple_SET_ITEM(t1, 1, ((PySliceObject *)v)->stop);
+   PyTuple_SET_ITEM(t1, 2, ((PySliceObject *)v)->step);
+   PyTuple_SET_ITEM(t2, 0, ((PySliceObject *)w)->start);
+   PyTuple_SET_ITEM(t2, 1, ((PySliceObject *)w)->stop);
+   PyTuple_SET_ITEM(t2, 2, ((PySliceObject *)w)->step);
...

I see two options:

* Ensure that t1 and t2 temporary objects are not tracked by the GC
* Modify slice_richcompare() to use strong references, rather than *borrowed* 
references

I wrote PR 11828 which implements the first solution. The second solution has a 
minor impact on performance.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35505] Test test_imaplib fail in test_imap4_host_default_value

2019-02-12 Thread miss-islington


Change by miss-islington :


--
pull_requests: +11860

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35505] Test test_imaplib fail in test_imap4_host_default_value

2019-02-12 Thread STINNER Victor

STINNER Victor  added the comment:


New changeset 3dc67d0316740e78e7cd014343f34d85908219b7 by Victor Stinner (Matěj 
Cepl) in branch 'master':
bpo-35505: Skip test_imap4_host_default_value if localhost listens on IMAP port 
(GH-11823)
https://github.com/python/cpython/commit/3dc67d0316740e78e7cd014343f34d85908219b7


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35810] Object Initialization Bug with Heap-allocated Types

2019-02-12 Thread Stefan Behnel


Stefan Behnel  added the comment:

Victor asked me for a review, so, well, what should I say? The intention seems 
right, and the patch also looks good to me.

>From the top of my head, I wouldn't know any problems this would produce with 
>Cython specifically, although it's worth testing. If we find anything, then 
>it's hopefully easy to adapt to in a point release, which Cython users can 
>then build their code with to support Py3.8+. That's the way it usually works 
>with Cython.

The main problem I see is that while this change may crash in some cases (the 
lucky cases that are easy to detect), it will leak references in others, which 
can be really difficult to detect. My own biased little gut feeling still wants 
me to believe that the impact of this could be somewhat low. Why? Well, how 
many heap-allocated types with a custom "tp_dealloc()" do you expect there to 
be? My feeling is that most existing code still uses statically allocated types 
for that. CPython has a couple of examples (that the PR adapts), but IIRC, 
that's mostly because some core devs wanted to test-ride parts of the newer 
type creation C-API in the standard library (a perfectly valid reason, but that 
also makes it a bad example). From the little valley that I sit in, I don't see 
a large bunch of other usages of that API out in the wild. That doesn't mean 
they are not there, and there might well be some large projects that could be 
bitten by this change. But I'm sure it's not the majority.

So, on the one hand, any breaking change to the C-API may make users end up 
with little maintained projects that they depend on and that break in Py3.y and 
later without anyone having access to the PyPI project account to push a fix 
release. Very annoying situation.

On the other hand, a breaking C-API change is not the only problematic case, 
and people have to deal with similar situations anyway. CPython changes are 
really just one of many, many ways to render your code unusable.

I would suggest clear, open communication of this. It's solving a bug. It makes 
CPython safer. It's not difficult to adapt your code, once you know it's 
affected. The usual PY_VERSION_HEX guard will do it for you. I think we should 
risk it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35961] test_slice: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

It's impressive *and* scary that such 13 years old bug only show up today...

I modified the GC threshold in the site module with this patch:

diff --git a/Lib/site.py b/Lib/site.py
index ad1146332b..68cdf105b1 100644
--- a/Lib/site.py
+++ b/Lib/site.py
@@ -557,6 +557,10 @@ def main():
 This function is called automatically when this module is imported,
 unless the python interpreter was started with the -S flag.
 """
+import gc, random
+n = random.randint(5, 100)
+gc.set_threshold(n)
+
 global ENABLE_USER_SITE
 
 orig_path = sys.path[:]


Running the test suite with this patch doesn't catch the bug :-(

$ ./python -m test -j0 -r -u all,-gui

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35505] Test test_imaplib fail in test_imap4_host_default_value

2019-02-12 Thread miss-islington


miss-islington  added the comment:


New changeset 009220eae2e6b456707c1c89ef3f344b682e89d5 by Miss Islington (bot) 
in branch '3.7':
bpo-35505: Skip test_imap4_host_default_value if localhost listens on IMAP port 
(GH-11823)
https://github.com/python/cpython/commit/009220eae2e6b456707c1c89ef3f344b682e89d5


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35961] test_slice: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread Josh Rosenberg


Josh Rosenberg  added the comment:

Victor found the same bug I found while I was composing this, posting only to 
incorporate proposed solution:

I *think* I have a cause for this, but someone else with greater understanding 
of the cycle collector should check me if the suggested fix has non-trivial 
performance implications (I suspect the answer is no, performance is 
unaffected).

slice_richcompare borrows its behavior from tuple by creating a temporary tuple 
for each slice, the delegating to the tuple comparison ( 
https://github.com/python/cpython/blob/master/Objects/sliceobject.c#L591 ).

Problem is, it uses borrowed references when creating said tuples, not owned 
references. Because test_slice's BadCmp.__eq__ is implemented in Python, the 
comparison can be interrupted by cycle collection during the __eq__ call. When 
then happens, there are precisely two references to the BadCmp object:

1. In the slice (owned)
2. In the temporary tuple (borrowed)

When a cycle collection occurs during the comparison, and subtract_refs ( 
https://github.com/python/cpython/blob/master/Modules/gcmodule.c#L398 ) is 
called, the BadCmp object in question is visited via both the slice and the 
tuple, and since it has no non-container objects referencing it, it ends up 
with the initial reference count of 1 attempting to drop to -1, and the 
assertion is violated. While the code of gcmodule.c appears to have been 
refactored since 3.7 so the assert occurs in a different function, with a 
slightly different message, it would break the same way in both 3.7 and master, 
and whether or not it triggers the bug, the broken behavior of 
slice_richcompare hasn't changed for a *long* time. 

Underlying problem would seem to be slice's richcompare believing it's okay to 
make a tuple from borrowed references, then make a call on it that can trigger 
calls into Python level code (and therefore into the cycle collector); 
everything else is behaving correctly here. I'm guessing the only reason it's 
not seen in the wild is that slices based on Python defined types are almost 
never compared at all, let alone compared on debug builds that would be 
checking the assert and with an accelerated cycle collection cycle that would 
make a hit likely.

Solution would be to stop trying to microoptimize slice_richcompare to avoid 
reference count manipulation and just build a proper tuple. It would even 
simplify the code since we could just use PyTuple_Pack, reducing custom code by 
replacing:

t1 = PyTuple_New(3);
if (t1 == NULL)
return NULL;
t2 = PyTuple_New(3);
if (t2 == NULL) {
Py_DECREF(t1);
return NULL;
}

PyTuple_SET_ITEM(t1, 0, ((PySliceObject *)v)->start);
PyTuple_SET_ITEM(t1, 1, ((PySliceObject *)v)->stop);
PyTuple_SET_ITEM(t1, 2, ((PySliceObject *)v)->step);
PyTuple_SET_ITEM(t2, 0, ((PySliceObject *)w)->start);
PyTuple_SET_ITEM(t2, 1, ((PySliceObject *)w)->stop);
PyTuple_SET_ITEM(t2, 2, ((PySliceObject *)w)->step);

with:

t1 = PyTuple_Pack(3, ((PySliceObject *)v)->start, ((PySliceObject 
*)v)->stop, ((PySliceObject *)v)->step);
if (t1 == NULL)
return NULL;
t2 = PyTuple_Pack(3, ((PySliceObject *)w)->start, ((PySliceObject 
*)w)->stop, ((PySliceObject *)w)->step);
if (t2 == NULL) {
Py_DECREF(t1);
return NULL;
}

and makes cleanup simpler, since you can just delete:

PyTuple_SET_ITEM(t1, 0, NULL);
PyTuple_SET_ITEM(t1, 1, NULL);
PyTuple_SET_ITEM(t1, 2, NULL);
PyTuple_SET_ITEM(t2, 0, NULL);
PyTuple_SET_ITEM(t2, 1, NULL);
PyTuple_SET_ITEM(t2, 2, NULL);

and let the DECREFs for t1/t2 do their work normally.

If for some reason the reference count manipulation is unacceptable, this 
*could* switch between two behaviors depending on whether or not 
start/stop/step are of known types (e.g. if all are NoneType/int, this could 
use the borrowed refs code path safely) where a call back into Python level 
code is impossible; given that slices are usually made of None and/or ints, 
this would remove most of the cost for the common case, at the expense of more 
complicated code. Wouldn't help numpy types though, and I suspect the cost of 
pre-checking the types for all six values involved would eliminate most of the 
savings.

Sorry for not submitting a proper PR; the work machine I use during the day is 
not suitable for development (doesn't even have Python installed).

--
nosy: +josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35961] test_slice: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread Tim Peters


Tim Peters  added the comment:

> It's impressive *and* scary that such 13 years old
> bug only show up today...

Then again, there's probably no other code in the world that compares slice 
objects ;-)

--
nosy: +tim.peters -josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35810] Object Initialization Bug with Heap-allocated Types

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

> I would suggest clear, open communication of this. It's solving a bug. It 
> makes CPython safer. It's not difficult to adapt your code, once you know 
> it's affected. The usual PY_VERSION_HEX guard will do it for you. I think we 
> should risk it.

It would be nice to give an example of code in the Porting guide (What's new in 
Python 3.8?).

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35961] test_slice: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread Josh Rosenberg


Josh Rosenberg  added the comment:

Ah, I see Victor posted an alternative PR that avoids the reference counting 
overhead by explicitly removing the temporary tuples from GC tracking. I'm 
mildly worried by that approach, only because the only documented use case for 
PyObject_GC_UnTrack is in tp_dealloc (that said, the code explicitly allows it 
to be called twice due to the Py_TRASHCAN mechanism, so it's probably safe so 
long as the GC design never changes dramatically). If slice comparison really 
is performance sensitive enough to justify this, so be it, but I'd personally 
prefer to reduce the custom code involved in a rarely used code path (we're not 
even caching constant slices yet, so no comparisons are likely to occur for 
99.99% of slices, right?).

--
nosy: +josh.r
versions: +Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35961] test_slice: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread STINNER Victor


Change by STINNER Victor :


--
pull_requests: +11861

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35961] test_slice: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

I wrote PR 11830: the other solution that I propose, use strong references 
rather than borrowed references.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35961] test_slice: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread Tim Peters


Tim Peters  added the comment:

Josh, I'd say the speed of this code doesn't matter one whit to anything.  Like 
you, I'd _prefer_ that the issue be fixed by building "real tuples" that own 
their own references, which would also (as you showed) allow for briefer, 
simpler, clearer code.  But I'm OK with playing obscure GC tricks too.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35961] test_slice: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

> Then again, there's probably no other code in the world that compares slice 
> objects ;-)

Well, that's a good explanation :-) So maybe there is no need to keep the 
micro-optimization and PR 11830 would be safer (use strong references), rather 
than PR 11828 (untrack the 2 internal tuples).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35961] test_slice: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

Oh, Tim Peters succeded somehow to remove Josh Rosenberg from the nosy list: I 
add him again ;-)

Thanks Josh for your analysis, it seems like we agree :-) I wrote PR 11830 to 
use strong references, as you proposed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35978] test_venv fails in Travis with GCC

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

Pablo: Can this issue be related bpo-35378?

commit 3766f18c524c57784eea7c0001602017d2122156
Author: Pablo Galindo 
Date:   Mon Feb 11 17:29:00 2019 +

bpo-35378: Fix multiprocessing.Pool references (GH-11627)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35961] test_slice: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread Tim Peters


Tim Peters  added the comment:

> Oh, Tim Peters succeded somehow to
> remove Josh Rosenberg from the nosy list: I add him again ;-)

Certainly wasn't my intent!  That happens too often on the tracker.  Thanks for 
noticing! :-(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35961] test_slice: gc_decref: Assertion "gc_get_refs(g) > 0" failed: refcount is too small

2019-02-12 Thread Josh Rosenberg


Josh Rosenberg  added the comment:

+1 on PR 11830 from me. Can always revisit if #11107 is ever implemented and it 
turns out that the reference count manipulation means startup is too slow due 
to all the slice interning triggered comparisons (unlikely, but theoretically 
possible I guess).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34172] multiprocessing.Pool and ThreadPool leak resources after being deleted

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

Pablo fixed bpo-35378 with:

New changeset 3766f18c524c57784eea7c0001602017d2122156 by Pablo Galindo in 
branch 'master':
bpo-35378: Fix multiprocessing.Pool references (GH-11627)
https://github.com/python/cpython/commit/3766f18c524c57784eea7c0001602017d2122156

Does this change also fix this issue? If not, can we attempt again to fix this 
issue?

Moreover, should we do something in Python 3.7? Sadly, I don't think that we 
can do anything for 3.7 and 2.7.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35378] multiprocessing.Pool.imaps iterators do not maintain alive the multiprocessing.Pool objects

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

Thanks Pablo for the fix ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35978] test_venv fails in Travis with GCC

2019-02-12 Thread Pablo Galindo Salgado

Pablo Galindo Salgado  added the comment:

That looks like a similar example of the problem that bpo35378 solves. But the 
PR is merged and the test keeps failing:

test test_venv failed -- Traceback (most recent call last):
  File "/home/travis/build/python/cpython/Lib/test/test_venv.py", line 309, in 
test_multiprocessing
out, err = check_output([envpy, '-c',
  File "/home/travis/build/python/cpython/Lib/test/test_venv.py", line 37, in 
check_output
raise subprocess.CalledProcessError(
subprocess.CalledProcessError: Command '['/tmp/tmplplx90pf/bin/python', '-c', 
'from multiprocessing import Pool; 
print(Pool(1).apply_async("Python".lower).get(3))']' died with 
.

https://travis-ci.org/python/cpython/jobs/492305033


Running that command locally on master does not die with SIGABRT:

❯ ./python -c 'from multiprocessing import Pool; 
print(Pool(1).apply_async("Python".lower).get(3))'; echo $?
python
/home/pablogsal/github/cpython/Lib/multiprocessing/pool.py:234: 
ResourceWarning: unclosed running multiprocessing pool 

  _warn(f"unclosed running multiprocessing pool {self!r}",
ResourceWarning: Enable tracemalloc to get the object allocation traceback
0

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35813] shared memory construct to avoid need for serialization between processes

2019-02-12 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Davin:

> This is my fault because I altered SharedMemoryManager to no longer support 
> functionality from SyncManager that I thought could be confusing to include.  
> I am just now discovering this and am not immediately sure if simply removing 
> the SharedMemoryManager-relevant lines from your patch is the right solution 
> but I wanted to mention this thought right away.

If SharedMemoryManager subclasses SyncManager then I *think* it should obey the 
SyncManager contract.  Regardless of the shared memory facility, the user may 
also want to "shared" regular proxies between processes.

(to be honest, I don't think the multiprocessing Manager facility is used a lot 
currently...)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35978] test_venv fails in Travis with GCC

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

I can reproduce the crash using the same command than the job.

Simplified commands to reproduce the bug:

./configure && make
./python -m venv venv
./venv/bin/python -m test test_venv -m 
test.test_venv.BasicTest.test_multiprocessing

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35813] shared memory construct to avoid need for serialization between processes

2019-02-12 Thread Davin Potts


Davin Potts  added the comment:

@Antoine:  SharedMemoryManager does not subclass SyncManager but it did 
previously.  This is the source of the confusion.  SharedMemoryManager 
subclasses BaseManager which does not provide Value, Array, list, dict, etc.

Agreed that the manager facility does not appear to see that much use in 
existing code.

When working with shared memory, I expect SharedMemoryManager to be much more 
popular.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35978] test_venv fails in Travis with GCC

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

The Python of the virtual environment created by the test doesn't work:

$ /tmp/tmp0vybyos2/bin/python -c pass
Could not find platform independent libraries 
Could not find platform dependent libraries 
Consider setting $PYTHONHOME to [:]
Fatal Python error: initfsencoding: failed to get the Python codec of the 
filesystem encoding
ModuleNotFoundError: No module named 'encodings'

Current thread 0x7f2956ab8740 (most recent call first):
Aborted

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35978] test_venv fails in Travis with GCC

2019-02-12 Thread STINNER Victor


STINNER Victor  added the comment:

Hum, test_executable() of test_venv is skipped if run in a venv, but not 
test_multiprocessing():

# If a venv is created from a source build and that venv is used to
# run the test, the pyvenv.cfg in the venv created in the test will
# point to the venv being used to run the test, and we lose the link
# to the source build - so Python can't initialise properly.
@skipInVenv
...
self.run_with_capture(venv.create, self.env_dir)
envpy = os.path.join(os.path.realpath(self.env_dir),
 self.bindir, self.exe)
...

def test_multiprocessing(self):
...
self.run_with_capture(venv.create, self.env_dir)
envpy = os.path.join(os.path.realpath(self.env_dir),
 self.bindir, self.exe)
...

The test has been added by:

commit 4e02f8f8b4baab63f927cfd87b401200ba2969e9
Author: Steve Dower 
Date:   Fri Jan 25 14:59:12 2019 -0800

--
nosy: +steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35155] Clarify Protocol Handlers in urllib.request Docs

2019-02-12 Thread Terry J. Reedy


Terry J. Reedy  added the comment:

Cheryl, you know rst better than me.  Is there any markup to indicate that a 
word is a placeholder, not meant to be taken literally?  Or is Denton's 
'' instead of 'placeholder' the best choice? See the PR and answer 
there.

--
nosy: +cheryl.sabella, orsenthil, terry.reedy
versions:  -Python 3.4, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35813] shared memory construct to avoid need for serialization between processes

2019-02-12 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

> SharedMemoryManager does not subclass SyncManager but it did previously.

Ah, right.  Then it's ok to change the tests IMO.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35810] Object Initialization Bug with Heap-allocated Types

2019-02-12 Thread Eddie Elizondo


Eddie Elizondo  added the comment:

Thanks for the thorough feedback Stefan!

I would like to just add an extra thing to everything you already mentioned: 

This change will NEVER cause a crash. The change that we are introducing is an 
incref to a type object (no decrefs). Thus, there are two-ish scenarios:

1) The type object is immortal: This means that the type does not incref/decref 
its refcount automatically on instance creation. Adding this incref will not 
affect the fact that it's already immortal. An example of this is 
structsequences. The change that I added in the PR is to convert it to an 
refcounted type (instead of immortal).

2.1) The type is recounted (automatically): Achieved through the generic 
allocation which already increfs the type. Given that this refactors that 
incref, then this behavior should stay exactly the same.

2.2) The type is refcounted (manually): Achieved by not using the generic 
allocation and instead using `PyObject_{,GC}_New{Var}`. To properly refcount 
this type, a manual incref is required after object instantiation. Usually, 
I've seen this pattern in very carefully engineered types where a NULL is 
jammed into `tp_new` to make it into a non-instantiable type. Examples of this 
are Modules/_tkinter.c and Modules/_curses_panel.c. Anyways, adding this incref 
will leak this type, but will not cause a crash.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35810] Object Initialization Bug with Heap-allocated Types

2019-02-12 Thread Eddie Elizondo


Eddie Elizondo  added the comment:

Now, with that in mind - can you guys point me to the right thing to do now - 
how can we move this forward? :) 

* Should I write something up in python-dev/Discourse?
* Do I need to update the PY_VERSION_HEX?
* Do I need to write an entry in the Porting guide?

Let me know what you think!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35505] Test test_imaplib fail in test_imap4_host_default_value

2019-02-12 Thread STINNER Victor

STINNER Victor  added the comment:

Thanks Matěj Cepl for the fix and thanks Petr Stupka for the bug report ;-)

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35810] Object Initialization does not incref Heap-allocated Types

2019-02-12 Thread Stefan Behnel


Stefan Behnel  added the comment:

Adding Christian Tismer to the nosy list since he might be able to elaborate on 
the impact on PySide (which IIRC uses the stable ABI, and thus, heap types).

--
components: +Extension Modules, Interpreter Core -Library (Lib)
nosy: +Christian.Tismer
title: Object Initialization Bug with Heap-allocated Types -> Object 
Initialization does not incref Heap-allocated Types
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35155] Clarify Protocol Handlers in urllib.request Docs

2019-02-12 Thread Matthew Barnett


Matthew Barnett  added the comment:

You could italicise the "protocol" part using asterisks, like this:

*protocol*_request

or this:

*protocol*\ _request

depending on the implementation of the rst software.

--
nosy: +mrabarnett

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35967] Better platform.processor support

2019-02-12 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Your proposal sounds fine to me.  You could fall back on platform.machine() 
instead of calling `uname` explicitly.

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35155] Clarify Protocol Handlers in urllib.request Docs

2019-02-12 Thread Cheryl Sabella


Cheryl Sabella  added the comment:

I don't think there is anything specifically defined for a placeholder.  
However, looking at the current urllib.request documentation, there are 2 
places that use the ``_`` method for a placeholder - .proxy and 
.proxy, with the latter explaining what  means.

> By default, ProxyHandler uses the environment variables named _proxy, 
> where  is the URL scheme involved.

That line was added by Georg Brandl (granted it was part of converting all the 
docs), which would lead me to believe that is the accepted way to do it.

Also, the argument clinic how-to has the following:
> The length of the string will be passed in to the impl function, just after 
> the string parameter, as a parameter named ``_length``.

I'll make a comment on the PR too.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35951] os.renames() creates directories if original name doesn't exist

2019-02-12 Thread Giampaolo Rodola'


Giampaolo Rodola'  added the comment:

The proposed change makes the problem a lot less likely to occur, but 
technically it doesn't fix it because if the src file/dir disappears between 
"os.path.exists(src)" and os.rename(src, dst)" calls you'll end up with a race 
condition. We may still want to do it, but can't make promises about full 
reliability.

Also, it seems to me this behavior should be expected, because the doc explains 
the whole thing basically happens as a 3-step process (create dst dirs, rename, 
remove old src path). As such the cleanup in case of failure should not be 
expected, nor is promised. Also, FileNotFoundError is just one. os.rename() may 
fail for other reasons (and still leave the dst directory tree behind). If 
there is something we can do here is probably make the doc more clear (it talks 
about "lack of permissions" when instead it should have talked on "any error".

Extra (unrelated): as I commented on the PR, there are no unit-tests for 
os.renames().

--
nosy: +giampaolo.rodola

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35936] Give modulefinder some much-needed updates.

2019-02-12 Thread Brandt Bucher


Change by Brandt Bucher :


--
nosy: +barry, brett.cannon, ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35942] posixmodule.c:path_converter() returns an invalid exception message for broken PathLike objects

2019-02-12 Thread Pablo Galindo Salgado


Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +11862
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35973] `growable_int_array type_ignores` in parsetok.c is not always freed.

2019-02-12 Thread Pablo Galindo Salgado


Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +11863
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue21360] mailbox.Maildir should ignore files named with a leading dot

2019-02-12 Thread Zackery Spytz


Zackery Spytz  added the comment:

I've created a PR for this issue.

--
nosy: +ZackerySpytz
versions: +Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



  1   2   >