[issue36048] Deprecate implicit truncating when convert Python numbers to C integers

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36041] email: folding of quoted string in display_name violates RFC

[Aaryn Tonita]

Aaryn Tonita  added the comment:

Hi David, the problem is in email._header_value_parser._refold_parse_tree.

Specifically, when the parsetree renders too long, it recursively gets newparts 
= list(part) (the children). When it does this to a BareQuotedString node, the 
child nodes are unquoted and unescaped and it just happily serializes these.

I thought I had attached a file that monkey patches the _refold_parse_tree 
function with a fixed version... let me try again.

--
Added file: https://bugs.python.org/file48158/address_folding_bug.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36048] Deprecate implicit truncating when convert Python numbers to C integers

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Numerous explicit calls of PyNumber_Index() which are used to protect from 
passing non-integral types to PyLong_AsLong() and like can be removed after the 
end of the deprecation period. I tried to mark calls which can be removed with 
comments, but virtually all calls can be removed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36047] socket file handle does not support stream write

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

I confirm that you don't use socket.makefile in write mode.

Python 3.7.2 (default, Jan 16 2019, 19:49:22) 
[GCC 8.2.1 20181215 (Red Hat 8.2.1-6)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> s = socket.socket()
>>> s.connect('localhost', 5432)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: connect() takes exactly one argument (2 given)
>>> s.connect(('localhost', 5432))
>>> S = s.makefile()
>>> print('hello world', file=S, flush=True)
Traceback (most recent call last):
  File "", line 1, in 
io.UnsupportedOperation: not writable
>>> S = s.makefile(mode='w')
>>> print('hello world', file=S, flush=True)
>>> 

I close the issue.

--
nosy: +matrixise
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35810] Object Initialization does not incref Heap-allocated Types

[Eddie Elizondo]

Eddie Elizondo  added the comment:

Bump to get a review on the PR: https://github.com/python/cpython/pull/11661. 

I believe all comments have now been addressed as well as adding a porting to 
3.8 guide.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Zahash Z]

New submission from Zahash Z :

There is no __repr__() method for the PriorityQueue class and LifoQueue class 
in the queue.py file

This makes it difficult to check the elements of the queue.

--
messages: 336053
nosy: Zahash Z
priority: normal
severity: normal
status: open
title: No __repr__() for queue.PriorityQueue and queue.LifoQueue
type: enhancement
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

but the __repr__ would have a limitation because you can't show all the 
elements from your queue, for example, if your queue contains 1000 items, the 
__repr__ will show the total items or just the ten first ones?

We need a compromise for that.

--
nosy: +matrixise

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

and there is an other issue, we need to iterate over the elements of the 
deque() :/ not really performant, just for a repr().

for my part, -1 if we have to iterate over the elements.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35925] test_httplib test_nntplib test_ssl fail on ARMv7 Debian buster bot (OpenSSL 1.1.1a)

[Charalampos Stratakis]

Charalampos Stratakis  added the comment:

SSLContext.minimum_version is added here on the master branch:

https://github.com/python/cpython/commit/698dde16f60729d9e3f53c23a4ddb8e5ffe818bf

But I'd be also reluctant to partially backport a new feature to fix the test 
suite.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Windson Yang]

Windson Yang  added the comment:

Hello, Zahash Z. May I ask what is your expected behavior for the return value 
of __repr__() from PriorityQueue. I'm agreed with Stéphane Wirtel, I don't 
think returning all the items would be a good idea, maybe we can improve it in 
another way. For instance, return the first and last element as well as the 
length.

--
nosy: +Windson Yang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36042] Setting __init_subclass__ and __class_getitem__ methods are in runtime doesnt make them class method.

[INADA Naoki]

INADA Naoki  added the comment:

__new__ is special function too.  It is converted as staticmethod when class 
creation.

Since manually converting to (static|class)method is allowed, I don't think
automatic conversion should be applied when attach after class creation.

$ python3
Python 3.7.2 (default, Feb 12 2019, 08:15:36)
[Clang 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> class C:
...   def __new__(cls):
... pass
...
>>> type(C.__dict__['__new__'])

>>> class D:
...   pass
...
>>> D.__new__ = lambda cls: cls
>>> type(D.__dict__['__new__'])


--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10278] add time.monotonic() method

[STINNER Victor]

STINNER Victor  added the comment:

This issue has been followed by the PEP 418 which added time.monotonic(), 
time.perf_counter() and time.process_time().

--
title: add time.wallclock() method -> add time.monotonic() method

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23428] Use the monotonic clock for thread conditions on POSIX platforms

[STINNER Victor]

STINNER Victor  added the comment:

INADA-san fixed bpo-12822 with:

New changeset 001fee14e0f2ba5f41fb733adc69d5965925a094 by Inada Naoki in branch 
'master':
bpo-12822: use monotonic clock for condvar if possible (GH-11723)
https://github.com/python/cpython/commit/001fee14e0f2ba5f41fb733adc69d5965925a094

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12822] NewGIL should use CLOCK_MONOTONIC if possible.

[STINNER Victor]

STINNER Victor  added the comment:

Thanks INADA-san for fixing this old issue!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

>>> from collections import deque
>>> d = deque()
>>> d.append('j')
>>> d.appendleft('f')
>>> d
deque(['f', 'j'])
>>> repr(d)
"deque(['f', 'j'])"

Maybe there is a solution,

in the code of deque_repr, we convert the deque to a list. We could do the same 
thing and take the first and the last element.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36031] add internal API function to effectively convert just created list to tuple

[STINNER Victor]

STINNER Victor  added the comment:

_PyList_ConvertToTuple(PyObject *v):
assert(Py_REFCNT(v) == 1);

I don't think that _PyList_ConvertToTuple() usage is common enough to justify 
this micro-optimization.

IMHO "Py_REFCNT(v) == 1" assumption is too strong. Python internals can be very 
surprising, especially when borrowered references and the garbage collector 
comes into the game.

I concur with Serhiy and Raymond: it's too risky with very low benefit. It is 
likely to have no significant impact on macro benchmarks like 
https://pyperformance.readthedocs.io/ ( https://speed.python.org/ ).

--
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35840] Control flow inconsistency on closed asyncio stream

[Emmanuel Arias]

Change by Emmanuel Arias :


--
nosy: +eamanu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36021] [Security][Windows] webbrowser: WindowsDefault uses os.startfile() and so can be abused to run arbitrary commands

[STINNER Victor]

STINNER Victor  added the comment:

> >>> os.startfile('file:///C:/Temp/test/test.exe')

Oh, startfile() also runs a program for an URL using file:// scheme? If yes, it 
becomes even more complex to fix this file :-/

How do you decide if an URL start with file:// is safe?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36030] add internal API function to create tuple without items array initialization

[STINNER Victor]

STINNER Victor  added the comment:

Raymond:
> Also, the timing loop is unrealistic.  We mostly care about small tuples.

Sergey: can you please run benchmarks on small tuples? Example, 1, 5, 10, 20 
and 100 items. Maybe not only tuple(list), but try some other operations?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35812] Don't log an exception from the main coroutine in asyncio.run()

[Emmanuel Arias]

Change by Emmanuel Arias :


--
nosy: +eamanu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

For this script, you could have the following output:

from queue import Queue
q = Queue()
print(repr(q))
q.put('a')
print(repr(q))
q.put('b')
print(repr(q))
q.put('c')
print(repr(q))


Queue()
Queue('a')
Queue('a','b')
Queue('a'...'c')

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

New output for Queue, PriorityQueue and LifoQueue


Queue()
Queue('a')
Queue('a','b')
Queue('a'...'c')
PriorityQueue()
PriorityQueue('a')
PriorityQueue('a','b')
PriorityQueue('a'...'c')
LifoQueue()
LifoQueue('a')
LifoQueue('a','b')
LifoQueue('a'...'c')

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Stéphane Wirtel]

Change by Stéphane Wirtel :


--
keywords: +patch
pull_requests: +11978
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

Just created a PR for this issue and I need to add the tests for the __repr__ 
method

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36048] Deprecate implicit truncating when convert Python numbers to C integers

[STINNER Victor]

STINNER Victor  added the comment:

I like the idea. Rejecting float but not decimal.Decimal is inconsistent. 
__index__ has been written explicitly for this purpose.

I'm always confused and lost in subtle details of the Python and C API in how 
they handle numbers, so I wrote some notes for myself:
https://pythondev.readthedocs.io/numbers.html

Some functions accept only int, others use __int__, others __index__. Some 
functions silently truncate into 32 or 64-bit signed integer. Some other raise 
a OverflowError or ValueError...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34486] "RuntimeError: release unlocked lock" when starting a thread

[Radek]

Radek  added the comment:

Any progress on this topic? I think I've encountered this (or similar) issue:

>Traceback (most recent call last):
>  File "logging/__init__.py", line 1944, in shutdown
>  File "logging/__init__.py", line 813, in acquire
>  File "site-packages/utils/signals.py", line 58, in signal_wrapper
>  File "utils/utils.py", line 147, in sigterm_handler
>SystemExit: 0
>
>During handling of the above exception, another exception occurred:
>
>Traceback (most recent call last):
>  File "logging/__init__.py", line 1954, in shutdown
>  File "logging/__init__.py", line 821, in release
>RuntimeError: cannot release un-acquired lock

--
nosy: +radek_kujawa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36048] Deprecate implicit truncating when convert Python numbers to C integers: use __index__, not __int__

[STINNER Victor]

Change by STINNER Victor :


--
title: Deprecate implicit truncating when convert Python numbers to C integers 
-> Deprecate implicit truncating when convert Python numbers to C integers: use 
__index__, not __int__

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36048] Deprecate implicit truncating when convert Python numbers to C integers: use __index__, not __int__

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-34423: "Overflow when casting from double to time_t, and_PyTime_t" 
or "How to reduce precision loss when converting arbitrary number to int or 
float?".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34423] Overflow when casting from double to time_t, and_PyTime_t.

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-36048: "Deprecate implicit truncating when convert Python numbers 
to C integers: use __index__, not __int__".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36030] add internal API function to create tuple without items array initialization

[Sergey Fedoseev]

Change by Sergey Fedoseev :


--
pull_requests: +11979

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Zahash Z]

Zahash Z  added the comment:

I raised this issue after creating a pull request and modifying the source 
code. so, people who want to create a pull request can sit back and relax cuz I 
got this

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Roundup Robot]

Change by Roundup Robot :


--
pull_requests: +11980

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36030] add internal API function to create tuple without items array initialization

[STINNER Victor]

STINNER Victor  added the comment:

I prefer PyTuple_FromArray() API idea. It's safer by design than you "list to 
tuple" function stealing references to list items if refcnt(list)==1.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22213] Make pyvenv style virtual environments easier to configure when embedding Python

[INADA Naoki]

Change by INADA Naoki :


--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Rémi Lapeyre]

Rémi Lapeyre  added the comment:

Hi @matrixise, is it really an issue to iterate over all the elements of the 
deque? 


__repr__ won't be called on performance sensitive path and we already do this 
for repr([0]*1), repr({i for i in range(1)}) and repr({i:i for i in 
range(1)})

--
nosy: +remi.lapeyre

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

Yep, we have a problem, two PRs for the same issue.

I have seen your issue in the bug tracker (not marked as "in progress") and I 
started to fix it where I have proposed some solutions and a PR.

For the next time, maybe you should create the issue, indicate you are working 
on the issue and fix it. 

I am really sorry, I didn't see that you was working on your issue because 
there was no associated PR on the issue. And now, I understand because the 
title of your PR was "issue #36049; added __repr__() for the 
queue.PriorityQueue and queue.LifoQueue classes".

Bedevere (a BOT) tries to map a PR with an issue if the title is correctly 
defined in Github.

For example: "bpo-36049: Added __repr__() for the queue.PriorityQueue and 
queue.LifoQueue classes"

Have a nice day,

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

@remi.lapeyre

In fact, I am not sure about the performance issue but in the code of 
_collectionsmodule.c#deque_repr we create a PySequence_List, it's a new list. 
So for me, but maybe I am wrong, we will iterate over the deque container for 
the creation of the new list.

now, I think it's not a big issue (to confirm by @rhettinger)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

static PyObject *
deque_repr(PyObject *deque)
{
PyObject *aslist, *result;
...
aslist = PySequence_List(deque);
...
if (((dequeobject *)deque)->maxlen >= 0)
result = PyUnicode_FromFormat("%s(%R, maxlen=%zd)",
  _PyType_Name(Py_TYPE(deque)), aslist,
  ((dequeobject *)deque)->maxlen);
else
result = PyUnicode_FromFormat("%s(%R)",
  _PyType_Name(Py_TYPE(deque)), aslist);
...
return result;
}

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36021] [Security][Windows] webbrowser: WindowsDefault uses os.startfile() and so can be abused to run arbitrary commands

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

Windows has the GetBinaryTypeW function, this one is used by pywin32, maybe I 
could develop a wrapper in os, like os.is_executable(path)

for Unix-like, os.is_executable(path) could use os.access(path, os.X_OK)
for Windows, the function would use GetBinaryTypeW.

my 2 cents.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36020] HAVE_SNPRINTF and MSVC std::snprintf support

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

Hi @steve

I'm not used to this, can you guide me?
Thanks

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Zahash Z]

Change by Zahash Z :


--
components: +Library (Lib)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36050] Why does http.client.HTTPResponse._safe_read use MAXAMOUNT

[Bruce Merry]

New submission from Bruce Merry :

While investigating poor HTTP read performance I discovered that reading all 
the data from a response with a content-length goes via _safe_read, which in 
turn reads in chunks of at most MAXAMOUNT (1MB) before stitching them together 
with b"".join. This can really hurt performance for responses larger than 
MAXAMOUNT, because
(a) the data has to be copied an additional time; and
(b) the join operation doesn't drop the GIL, so this limits multi-threaded 
scaling.

I'm struggling to see any advantage in doing this chunking - it's not saving 
memory either (in fact it is wasting it).

To give an idea of the performance impact, changing MAXAMOUNT to a very large 
value made a multithreaded test of mine go from 800MB/s to 2.5GB/s (which is 
limited by the network speed).

--
components: Library (Lib)
messages: 336081
nosy: bmerry
priority: normal
severity: normal
status: open
title: Why does http.client.HTTPResponse._safe_read use MAXAMOUNT
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36051] (Performance) Drop the GIL during large bytes.join operations?

[Bruce Merry]

New submission from Bruce Merry :

A common pattern in libraries doing I/O is to receive data in chunks, put them 
in a list, then join them all together using b"".join(chunks). For example, see 
http.client.HTTPResponse._safe_read. When the output is large, the memory 
copies can block the interpreter for a non-trivial amount of time, and prevent 
multi-threaded scaling. If the GIL could be dropped during the memcpys it could 
improve parallel I/O performance in some high-bandwidth scenarios (36050 
mentions a case where I've run into this serialisation bottleneck in practice).

Obviously it could hurt performance to drop the GIL for small cases. As far as 
I know numpy uses thresholds to decide when it's worth dropping the GIL and it 
seems to work fairly well.

--
components: Interpreter Core
messages: 336082
nosy: bmerry
priority: normal
severity: normal
status: open
title: (Performance) Drop the GIL during large bytes.join operations?
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35993] incorrect use of released memory in Python/pystate.c line 284

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

@eric Could you help me for the tests of my PR?

Thank you

--
nosy: +eric.snow

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35995] use SSL encrypted socket on logging.handlers.SMTPHandler

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

@lidayan

I think it's a misuse of the logging.handlers.SMTPHandler class.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Rémi Lapeyre]

Rémi Lapeyre  added the comment:

Note: when repr do not round trip, I think it's common practice to use brackets:



instead of:
LifoQueue('a'...'c')


Example with Regex from https://docs.python.org/3/howto/regex.html:

>>> re.match(r'From\s+', 'From amk Thu May 14 19:12:10 1998')  


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

Fixed

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36051] (Performance) Drop the GIL during large bytes.join operations?

[Rémi Lapeyre]

Change by Rémi Lapeyre :


--
nosy: +remi.lapeyre

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
assignee:  -> rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36051] Drop the GIL during large bytes.join operations?

[SilentGhost]

Change by SilentGhost :


--
title: (Performance) Drop the GIL during large bytes.join operations? -> Drop 
the GIL during large bytes.join operations?
type:  -> performance
versions: +Python 3.8 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35993] incorrect use of released memory in Python/pystate.c line 284

[STINNER Victor]

STINNER Victor  added the comment:


New changeset b5409dacc4885146a27d06482b346e55fa12d2ec by Victor Stinner 
(Stéphane Wirtel) in branch 'master':
bpo-35993: Fix _PyInterpreterState_DeleteExceptMain() (GH-11852)
https://github.com/python/cpython/commit/b5409dacc4885146a27d06482b346e55fa12d2ec


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Rémi Lapeyre]

Rémi Lapeyre  added the comment:

See also https://bugs.python.org/issue35889 which add repr to sqlite3.Row

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35993] incorrect use of released memory in Python/pystate.c line 284

[STINNER Victor]

STINNER Victor  added the comment:

Thanks wangjiangqiang for the bug report and thanks Stéphane Wirtel for the fix!

--
components: +Interpreter Core
resolution:  -> fixed
stage:  -> resolved
status: open -> closed
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34464] There are inconsitencies in the treatment of True, False, None, and __debug__ keywords in the docs

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

Hi @David,

Maybe we could remove 3.6 from the list because 3.6 is in security mode and not 
bugfix.

Is there a security issue with this bug?

--
nosy: +matrixise

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36021] [Security][Windows] webbrowser: WindowsDefault uses os.startfile() and so can be abused to run arbitrary commands

[STINNER Victor]

STINNER Victor  added the comment:

> Windows has the GetBinaryTypeW function, this one is used by pywin32, maybe I 
> could develop a wrapper in os, like os.is_executable(path)

I don't think that it would detect .BAT or .VBS scripts as executable, whereas 
we don't want to execut such scripts with webbrowser.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36021] [Security][Windows] webbrowser: WindowsDefault uses os.startfile() and so can be abused to run arbitrary commands

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

Sure we don't want to execute these kinds of scripts but we could mix with 
GetBinaryTypeW and add a check on the extensions.

for example and simplified protocode

if is_executable(path) or is_script(path):
   raise ...
os.startfile(path)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36041] email: folding of quoted string in display_name violates RFC

[R. David Murray]

R. David Murray  added the comment:

I'm afraid I don't have time to parse through the file you uploaded.  Can you 
produce a pull request or a diff showing your fix?  And ideally some added 
tests :)  But whatever you can do is great, if you don't have time maybe 
someone else will pick it up (I unfortunately don't have time, though I should 
be able to do a review of a PR).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36021] [Security][Windows] webbrowser: WindowsDefault uses os.startfile() and so can be abused to run arbitrary commands

[STINNER Victor]

STINNER Victor  added the comment:

> Sure we don't want to execute these kinds of scripts but we could mix with 
> GetBinaryTypeW and add a check on the extensions.

Windows has a convenient feature: if you ask to run "program", Windows tries to 
run "program.exe", "program.bat", etc.

Example:
---
C:\vstinner>del hello.txt

C:\vstinner>type hello.bat
echo "Hello from hello.bat" > /vstinner/hello.txt

C:\vstinner>\vstinner\python\master\python
Python 3.8.0a0 (heads/master:8f59ee01be, Jan 25 2019, 01:16:59) [MSC v.1915 64 
bit (AMD64)] on win32
>>> import os
>>> os.startfile(r"c:\vstinner\hello")
>>> with open(r"c:\vstinner\hello.txt") as fp: print(fp.read())
...
"Hello from hello.bat"
---

os.startfile(r"c:\vstinner\hello") <= "hello" filename has no extension

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36035] pathlib.Path().rglob() breaks with broken symlinks

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

3.5 is in security mode, we can remove 3.5 from the list for this issue.

--
versions:  -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36021] [Security][Windows] webbrowser: WindowsDefault uses os.startfile() and so can be abused to run arbitrary commands

[STINNER Victor]

STINNER Victor  added the comment:

Maybe webbrowser must be changed to become *very strict*. For example, raise an 
error if the URL doesn't start with "http://"; or "https://";. But add an option 
to opt-in for "unsafe" URLs with a warning in the doc to explain the risk on 
Windows?

Another option is to add an optional callback to validate the URL. As the 
'verify' parameter of logging.config.listen():
https://docs.python.org/dev/library/logging.config.html#logging.config.listen

"pydoc -b" runs a local HTTP server but it uses regular "http://"; URLs, it 
doesn't use file://.

Maybe only Windows should be modified, Unix is safe, no?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36021] [Security][Windows] webbrowser: WindowsDefault uses os.startfile() and so can be abused to run arbitrary commands

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

+1

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36021] [Security][Windows] webbrowser: WindowsDefault uses os.startfile() and so can be abused to run arbitrary commands

[Eryk Sun]

Eryk Sun  added the comment:

> Windows has the GetBinaryTypeW function

ShellExecute[Ex] doesn't check for a PE image. It uses the file extension, and 
a tangled web of registry settings to determine what to execute. If a file 
should run directly via CreateProcess, you'll find the template command starts 
with the target file ("%1" or "%L"). For example:

>>> AssocQueryStringW(ASSOCF_INIT_IGNOREUNKNOWN, ASSOCSTR_COMMAND,
... '.exe', 'open', command, n)
0
>>> print(command.value)
"%1" %*

OTOH, a script requires an interpreter, e.g for .py files:

>>> AssocQueryStringW(ASSOCF_INIT_IGNOREUNKNOWN, ASSOCSTR_COMMAND,
... '.py', 'open', command, n)
0
>>> print(command.value)
"C:\WINDOWS\py.exe" "%L" %*

Except .bat and .cmd scripts are executed directly via the %ComSpec% 
interpreter:

>>> AssocQueryStringW(ASSOCF_INIT_IGNOREUNKNOWN, ASSOCSTR_COMMAND,
... '.bat', 'open', command, n)
0
>>> print(command.value)
"%1" %*

One bit of metadata we can check is the file's "PerceivedType": unknown=0, 
text, image, audio, video, compressed, document, system, application, 
game-media, contacts. If a file's type is unknown (0), system (7), or 
application (8), or if getting the type fails, we probably don't want to run 
it. For example:

>>> _ = AssocGetPerceivedType(".txt", ptype, flags, None); print(ptype[0])
1
>>> _ = AssocGetPerceivedType(".jpg", ptype, flags, None); print(ptype[0])
2
>>> _ = AssocGetPerceivedType(".mp3", ptype, flags, None); print(ptype[0])
3
>>> _ = AssocGetPerceivedType(".mp4", ptype, flags, None); print(ptype[0])
4
>>> _ = AssocGetPerceivedType(".zip", ptype, flags, None); print(ptype[0])
5
>>> _ = AssocGetPerceivedType(".html", ptype, flags, None); print(ptype[0])
6
>>> _ = AssocGetPerceivedType(".sys", ptype, flags, None); print(ptype[0])
7
>>> _ = AssocGetPerceivedType(".exe", ptype, flags, None); print(ptype[0])
8
>>> _ = AssocGetPerceivedType(".com", ptype, flags, None); print(ptype[0])
8
>>> _ = AssocGetPerceivedType(".bat", ptype, flags, None); print(ptype[0])
8
>>> _ = AssocGetPerceivedType(".cmd", ptype, flags, None); print(ptype[0])
8

Except for a small number of hard-code definitions, the PerceivedType has to be 
defined for a filetype, and it's optional. It gets set either in the 
file-extension key under [HKCU|HKLM]\Software\Classes, or in the 
SystemFileAssocations subkey, or probably in 10 other locations sprawled across 
the registry. Python's installer doesn't set the PerceivedType of .py files to 
the application type (8), but it should. 

Another bit of metadata is the MIME "Content Type". This is also optional 
information provided in a filetype definition. For example:

>>> AssocQueryStringW(ASSOCF_INIT_IGNOREUNKNOWN, ASSOCSTR_CONTENTTYPE,
... '.exe', 'open', mtype, n)
0
>>> print(mtype.value)
application/x-msdownload

>>> AssocQueryStringW(ASSOCF_INIT_IGNOREUNKNOWN, ASSOCSTR_CONTENTTYPE,
... '.html', 'open', mtype, n)
0
>>> print(mtype.value)
text/html

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34464] There are inconsitencies in the treatment of True, False, None, and __debug__ keywords in the docs

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

__debug__ is not a keyword. And the error message has been changed in 3.8.

But it is a special enough. You can not use this name in any assignment context:

>>> __debug__ = 1
  File "", line 1
SyntaxError: cannot assign to __debug__
>>> for __debug__ in []: pass
... 
  File "", line 1
SyntaxError: cannot assign to __debug__
>>> with cm() as __debug__: pass
... 
  File "", line 1
SyntaxError: cannot assign to __debug__
>>> class __debug__: pass
... 
  File "", line 1
SyntaxError: cannot assign to __debug__
>>> def __debug__(): pass
... 
  File "", line 1
SyntaxError: cannot assign to __debug__
>>> def foo(__debug__): pass
... 
  File "", line 1
SyntaxError: cannot assign to __debug__
>>> import __debug__
  File "", line 1
SyntaxError: cannot assign to __debug__

You can not even assign to an attribute named __debug__!

>>> x.__debug__ = 1
  File "", line 1
SyntaxError: cannot assign to __debug__

The assignment operator is the only exception here, and this looks like a bug.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36052] Assignment operator allows to assign to __debug__

[Serhiy Storchaka]

New submission from Serhiy Storchaka :

All ways of assigning to __debug__ are forbidden:

>>> __debug__ = 1
  File "", line 1
SyntaxError: cannot assign to __debug__
>>> for __debug__ in []: pass
... 
  File "", line 1
SyntaxError: cannot assign to __debug__
>>> with cm() as __debug__: pass
... 
  File "", line 1
SyntaxError: cannot assign to __debug__
>>> class __debug__: pass
... 
  File "", line 1
SyntaxError: cannot assign to __debug__
>>> def __debug__(): pass
... 
  File "", line 1
SyntaxError: cannot assign to __debug__
>>> def foo(__debug__): pass
... 
  File "", line 1
SyntaxError: cannot assign to __debug__
>>> import __debug__
  File "", line 1
SyntaxError: cannot assign to __debug__

The only exception is the assignment operator.

>>> (__debug__ := 'spam')
'spam'
>>> globals()['__debug__']
'spam'

This looks like a bug.

--
components: Interpreter Core
messages: 336100
nosy: emilyemorehouse, gvanrossum, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Assignment operator allows to assign to __debug__
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35933] python doc does not say that the state kwarg in Pickler.save_reduce can be a tuple (and not only a dict)

[Pierre Glaser]

Change by Pierre Glaser :


--
keywords: +patch
pull_requests: +11981
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36021] [Security][Windows] webbrowser: WindowsDefault uses os.startfile() and so can be abused to run arbitrary commands

[Steve Dower]

Steve Dower  added the comment:

> Maybe webbrowser must be changed to become *very strict*.

This is the only acceptable suggestion I've seen (since my suggestion ;) )

I'd propose making it very strict by *requiring* a browser to be detected. So 
remove the os.startfile default and always require Chrome/Edge/etc. to be 
found. If they're not, you get an exception.

Personally, I'd hate this behaviour :) But for my cases I'd just switch to 
os.startfile unconditionally (as I only use this in my own scripts and not 
libraries).

One other thing to factor in is that if you use os.startfile to launch a 
malicious program, it will first be scanned by any anti-malware or antivirus 
software, and likely also by Windows SmartScreen. So you're not exactly getting 
arbitrary execution. It also only runs in the context of the current user, so 
there's not necessarily any escalation here.

All in all, I'd label this a vulnerability in applications that use 
webbrowser.open(), rather than in webbrowser.open() itself. The function is 
doing exactly what it is told, and if someone is passing untrusted input, then 
they'll get the exact untrusted output they expect.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35933] python doc does not say that the state kwarg in Pickler.save_reduce can be a tuple (and not only a dict)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

A slotted class will have a dict also when it inherits it from a non-slotted 
class. This is why the base class of slotted class should have slots if you do 
not want an instance dict.

__getstate__ and __setstate__ for slotted classes are described in PEP 307. 
Unfortunately this was not copied to the module documentation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36052] Assignment operator allows to assign to __debug__

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

You need to add for the target of the assignment operator the same check as for 
other targets.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36052] Assignment operator allows to assign to __debug__

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

An idea for the fix? a direction where I could find the solution? ;-)

--
nosy: +matrixise

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35933] python doc does not say that the state kwarg in Pickler.save_reduce can be a tuple (and not only a dict)

[Pierre Glaser]

Pierre Glaser  added the comment:

I added a PR with a small patch to document this behavior and reconcile 
_pickle.c and pickle.py

Some explanations on why I am pushing this forward:
 
Pickling instances of classes/subclasses with slots is done natively for pickle 
protocol >= 2. Mentioning this behavior in the docs should *not* make the user 
worry about implementing custom __getstate__ methods just to preserve slots.

Here is the reason why I think this functionality (allowing state and 
slotstate) is worth documenting: pickle gives us a lot of flexibility for 
reducing thanks to the dispatch_table. But unpickling is rather rigid: if no 
__setstate__ exists, we have to deal with the default state updating procedure 
present in load_build. Might as well document all of it ;)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35933] python doc does not say that the state kwarg in Pickler.save_reduce can be a tuple (and not only a dict)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

See also issue26579 which propose to add a function or method for standard 
implementation of __getstate__ and use it consistently in custom __getstate__ 
implementations. I am not sure about API yet.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36048] Deprecate implicit truncating when convert Python numbers to C integers: use __index__, not __int__

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I am not sure what to with the int constructor. Should it try __index__ before 
__int__? Or just make __index__ without __int__ setting the nb_int slot as was 
proposed by Nick in issue33039?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36000] __debug__ is a keyword but not a keyword

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36041] email: folding of quoted string in display_name violates RFC

[Aaryn Tonita]

Aaryn Tonita  added the comment:

Sure thing, I'll try to produce something tomorrow.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36052] Assignment operator allows to assign to __debug__

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

ok, I will check for the assignment operator in the code.

maybe in the AST (Python-ast.c) but I think not

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36052] Assignment operator allows to assign to __debug__

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

ok, I found the commit of @emily 8f59ee01be3d83d5513a9a3f654a237d77d80d9a

and will try to find the assignment operator :=

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36053] pkgutil.walk_packages jumps out from given path if there is package with the same name in sys.pah

[Piotr Karkut]

New submission from Piotr Karkut :

When walk_packages encounter a package with a name that is available in 
sys.path, it will abandon the current package, and start walking the package 
from the sys.path.

Consider this file layout:

```
PYTHONPATH/
├──package1/
|   ├──core   
|   |   ├──some_package/
|   |   |   ├──__init__.py
|   |   |   └──mod.py
|   |   └──__init__.py
|   └──__init__.py
└──some_package/
   |   ├──__init__.py
   |   └──another_mod.py
   └──__init__.py
```

The result of walking package1 will be:

```
>> pkgutil.walk_packages('PYTHONPATH/package1')

ModuleInfo(module_finder=FileFinder('PYTHONPATH/package1/core'), 
name='some_package', ispkg=True)
ModuleInfo(module_finder=FileFinder('PYTHONPATH/some_package), 
name='another_mod', ispkg=False)
```

I'm not sure if it is a security issue, but it definitely should not jump off 
the given path.

--
components: Library (Lib)
messages: 336111
nosy: karkucik
priority: normal
severity: normal
status: open
title: pkgutil.walk_packages jumps out from given path if there is package with 
the same name in sys.pah
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36052] Assignment operator allows to assign to __debug__

[Emily Morehouse]

Emily Morehouse  added the comment:

You should look in Python/ast.c. The naming convention follows some form of 
"named expression" (e.g. NamedExpr, ast_for_namedexpr).

I'll have more time to look later this week, but let me know if you have any 
questions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36053] pkgutil.walk_packages jumps out from given path if there is package with the same name in sys.path

[Piotr Karkut]

Change by Piotr Karkut :


--
title: pkgutil.walk_packages jumps out from given path if there is package with 
the same name in sys.pah -> pkgutil.walk_packages jumps out from given path if 
there is package with the same name in sys.path

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36052] Assignment operator allows to assign to __debug__

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

Hi @emily

Thank you for your help.

In fact, I have started to read Python-ast.c and ast.c, they are the main files 
for this kind of operations.

Thank you again,

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36053] pkgutil.walk_packages jumps out from given path if there is package with the same name in sys.path

[Piotr Karkut]

Change by Piotr Karkut :


--
keywords: +patch
pull_requests: +11982
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I guess you hurried to write the code. It has not yet been decided whether to 
expose the content of the queue in its repr at all. There is no public API for 
accessing the content of the queue without removing items from the queue, and I 
think it is intentional.

What is your use case? Why you need to change the repr of queues?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36052] Assignment operator allows to assign to __debug__

[Stéphane Wirtel]

Change by Stéphane Wirtel :


--
assignee:  -> matrixise

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36052] Assignment operator allows to assign to __debug__

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Look at compile.c. The code for restricting other assignments is located here.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Zahash Z]

Zahash Z  added the comment:

If you look at the queue.PriorityQueue class, there is self.queue = [ ].

That's because priority queue uses list data structure to implement the min
heap data structure (on which priority queue is based on).

So the list can be represented.
There is no need to remove anything.

On Wed, 20 Feb 2019, 10:17 pm Serhiy Storchaka, 
wrote:

>
> Serhiy Storchaka  added the comment:
>
> I guess you hurried to write the code. It has not yet been decided whether
> to expose the content of the queue in its repr at all. There is no public
> API for accessing the content of the queue without removing items from the
> queue, and I think it is intentional.
>
> What is your use case? Why you need to change the repr of queues?
>
> --
> nosy: +serhiy.storchaka
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36052] Assignment operator allows to assign to __debug__

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +11984
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36054] Way to detect CPU count inside docker container

[Keir Lawson]

New submission from Keir Lawson :

There appears to be no way to detect the number of CPUs allotted to a Python 
program within a docker container.  With the following script:

import os

print("os.cpu_count(): " + str(os.cpu_count()))
print("len(os.sched_getaffinity(0)): " + str(len(os.sched_getaffinity(0

when run in a container (from an Ubuntu 18.04 host) I get:

docker run -v "$PWD":/src/ -w /src/ --cpus=1 python:3.7 python detect_cpus.py
os.cpu_count(): 4
len(os.sched_getaffinity(0)): 4

Recent vesions of Java are able to correctly detect the CPU allocation:

docker run -it --cpus 1 openjdk:10-jdk
Feb 20, 2019 4:20:29 PM java.util.prefs.FileSystemPreferences$1 run
INFO: Created user preferences directory.
|  Welcome to JShell -- Version 10.0.2
|  For an introduction type: /help intro

jshell> Runtime.getRuntime().availableProcessors()
$1 ==> 1

--
components: Library (Lib)
messages: 336117
nosy: keirlawson
priority: normal
severity: normal
status: open
title: Way to detect CPU count inside docker container
type: performance
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Raymond Hettinger]

Raymond Hettinger  added the comment:

[Zahash Z] (Zahash Z)]
> If you look at the queue.PriorityQueue class, there is 
> self.queue = [ ].

We really don't want to expose the internals here.  They are subject to change 
and access to them is guarded by locks.

> It has not yet been decided whether to expose the content
> of the queue in its repr at all. There is no public API for 
> accessing the content of the queue without removing items 
> from the queue, and I think it is intentional.

I agree with Serhiy.

Let's close this feature request.  It doesn't make sense for queues and is at 
odds with their design and intended uses.

--
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35956] Sort documentation could be improved for complex sorting

[Raymond Hettinger]

Change by Raymond Hettinger :


--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36052] Assignment operator allows to assign to __debug__

[Stéphane Wirtel]

Change by Stéphane Wirtel :


--
pull_requests: +11985

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36052] Assignment operator allows to assign to __debug__

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

When I started working on this I did not notice the extra comments here.
 
I will close my PR so Stéphane can do the PR.

--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36039] Replace append loops with list comprehensions

[Raymond Hettinger]

Raymond Hettinger  added the comment:

I concur with all the other -1 comments and will mark this a closed.

FWIW, we generally discourage sweeping across the library with minor rewrites.  
Guido articulated a principle of "holistic refactoring" where we make code 
improvements while working on the module as a whole.  This helps makes sure 
that code changes are made in the context of a thorough understanding of what 
the module is trying to do.  This also helps us reduce maintenance-induced-bugs 
where minor code adjustments create new bugs that weren't there before.

Thank you for taking a look at the source and please continue to do so. If you 
find a specific case that is problematic, feel free to post that one particular 
case.  That said, a better use of time is to take one of the many open tracker 
issues, research it, and propose a fix.

--
nosy: +rhettinger
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36042] Setting __init_subclass__ and __class_getitem__ methods are in runtime doesnt make them class method.

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I don't like complicating the code and adding a performance penalty to support 
such uncommon case.

The documentation for __class_getitem__ precisely describes the current 
behavior: "when defined in the class body, this method is implicitly a class 
method". Nothing should be changed here.

The documentation for __init_subclass__ could be rewritten in similar words.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

Hi @rhettinger

Thank you for the review of this issue.

--
resolution: rejected -> 
stage: resolved -> patch review
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36042] Setting __init_subclass__ and __class_getitem__ methods are in runtime doesnt make them class method.

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:

I totally agree with Serhiy

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36052] Assignment operator allows to assign to __debug__

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

Thank you Pablo, if I see you at PyCon, maybe we could drink a good beer ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36049] No __repr__() for queue.PriorityQueue and queue.LifoQueue

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

If you treat the "queue" attribute as a part the public API, just use 
repr(q.queue) instead of repr(q).

--
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36054] Way to detect CPU count inside docker container

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

I would like to work on this issue.

--
nosy: +matrixise

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36048] Deprecate implicit truncating when convert Python numbers to C integers: use __index__, not __int__

[Rémi Lapeyre]

Rémi Lapeyre  added the comment:

> I am not sure what to with the int constructor. Should it try __index__ 
> before __int__? Or just make __index__ without __int__ setting the nb_int 
> slot as was proposed by Nick in issue33039?

Shouldn't it try only __int__ since this will default to __index__ once #33039 
is closed?

--
nosy: +remi.lapeyre

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com