[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Roundup Robot]

Roundup Robot added the comment:

New changeset d755f75019c8 by Victor Stinner in branch 'default':
Issue #23517: Skip a datetime test on Windows
https://hg.python.org/cpython/rev/d755f75019c8

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

On 05.09.2015 03:49, Alexander Belopolsky wrote:
> 
> Alexander Belopolsky added the comment:
> 
> Hmm, on Mac OSX "%" and "A%" are valid format strings:
> 
 time.strftime("%")
> '%'
 time.strftime("A%")
> 'A%'
> 
> Mark's experiments show that on Windows they are not. What about the other 
> platforms affected by the patch?  I am concerned about the bottom part of the 
> patch.

Trailing '%' are valid on Linux, just as using unsupported format
codes (those as passed through as is).

On Windows, the C lib strftime() segfaults with a trailing '%', just
as it does with unsupported format codes.

I have this code in mxDateTime to check for this:

static
int _mxDateTime_CheckWindowsStrftime(char *fmt,
 struct tm *tm)
{
char *p;

/* Range checks */
Py_Assert(tm->tm_sec < 60,
  PyExc_ValueError,
  ".strftime() cannot format leap seconds on Windows");
Py_Assert(tm->tm_mday <= 31,
  PyExc_ValueError,
  ".strftime() cannot format days > 31 on Windows");

/* Scan format string for invalid codes */
for (p = fmt; *p != '\0'; p++) {
register char code;
if (*p != '%')
continue;
code = *++p;
/* Check for supported format codes; see
   https://msdn.microsoft.com/en-us/library/fe06s4ak.aspx */
switch (code) {
case 'a':
case 'A':
case 'b':
case 'B':
case 'c':
case 'd':
case 'H':
case 'I':
case 'i':
case 'm':
case 'M':
case 'p':
case 'S':
case 'U':
case 'w':
case 'W':
case 'x':
case 'X':
case 'y':
case 'Y':
case 'z':
case 'Z':
case '%':
continue;
case '\0':
Py_Error(PyExc_ValueError,
 "format string may not end with a '%' character "
 "on Windows");
default:
Py_ErrorWithArg(PyExc_ValueError,
"format code '%c' not supported on Windows",
code);
}
}
return 0;

  onError:
return -1;
}

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16180] cannot quit pdb when there is a syntax error in the debuggee (must kill it)

[Xavier de Gaye]

Xavier de Gaye added the comment:

Slightly better in that pdb exits in case of a syntax error instead of 
proposing to restart the program which does not make sense.

A test case is included.

--
Added file: http://bugs.python.org/file40372/pdb_syntax_error.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22995] Restrict default pickleability

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

See also issue14350 and issue24900. Both will be solved by disabling pickling 
when tp_new is NULL (a part of proposed patch). So that I think the patch 
should be applied to all releases.

--
type: enhancement -> behavior
versions: +Python 2.7, Python 3.4, Python 3.6
Added file: http://bugs.python.org/file40373/pickle_restrictions_3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25007] Add support of copy protocol to zlib compressors and decompressors

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

zlib compressor and decompressor objects can be copied with the copy() method, 
but not with copy.copy(). It is easy to add support of copy protocol to them 
(just add __copy__() as an alias to copy()).

--
components: Library (Lib)
messages: 249902
nosy: nadeem.vawda, serhiy.storchaka, twouters
priority: normal
severity: normal
status: open
title: Add support of copy protocol to zlib compressors and decompressors
type: enhancement
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Nick Coghlan]

Nick Coghlan added the comment:

3.5.0 PR for the change is at 
https://bitbucket.org/larry/cpython350/pull-requests/16

This *is* a regression (since it's an unintended behavioural change in 
imp.load_dynamic), but it's one with a relatively straightforward workaround, 
so it would also be reasonable to defer fixing it to 3.5.1.

On the other hand, the risk is low, since the only functional change is in 
imp.load_dynamic itself, rather than in the core import machinery (that calls 
the underlying loader methods directly rather than going through the imp module 
API).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[eryksun]

eryksun added the comment:

> On Windows, the C lib strftime() segfaults with a trailing '%', 
> just as it does with unsupported format codes.

No, it doesn't segfault; it invokes the invalid parameter handler (IPH). This 
could always be configured for the whole process, but with VC 14 it can also be 
configured per thread. I think it was Steve Dower who updated time_strftime to 
take advantage of this feature in 3.5, using the new macros 
_Py_BEGIN_SUPPRESS_IPH and _Py_END_SUPPRESS_IPH. This allowed removing the 
following check that used to be in the code prior to 3.5:

if (outbuf[1]=='\0' ||
!strchr("aAbBcdHIjmMpSUwWxXyYzZ%", outbuf[1]))
{
PyErr_SetString(PyExc_ValueError, "Invalid format string");
Py_DECREF(format);
return NULL;
}

With this change the time module no longer has to make assumptions about what 
the CRT considers to be a valid format string in order to avoid invoking the 
IPH. However, this also accidentally removed checking whether outbuf[1]=='\0'. 
Now, instead of calling the default IPH that terminates the process, Python's 
_silent_invalid_parameter_handler gets called, which of course does nothing by 
design:

>>> import time
>>> time.strftime('A%')
Breakpoint 0 hit
python35!_silent_invalid_parameter_handler:
`5eadae50 c2  ret 0
0:000> k8
Child-SP  RetAddr   Call Site
`002af018 07fe`e9d8d2ab 
python35!_silent_invalid_parameter_handler
`002af020 07fe`e9d8d2c9 ucrtbase!invalid_parameter+0x103
`002af060 07fe`e9dafedc ucrtbase!invalid_parameter_noinfo+0x9
`002af0a0 07fe`e9dac359 ucrtbase!Wcsftime_l+0x168
`002af140 07fe`e9dac3f5 ucrtbase!Strftime_l+0x155
`002af1d0 `5e9fc785 ucrtbase!strftime+0x15
`002af210 `5ea7d5c2 python35!time_strftime+0x1f5
`002af2b0 `5eaf8fd0 python35!PyCFunction_Call+0x122

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[Larry Hastings]

Larry Hastings added the comment:

Rather than debating about how various platforms handle malformed format 
strings for strftime(), and whether or not they crash, we should simply prevent 
the native strftime() function from seeing them in the first place.

I'd like the "v3" patch in 3.5.0rc3.  Can someone create a pull request?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24964] Add tunnel CONNECT response headers to httplib / http.client

[Thomas Belhalfaoui]

Thomas Belhalfaoui added the comment:

Terry: Thanks for the form, I just filled it.

Martin: Thanks for sending your patch. I will dive into it, and try to figure 
out how to add support for passing in the tunnel details to the HTTPSConnection 
constructor.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24305] The new import system makes it inconvenient to correctly issue a deprecation warning for a module

[Larry Hastings]

Larry Hastings added the comment:

Okay.  Right now creating server-side clones is broken.  So I have repurposed 
one of my existing (old, dead) server-side clones for testing this.  It's 
called "ssh://h...@hg.python.org/larry/path_error2".

I just fired off this change on all the "custom" buildbots.  I'm going to bed.  
If it passed everything in the morning I'll just check it in to 3.5.0, and hail 
mary.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[eryksun]

eryksun added the comment:

> Rather than debating about how various platforms handle malformed 
> format strings for strftime(), and whether or not they crash, we 
> should simply prevent the native strftime() function from seeing 
> them in the first place.

Of course the check needs to be restored. I wasn't suggesting to go back to the 
default IPH to have it terminate the process. That won't help, anyway. The 
potential over-read is in the for loop, before calling strftime. I just thought 
the context for the accidental removal was relevant. Care needs to be taken 
when removing checks that were put in place to avoid the default IPH, because 
those checks may serve a dual purpose in the surrounding code.

The patch also fixes the AIX/Sun code, which needs to be applied to 3.4 as 
well. I don't think this issue applies to 3.2 and 3.3.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

> It doesn't matter who's consuming the rounding of a binary
> float to decimal microseconds

That is true, but what does matter is who is producing the incoming floats.   
Consider an extreme case of a timer that ticks twice a microsecond and gives 
you results in a timespec struct style sec, nsec pair.  If you convert those to 
timedeltas with timedelta(0, sec, nsec/1000), you will see half of nsec/1000 
floats ending in .5 and half ending with .0.  If you have a large sample of 
(sec, nsec) measurements, the mean of corresponding timedeltas will be 0.25µs 
larger with the half-up rounding than the actual mean of your samples.

Is round-half-to-even a panacea?  Of course not.  In an extreme case if all 
your measurements are 1.5µs - it will not help at all, but in a more realistic 
sample you are likely to have an approximately equal number of even and odd µs 
and the Dutch rounding will affect the mean much less than round-half-up.

As you said, for most uses none of this matters, but we are not discussing a 
change to timedelta(0, s) rounding here.  All I want from this issue is to keep 
the promise that we make in the docs:

On the POSIX compliant platforms, utcfromtimestamp(timestamp) is equivalent to 
the following expression:

datetime(1970, 1, 1) + timedelta(seconds=timestamp)

https://docs.python.org/3/library/datetime.html#datetime.datetime.utcfromtimestamp

Tim, while we have this entertaining theoretical dispute, I am afraid we are 
leaving Victor confused about what he has to do in his patch.  I think we agree 
that fromtimestamp() should use Dutch rounding.  We only disagree why.

If this is the case, please close this thread with a clear advise to use 
round-half-to-even and we can continue discussing the rationale privately or in 
a different forum.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Tim Peters]

Tim Peters added the comment:

Victor, sorry if I muddied the waters here:  Alex & I do agree nearest/even 
must be used.  It's documented for timedelta already, and the 
seconds-from-the-epoch invariant Alex showed is at least as important to 
preserve as round-tripping.

Alex, agreed about the overall mean in the example, but expect that continuing 
to exchange contrived examples isn't really a major life goal for either of us 
;-)  Thanks for playing along.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25008] Deprecate smtpd

[Brett Cannon]

New submission from Brett Cannon:

As mentioned in passing in issue #25002, perhaps smtpd should be deprecated or 
at least be updated to use asyncio.

--
components: Library (Lib)
messages: 249911
nosy: brett.cannon
priority: normal
severity: normal
stage: needs patch
status: open
title: Deprecate smtpd
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25002] Deprecate asyncore/asynchat

[Brett Cannon]

Brett Cannon added the comment:

I opened http://bugs.python.org/issue25008 to discuss deprecating smtpd since 
this issue is about asyncore/asynchat.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25008] Deprecate smtpd

[Brett Cannon]

Brett Cannon added the comment:

I should mention one of the motivations behind deprecating smtpd is that no one 
should be using that module to run an SMTP server and it doesn't provide 
utility like http.server does and so we probably should drop the code to 
prevent anyone accidentally using it and not have to maintain it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25009] queue.Queue() does not validate the maxsize argument

[Alex Willmer]

New submission from Alex Willmer:

The maxsize argument when initializing a Queue is expected to be an int 
(technically anything that can be compared to an int). However the class takes 
any value. In Python 3 this throws "TypeError: unorderable types" once e.g. 
.put() is called.

On the basis that errors should not pass silent, should maxsize be checked for 
compatibility at initialization time? e.g.

Desired:

>>> import queue
>>> q = queue.Queue(range(10))
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python3.5/queue.py", line nnn, in __init__()
...
TypeError: 'range' object cannot be interpreted as an integer


Actual:

Python 3.5.0rc2 (default, Aug 25 2015, 20:29:07) 
[GCC 5.2.1 20150825] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import queue
>>> q = queue.Queue(range(10)) # Silently accepts an invalid maxsize
>>> q.put('foo')
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python3.5/queue.py", line 127, in put
if self.maxsize > 0:
TypeError: unorderable types: range() > int()

--
components: Library (Lib)
messages: 249914
nosy: Alex.Willmer
priority: normal
severity: normal
status: open
title: queue.Queue() does not validate the maxsize argument
type: behavior
versions: Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25010] minor typo in PCBuild readme.txt

[Shaun Walbridge]

New submission from Shaun Walbridge:

A minor documentation fix. Currently, the top-level PCBuild readme.txt mentions 
in the final section 'Your Own Extension DLLs' that the example project is 
located in ../PC/example/, but the example location is ../PC/example_nt/.

--
assignee: docs@python
components: Documentation
files: pyd-typo.diff
keywords: patch
messages: 249915
nosy: docs@python, scw
priority: normal
severity: normal
status: open
title: minor typo in PCBuild readme.txt
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file40374/pyd-typo.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25005] webbrowser breaks on query strings with multiple fields on Windows

[Steve Dower]

Steve Dower added the comment:

Here's an alternative to backing out the change, and it's simpler than I 
expected when I said it would be too much for 3.5.0.

We add an 'arguments' parameter to os.startfile and use that instead of 
subprocess.call(shell=True). The underlying ShellExecute call does not do any 
argument processing, so you can pass through any arguments you like.

In the attached patch, I only added the argument for when Unicode strings are 
used, since byte strings are deprecated, but it's fairly trivial to add it to 
both. I'll add a backout patch next so they can be compared.

--
keywords: +patch
Added file: http://bugs.python.org/file40375/25005_1.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25005] webbrowser breaks on query strings with multiple fields on Windows

[Steve Dower]

Steve Dower added the comment:

Patch for backing out #8232's changes.

--
Added file: http://bugs.python.org/file40376/25005_2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25005] webbrowser breaks on query strings with multiple fields on Windows

[Steve Dower]

Steve Dower added the comment:

Patch 1 also requires a minor update to Doc\library\os.rst:

-.. function:: startfile(path[, operation])
+.. function:: startfile(path[, operation[, arguments]])
...
+*arguments* is passed to the underlying :c:func:`ShellExecute`
+call. Its format is determined by the target file and operation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[Steve Dower]

Steve Dower added the comment:

I'll do the PR.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[Steve Dower]

Steve Dower added the comment:

> Rather than debating about how various platforms handle malformed 
> format strings for strftime(), and whether or not they crash, we 
> should simply prevent the native strftime() function from seeing 
> them in the first place.

The crash is nothing to do with the native strftime() function - Python was 
crashing because it *tried* to prevent malformed strings and we (I) got it 
wrong. If we were simply passing the string through unchecked this would not 
have been an issue (or it would have been an issue against the CRT).

PR at 
https://bitbucket.org/larry/cpython350/pull-requests/18/issue-24917-time_strftime-buffer-over-read/diff

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24910] Windows MSIs don't have unique display names

[Roundup Robot]

Roundup Robot added the comment:

New changeset 3594400f3202 by Steve Dower in branch '3.5':
Issue #24910: Windows MSIs now have unique display names.
https://hg.python.org/cpython/rev/3594400f3202

New changeset 0295d2cdc9d3 by Steve Dower in branch 'default':
Issue #24910: Windows MSIs now have unique display names.
https://hg.python.org/cpython/rev/0295d2cdc9d3

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24910] Windows MSIs don't have unique display names

[Steve Dower]

Changes by Steve Dower :


--
resolution:  -> fixed
stage: needs patch -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23551] IDLE to provide menu options for using PIP

[Terry J. Reedy]

Terry J. Reedy added the comment:

This is really needed. A number of recent threads on python-list indicate that 
beginners are having problems with using pip.

Donald, can you answer Saimadhav's questions two messages above?

Saimadhav, can you post whatever you have done so far?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23551] IDLE to provide menu link to PIP gui.

[Terry J. Reedy]

Terry J. Reedy added the comment:

Nick, Paul, Marcus, can any of you answer Saimadhav's question 3 messages up?

--
nosy: +Marcus.Smith, ncoghlan, paul.moore
title: IDLE to provide menu options for using PIP -> IDLE to provide menu link 
to PIP gui.
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23551] IDLE to provide menu link to PIP gui.

[Donald Stufft]

Donald Stufft added the comment:

pip doesn't really support being called as an API, there's been requests for it 
before but nobody has yet come forward to specify what parts in particular they 
need. The practical effect of this is that there's no backwards compatibility 
promises for anything you import from pip currently. I would suggest using a 
subprocess to communicate with pip unless you can define what operations you 
need so the pip developers can settle on a real public API for them.

As for your actual questions:

1. I'm pretty sure it requires restarting the process because pkg_resources has 
a cache and pip currently isn't designed in a way that it assumes you're going 
to freeze in anything but it's own process.

2. I think this depends on what version of of Python it's being used with and 
where the user installed it to. I think Steve Dower has switched Python 3.5's 
default install location to Program Files which does require it to be run as an 
administrator if you want to install into the site-packages instead of the user 
packages directory.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23551] IDLE to provide menu link to PIP gui.

[Donald Stufft]

Changes by Donald Stufft :


--
nosy: +steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[STINNER Victor]

STINNER Victor added the comment:

> Alex & I do agree nearest/even must be used.

Ok, so Python 3.6 should be updated to use ROUND_HALF_EVEN, and then
updated patches should be backported to Python 3.4 and 3.5. Right?

Right now, Python 3.6 uses ROUND_HALF_UP.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23517] datetime.utcfromtimestamp rounds results incorrectly

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

You may find it easier to start with a patch for 3.5 which is the only time 
sensitive task.  I'll be happy to review your code in whatever form you find it 
easier to submit, but I believe in hg it is easier to forward port than to 
backport.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23551] IDLE to provide menu link to PIP gui.

[Mark Lawrence]

Mark Lawrence added the comment:

Can we use ideas from 
https://sites.google.com/site/pydatalog/python/pip-for-windows ?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25011] Smarter rl complete: hide private and special names

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

When press tabulation just after the dot, the output is too long. It contains 
all attributes, most of them are dunder names.

>>> int.
int.__abs__(int.__doc__ int.__int__(
int.__pos__(int.__round__(  int.__truediv__(
int.__add__(int.__eq__( int.__invert__( 
int.__pow__(int.__rpow__(   int.__trunc__(
int.__and__(int.__flags__   int.__itemsize__
int.__prepare__(int.__rrshift__(int.__weakrefoffset__
int.__base__(   int.__float__(  int.__le__( 
int.__qualname__int.__rshift__( int.__xor__(
int.__bases__   int.__floor__(  int.__lshift__( 
int.__radd__(   int.__rsub__(   int.bit_length(
int.__basicsize__   int.__floordiv__(   int.__lt__( 
int.__rand__(   int.__rtruediv__(   int.conjugate(
int.__bool__(   int.__format__( int.__mod__(
int.__rdivmod__(int.__rxor__(   int.denominator
int.__call__(   int.__ge__( int.__module__  
int.__reduce__( int.__setattr__(int.from_bytes(
int.__ceil__(   int.__getattribute__(   int.__mro__ 
int.__reduce_ex__(  int.__sizeof__( int.imag
int.__class__(  int.__getnewargs__( int.__mul__(
int.__repr__(   int.__str__(int.mro(
int.__delattr__(int.__gt__( int.__name__
int.__rfloordiv__(  int.__sub__(int.numerator
int.__dict__int.__hash__(   int.__ne__( 
int.__rlshift__(int.__subclasscheck__(  int.real
int.__dictoffset__  int.__index__(  int.__neg__(
int.__rmod__(   int.__subclasses__( int.to_bytes(
int.__dir__(int.__init__(   int.__new__(
int.__rmul__(   int.__subclasshook__(   
int.__divmod__( int.__instancecheck__(  int.__or__( 
int.__ror__(int.__text_signature__  

Proposed patch hides underscored names and show dunder names only if a prefix 
starts with '__', and private members only if a prefix starts with '_'.

>>> int.
int.bit_length(  int.conjugate(   int.denominator  int.from_bytes(  int.imag
 int.mro( int.numeratorint.real int.to_bytes(
>>> int.__
int.__abs__(int.__divmod__( int.__init__(   
int.__neg__(int.__rlshift__(int.__sub__(
int.__add__(int.__doc__ int.__instancecheck__(  
int.__new__(int.__rmod__(   int.__subclasscheck__(
int.__and__(int.__eq__( int.__int__(
int.__or__( int.__rmul__(   int.__subclasses__(
int.__base__(   int.__flags__   int.__invert__( 
int.__pos__(int.__ror__(int.__subclasshook__(
int.__bases__   int.__float__(  int.__itemsize__
int.__pow__(int.__round__(  int.__text_signature__
int.__basicsize__   int.__floor__(  int.__le__( 
int.__prepare__(int.__rpow__(   int.__truediv__(
int.__bool__(   int.__floordiv__(   int.__lshift__( 
int.__qualname__int.__rrshift__(int.__trunc__(
int.__call__(   int.__format__( int.__lt__( 
int.__radd__(   int.__rshift__( int.__weakrefoffset__
int.__ceil__(   int.__ge__( int.__mod__(
int.__rand__(   int.__rsub__(   int.__xor__(
int.__class__(  int.__getattribute__(   int.__module__  
int.__rdivmod__(int.__rtruediv__(   
int.__delattr__(int.__getnewargs__( int.__mro__ 
int.__reduce__( int.__rxor__(   
int.__dict__int.__gt__( int.__mul__(
int.__reduce_ex__(  int.__setattr__(
int.__dictoffset__  int.__hash__(   int.__name__
int.__repr__(   int.__sizeof__( 
int.__dir__(int.__index__(  int.__ne__( 
int.__rfloordiv__(  int.__str__(

(The completion of int._ returns nothing because int has no private members).

--
components: Library (Lib)
files: rlcomplete_filter_private.patch
keywords: patch
messages: 249928
nosy: pitrou, serhiy.storchaka, twouters
priority: normal
severity: normal
stage: patch review
status: open
title: Smarter rl complete: hide private and special names
type: enhancement
versions: Python 3.6
Added file: http://bugs.python.org/file40377/rlcomplete_filter_private.patch

___
Python tracker 

___
_

[issue25009] queue.Queue() does not validate the maxsize argument

[Raymond Hettinger]

Raymond Hettinger added the comment:

The norm for pure python code is to duck type when objects are used rather than 
when they are passed in.

The Queue module is very old and for the most part, its straight-forward design 
has worked out well (i.e. there aren't any real usability issues in actual 
practice).

Accordingly, I'm going to decline the request.

--
assignee:  -> rhettinger
nosy: +rhettinger
resolution:  -> rejected
status: open -> closed
type: behavior -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25003] os.urandom() should call getrandom(2) not getentropy(2)

[Raymond Hettinger]

Changes by Raymond Hettinger :


--
assignee:  -> haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24912] The type of cached objects is mutable

[Mark Shannon]

Mark Shannon added the comment:

Why has the change allowing the __class__ attribute of modules been merged?

It is not necessary (as I have stated repeatedly) and not known to be safe.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24912] The type of cached objects is mutable

[Larry Hastings]

Larry Hastings added the comment:

Because the BDFL asked that it be so.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17582] xml.etree.ElementTree does not preserve whitespaces in attributes

[Raymond Hettinger]

Raymond Hettinger added the comment:

Stefan, can you opine on the patches and whether they should be backported?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24969] functools.lru_cache: a way to set decorator arguments at runtime

[Raymond Hettinger]

Raymond Hettinger added the comment:

Sorry, I'm going to reject this one.  This option was considered during the 
design of the LRU cache and not included because it complicated the design, 
because the use cases were not common (the norm is set and forget), and because 
the __wrapped__ attribute provides a means of rewrapping or unwrapping as 
needed (i.e. there are reasonable workarounds).

--
resolution:  -> rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24999] Segfault in test_re.test_sre_character_class_literals() when Python is compiled by ICC

[Raymond Hettinger]

Changes by Raymond Hettinger :


--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24912] The type of cached objects is mutable

[Mark Shannon]

Mark Shannon added the comment:

Well, there are important reasons why not to make  the __class__ attribute of 
modules mutable. 

First of all, there is no valid rationale.

How do know for sure that modifying the class of the sys or builtins module is 
not going to cause much the same problems the changing the class of ints did?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25008] Deprecate smtpd

[Raymond Hettinger]

Raymond Hettinger added the comment:

Bringing in Eric V Smith's comment in
http://bugs.python.org/issue25002#msg249873 :

'''
I use smtpd.py for testing my code. But it's not such a big deal that I 
couldn't live without it. If I have some time to burn, I might convert it to 
asyncio so I can continue to use it.

'''

Bringing in R David Murray's comment in 
http://bugs.python.org/issue25002#msg249882 :

'''smtpd is used for testing smtplib.  It is also used in test_logging.  I 
would object to having it removed, but I suspect we can manage to rewrite it by 
3.8.  Maybe Eric and I can collaborate on it.
'''

--
nosy: +barry, rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25012] pathlib should allow converting to absolute paths without resolving symlinks

[David Barnett]

New submission from David Barnett:

There doesn't seem to be any helper in pathlib to expand a relative path to an 
absolute path *without* resolving symlinks.

For example, if I want to convert
  pathlib.Path('some/path')
to
  pathlib.Path('/full/path/to/some/path')
where some/path is a symlink, my best option seems to be
  pathlib.Path(os.path.abspath(str(pathlib.Path('some/path'

--
components: Library (Lib)
messages: 249936
nosy: mu_mind
priority: normal
severity: normal
status: open
title: pathlib should allow converting to absolute paths without resolving 
symlinks
type: behavior
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24999] Segfault in test_re.test_sre_character_class_literals() when Python is compiled by ICC

[Tim Peters]

Tim Peters added the comment:

The longobject.c warnings are almost certainly unrelated to the test_re crash.

If shifting right twice (adding parens for clarity):

(LONG_MAX >> PyLong_SHIFT) >> PyLong_SHIFT.

squashes the warnings, that would be a substantially clearer way to express the 
intent than the

SIZEOF_LONG*CHAR_BIT-1 >= 2*PyLong_SHIFT

spelling.  Adding a comment *explaining* the intent would be even better.

For the segfault issue, best guess is that it's a compiler optimization bug.  
Not common as mud, or even as common as nuisance warnings, but not all that 
rare either ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24912] The type of cached objects is mutable

[Mark Shannon]

Mark Shannon added the comment:

Oh, and has anyone considered the potential impact this might have on Jython or 
PyPy? Modules are quite fundamental to the way that Python works.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23551] IDLE to provide menu link to PIP gui.

[Steve Dower]

Steve Dower added the comment:

I actually made the default per-user, specifically for cases like this. With a 
default install that the current user has done, pip does not need to be run as 
admin.

If the system administrator has installed Python for all users, then --user 
needs to be specified to run without admin. (Could the admin add config to pip 
for all users to make this the default? I know we discussed it - don't remember 
what the result was.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24912] The type of cached objects is mutable

[Roundup Robot]

Roundup Robot added the comment:

New changeset 27cc5cce0292 by Guido van Rossum in branch '3.5':
Issue #24912: Prevent __class__ assignment to immutable built-in objects.
https://hg.python.org/cpython/rev/27cc5cce0292

New changeset 1c55f169f4ee by Guido van Rossum in branch '3.5':
Issue #24912: Prevent __class__ assignment to immutable built-in objects. 
(Merge 3.5.0 -> 3.5)
https://hg.python.org/cpython/rev/1c55f169f4ee

New changeset b045465e5dba by Guido van Rossum in branch 'default':
Issue #24912: Prevent __class__ assignment to immutable built-in objects. 
(Merge 3.5 -> 3.6)
https://hg.python.org/cpython/rev/b045465e5dba

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25012] pathlib should allow converting to absolute paths without resolving symlinks

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

pathlib.Path.cwd() / pathlib.Path('some/path')

--
nosy: +pitrou, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25008] Deprecate smtpd

[Raymond Hettinger]

Raymond Hettinger added the comment:

FWIW, I would like to see the module live on (after conversion to asyncio).  
Doug Hellman's example at https://pymotw.com/2/smtpd/ makes for a nice 
demonstration of Python's capabilities.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24912] The type of cached objects is mutable

[Guido van Rossum]

Guido van Rossum added the comment:

Mark, please calm down. Modules are incredibly simple objects compared to 
classes -- and yet you can change the __class__ of a class. You can also 
directly modify the __dict__ of a module. You can shoot yourself in the foot 
phenomenally this way, but that's not the point. The point is not to violate 
*internal* constraints of the interpreter. Changing immutable objects was 
violating such a constraint. Changing a module's __class__ is not.

--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24912] The type of cached objects is mutable

[Mark Shannon]

Mark Shannon added the comment:

Guido,
I just think this change is misguided. The original rationale for the change 
just doesn't hold water.

If you want the change anyway, then fine, but this seems an odd way to 
introduce it.

--
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23551] IDLE to provide menu link to PIP gui.

[Terry J. Reedy]

Terry J. Reedy added the comment:

Pip's rather extensive command-line API can also be used as a code API. The 
signature of pip.main is (args=None), where args = sys.argv[1:] if not passed. 
I presume this will not change. Reusing the command-list API had the advantage 
of reusing the existing docs, help output, and knowlege of how to use pip.

The API is to run 'status = pip.main(args)', where args is a list containing 
quoted versions of the command-line args, the same as one would pass to 
subprocess.Popen.  Output is strings written to stdout (and stderr), which can 
be captured with StringIO. While not as nice as getting, say, a list of strings 
from a list_command function, I consider this better than repeatedly calling 
Popen and getting encoded bytes (in whatever encoding) from pipes.

I tested this with list, show pip, help install, and install --upgrade pip.  
After a shell restart,  import pip imported the new version. I am working on a 
wrapper for the pip.main call.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23551] IDLE to provide menu link to PIP gui.

[Donald Stufft]

Donald Stufft added the comment:

Yea ``pip.main(args)`` won't change. I'm not sure how well parts of pip will 
handle being in a persistent-ish process, but the API itself will work.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23551] IDLE to provide menu link to PIP gui.

[Donald Stufft]

Donald Stufft added the comment:

There is a "site" config file which works for all installs on that particular 
machine, there is not a per Python configuration file, though I don't see why 
we couldn't add one.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24912] The type of cached objects is mutable

[Nathaniel Smith]

Nathaniel Smith added the comment:

I didn't want to get further into the weeds of debating basic design points on 
the tracker while blocking rc3, but, for the record -- the proposal in 
msg249504 that one could keep the namespace of an old and new module object 
effectively in sync by defining __getattr__ on the new module doesn't work 
AFAICT. (I assume this is the basis of Mark's assertion that the change is 
unmotivated.) Going through __getattr__ creates an unacceptable performance hit 
on attribute access. You can avoid this by copying the namespace entries over 
(b/c __getattr__ is only called as a fallback when __dict__ lookup fails), but 
then you have the same problem as before of needing to keep things in sync. 
(And remember that module globals can change at any time -- the __globals__ 
pointer for any functions defined in that module will continue to point to the 
old namespace.)

Re: PyPy: last I heard their main complaint about __class__ assignment was that 
it was hard to restrict it to *only* handle the cases that CPython does. (They 
do have a concept of "heap types"; AFAICT it's just a flag that says "the 
CPython version of this type has some quirks that we need to check for and 
emulate".) Not sure about Jython, but I doubt they have a natural heap/non-heap 
type distinction either.

Thanks Guido for handling the patches!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16180] cannot quit pdb when there is a syntax error in the debuggee (must kill it)

[Roundup Robot]

Roundup Robot added the comment:

New changeset 26c4db1a0aea by Terry Jan Reedy in branch '2.7':
Issue #16180: Exit pdb if file has syntax error, instead of trapping user
https://hg.python.org/cpython/rev/26c4db1a0aea

New changeset 2d4aac2ab253 by Terry Jan Reedy in branch '3.4':
Issue #16180: Exit pdb if file has syntax error, instead of trapping user
https://hg.python.org/cpython/rev/2d4aac2ab253

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16180] cannot quit pdb when there is a syntax error in the debuggee (must kill it)

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
resolution:  -> fixed
stage: test needed -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[Larry Hastings]

Larry Hastings added the comment:

Pull request accepted.  Please forward-merge.  Thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23551] IDLE to provide menu link to PIP gui.

[Terry J. Reedy]

Terry J. Reedy added the comment:

The persistence issue is why I tried several calls before posting above. I am 
hoping that none of your command functions leave pip unusable, or if they do, 
you would be willing to have them fixed.

Attached is promised wrapper.

--
Added file: http://bugs.python.org/file40378/tem_pip.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23551] IDLE to provide menu link to PIP gui.

[Donald Stufft]

Donald Stufft added the comment:

Yea, we'd be willing to fix things where we can. I think the biggest problem 
you'll run into is probably going to be pkg_resources and it's module scoped 
cache of the sys.path and what items are installed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25008] Deprecate smtpd

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

Removing smtpd would definitely be a hardship for me right now, probably for 
obvious reasons.  I use it in testing frameworks, and even wrote a library 
called lazr.smtptest that is built around smtpd.  In Mailman, we have an LMTP 
server built on smtpd that implements *the* official way of getting mail into 
the system from an upstream mail server.  Mailman 3 core FWIW is Python 3.4 and 
3.5 compatible.

That all being said, I've also been thinking lately about an asyncio-based 
reimplementation of SMTP and LMTP.  asyncore/chat and smtpd are not the easiest 
modules to work with, extend, understand, or debug.  I'd be very happy to 
collaborate with folks on an asyncio-based version.  Maybe we can put that out 
as a third party module, with a long term plan on pulling it into the stdlib 
and promoting it instead of smtpd.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24912] The type of cached objects is mutable

[Eric Snow]

Eric Snow added the comment:

While I recognize the practicality/purity argument here, I somewhat agree with 
Mark.  Assigning to module.__class__ makes sense for the use case, but it does 
open up a number of negative possible side effects (e.g. changing 
sys.__class__).  Ideally it would be restricted to just the currently running 
module, but doing that through __class__ assignment is not a great idea.  
Having a dedicated builtin function or even syntax would make more sense.

In some ways I'm reminded of __metaclass__ in Python 2 class definitions.  
Something like that has come up before but didn't get a lot of momentum at the 
time.  Ultimately I'd like to see a more explicit mechanism to meet the need 
that motivated the __class__ change here (as well as the 
replace-this-module-in-sys-modules technique). [1] While you can already 
accomplish this via a custom finder, that's not the most convenient tool for 
this sort of situation.

FWIW, if assigning to sys.modules[__name__].__class__ is the blessed way then 
we should make it official in the language reference.  The same goes for 
modules replacing themselves in sys.modules.  If we have reservations about 
making either official (which I for one do [2]) then we should work on a more 
appropriate official solution.


[1] It all boils down to customizing the current module's type from within the 
module rather than from outside (using a custom loader).
[2] Neither spelling is very friendly.

--
nosy: +brett.cannon, eric.snow

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23551] IDLE to provide menu link to PIP gui.

[Terry J. Reedy]

Terry J. Reedy added the comment:

You are right. While multiple info requests work, I found a test that fails. 
'list' showed that I had Pillow 2.7.0.  'install -U Pillow' updated to 2.9.0'. 
'list' again still showed 2.7.0. 'show Pillow' then failed because 2.7.0 could 
not be found.  After an install, we need to delete the one local reference to 
pip(.main), which should remain the only reference, and reload to refresh that 
cache.

There is an obscure bug in either python, pip, or tem_pip such that 
runpip('-h') silently crashes python in both console and Idle. I will try to 
localize it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[Roundup Robot]

Roundup Robot added the comment:

New changeset 09b62202d9b7 by Steve Dower in branch '3.5':
Issue #24917: time_strftime() Buffer Over-read. Patch by John Leitch.
https://hg.python.org/cpython/rev/09b62202d9b7

New changeset 8b81b7ad2d0a by Steve Dower in branch '3.5':
Issue #24917: Moves NEWS entry under Library.
https://hg.python.org/cpython/rev/8b81b7ad2d0a

New changeset a29b49d57769 by Steve Dower in branch '3.4':
Issue #24917: time_strftime() Buffer Over-read. Patch by John Leitch.
https://hg.python.org/cpython/rev/a29b49d57769

New changeset 6fc744ac3953 by Steve Dower in branch 'default':
Issue #24917: time_strftime() Buffer Over-read. Patch by John Leitch.
https://hg.python.org/cpython/rev/6fc744ac3953

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Larry Hastings]

Larry Hastings added the comment:

Pull request accepted.  I had to do it manually, as I got a Misc/NEWS merge 
conflict.  But from where I'm sitting it looks like Bitbucket understands the 
pull request was accepted and merged.

Please forward-merge.  Thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24748] Change of behavior for importlib between 3.4 and 3.5 with DLL loading

[Larry Hastings]

Changes by Larry Hastings :


--
assignee:  -> ncoghlan
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[Steve Dower]

Steve Dower added the comment:

I applied the AIX fix to 3.4 (introduced in #19634). Python 3.2 and 3.3 aren't 
affected.

--
assignee:  -> steve.dower
resolution:  -> fixed
stage: patch review -> commit review
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23551] IDLE to provide menu link to PIP gui.

[Saimadhav Heblikar]

Saimadhav Heblikar added the comment:

Hi Terry,

Unfortunately, I had to perform a system reinstall, which wiped out the patch.

I dont mind doing it again, but it might take until the weekend to complete.

Also, now my questions from earlier are answered, it should be easier.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[Larry Hastings]

Larry Hastings added the comment:

The tests from this patch fail on Linux.

-
First: There is no trailing % test on Linux, and glibc's strftime() happily 
ignores a trailing %, so no ValueError is raised.

Python should do either one or the other of the following:

1) Python should enforce no trailing % in the strftime format string,
   or

2) the test suite shouldn't assume that a trailing % in the strftime
   value string raises a ValueError.

I can live with either of these, not sure what the right decision is.

-
Second: The test from the patch assumes that strftime('%#') will raise a 
ValueError.  Again, strftime in Linux glibc happily accepts "%#" as a format 
string and thus no ValueError is raised.

Python is agnostic about native format units in the strftime() format string.  
Therefore I strongly assert that Python must not assume that "%#" is an illegal 
format string.  Therefore the tests must not assume that "%#" raises ValueError.

Given that the code used to crash, I do want the code path exercised in the 
test suite.  So I propose that the test attempt time.strftime('%#') and accept 
either success or ValueError.


-

Given that I've accepted this patch into 3.5.0, and it's now blocking my 
release, it is implicitly a "release blocker".  I need to resolve this tonight 
before I can tag 3.5.0rc3.  I'm going to dinner, and maybe we can have a quick 
discussion and come to a decision in the next hour or two.


p.s. The checkin also flunked PEP 7.  *sigh*

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[John Leitch]

John Leitch added the comment:

Is there a way to see what style guidelines have been violated? The only thing 
I can think of is the curly braces in the Windows check, but I was following 
the conventions of the surrounding code.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[Larry Hastings]

Larry Hastings added the comment:

The starting curly brace goes on the same line as the statement starting the 
block.  Keywords followed by a left parenthesis get a space between the keyword 
and the parenthesis.  It's a small matter, I'm really much more interested in 
reconciling the behavior of strftime and its tests.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[Larry Hastings]

Larry Hastings added the comment:

Sorry, maybe you inherited those violations.  I was in a hurry and not in a 
charitable frame of mind.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[Steve Dower]

Steve Dower added the comment:

I'll fix the PEP 7 stuff (mostly inherited).

MSVC can also handle '%#' and '%' as format strings (producing '' in both 
cases). If that matches libc behaviour, I see no reason to raise a ValueError 
here apart from consistency with previous Python releases.

If consistency with system C libraries is preferred, then I'll change the 
condition to simply break out of the loop and make the test call the function 
expecting success. I assume there's an AIX buildbot we can watch to see the 
impact on that part?

--
nosy: +georg.brandl
priority: normal -> release blocker
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[John Leitch]

John Leitch added the comment:

Yikes--your comment prompted me to look at the check-in, and it seems my patch 
wasn't properly applied. The curly braces got tweaked, which is minor as you 
stated, but more importantly the AIX code should not decref format. That could 
introduce problems bigger than what this patch was attempting to fix.

And, not to dwell, but where do you see a keyword immediately followed by a 
left parens? I want to make sure everything is properly polished in the future, 
and the only thing I see is the untouched "for".

Regarding your initial concerns:
1) I think we should enforce no trailing % so as to not pass format strings 
that may cause undefined behavior.

2) How about expecting ValueError on Windows/AIX, and pass on all other 
platforms?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[Steve Dower]

Steve Dower added the comment:

Whoops, must have done a bad copy-paste to get that DECREF in there (I couldn't 
apply the patch directly because it didn't come from an HG repo, so I had to do 
it by hand).

MSVC seems somewhat inconsistent about its response:
>>> strftime('aaa%')
''
>>> strftime('%')
Traceback (most recent call last):
  File "", line 1, in 
ValueError: Invalid format string

How closely do we want to simply expose the underlying C library's behaviour?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[John Leitch]

John Leitch added the comment:

If it's so wildly inconsistent, it's my opinion that Python should perform its 
own validation to achieve better cross-platform support. The alternative is 
playing a never ending game of whack-a-mole, or just accepting that format 
strings may cause exceptions in some platforms but not others.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25012] pathlib should allow converting to absolute paths without resolving symlinks

[eryksun]

eryksun added the comment:

There's a public method that's almost what you want:

>>> print(pathlib.Path.absolute.__doc__)
Return an absolute version of this path.  This function works
even if the path doesn't point to anything.

No normalization is done, i.e. all '.' and '..' will be kept along.
Use resolve() to get the canonical path to a file.

However, it appears to be only accidentally public. It isn't tested; it isn't 
mentioned in the docs; and it doesn't do the [expected] path normalization. 
Currently it's used as a default in resolve() if self._flavour.resolve(self) 
returns None.

--
nosy: +eryksun
versions: +Python 3.4 -Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[Steve Dower]

Steve Dower added the comment:

Having now read over this whole issue, I don't actually see where the security 
vulnerability is (on Windows at least).

This is a user-mode read, so it can only access memory in the same process, and 
it doesn't display it anywhere. The worst that can happen is that it hits an 
unreadable page and crashes, which falls under "undefined behaviour due to 
invalid input".

I think we should just revert the patch completely.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[Larry Hastings]

Larry Hastings added the comment:

I have convinced myself that disallowing trailing % marks on all platforms is 
the right thing to do.  Whether or not the underlying platform tolerates it, it 
is never *valid* input to strftime.

I have a patch incubating at home.  I'll put it up for review when I get back, 
maybe an hour.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[Steve Dower]

Steve Dower added the comment:

I'll get my commits out of the way to make the buildbots green again.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[Larry Hastings]

Larry Hastings added the comment:

Oh, maybe it was all like that before.  Sorry, I was in a hurry and not in a 
charitable frame of mind.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[Roundup Robot]

Roundup Robot added the comment:

New changeset 73227e857d6b by Steve Dower in branch '3.5':
Issue #24917: Backed out changeset 09b62202d9b7
https://hg.python.org/cpython/rev/73227e857d6b

New changeset 743924064dc8 by Steve Dower in branch 'default':
Issue #24917: Backed out changeset 09b62202d9b7
https://hg.python.org/cpython/rev/743924064dc8

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[Steve Dower]

Steve Dower added the comment:

Yeah, I came in late and in too much of a rush too :(

Backed out the broken changes, and now I'm off to bed. Not convinced there's 
anything that needs fixing here, and IIRC we argued through the different 
platform behaviours on the issue when I removed the null check that was 
preventing the overread.

If we do anything now, it should just be `if (outbuf[1] == '\0') break` to 
avoid the overread. strftime() is probably too far gone to make it consistent 
across platforms now.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[John Leitch]

John Leitch added the comment:

Yes, this is a user-mode read, but I disagree with the assertion that it's not 
possible to use this to disclose memory. While it isn't as critical as 
something that outright dumps memory, there is logic that throws exceptions 
based on values encountered while reading outside the bounds of the buffer. 
This could be used as a channel to infer what is or isn't in adjacent memory. 
That it's user-mode doesn't matter--if an application exposes the format string 
as attack surface, suddenly process memory can be probed. So, it's not 
heartbleed, but it does have security implications. If you'd like, I can take a 
shot at building a PoC.

Further, it's best to err on the side of caution with bugs like these; just 
because it doesn't seem like major issue now doesn't mean someone won't come 
along in the future and prove otherwise.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[Larry Hastings]

Larry Hastings added the comment:

I'm going to back this out of 3.5.0rc3.  I'm still willing to discuss accepting 
it into 3.5.0 final in some form, but for now I don't want to hold up the 
release.

Steve: it should never be possible to crash the Python interpreter with 
well-formed Python code.  Whether or not it's possible to exploit it, I do 
think this crash is something the interpreter should prevent.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24917] time_strftime() Buffer Over-read

[eryksun]

eryksun added the comment:

> MSVC seems somewhat inconsistent about its response:
> >>> strftime('aaa%')
> ''

That's not due to MSVC. It's setting errno to EINVAL. The problem is that 
time_strftime is testing (buflen > 0 || i >= 256 * fmtlen). The initial value 
of the outbuf size i is 1024, so when (fmtlen <= 4), the value of (256 * 
fmtlen) is less than or equal to i, and it assumes "the format yields an empty 
result" without considering the value of errno. So instead of raising an 
exception for EINVAL, it calls PyUnicode_DecodeLocaleAndSize to return an empty 
string:

>>> strftime('aaa%')
Breakpoint 1 hit
ucrtbase!strftime:
07fe`e2bac3e0 4883ec38sub rsp,38h
0:000> gu
python35!time_strftime+0x1f5:
`5de9c785 488bcb  mov rcx,rbx
0:000> be 0; g
Breakpoint 0 hit
ucrtbase!_errno:
07fe`e2b341b0 48895c2408  mov qword ptr [rsp+8],rbx 
ss:`0028f320=

errno is 22:

0:000> pt; dd @rax l1
`002ddb50  0016
0:000> bd 0; g
Breakpoint 2 hit
python35!PyUnicode_DecodeLocaleAndSize:
`5df55070 4053pushrbx
0:000> k 2
Child-SP  RetAddr   Call Site
`0028f318 `5de9c81a python35!PyUnicode_DecodeLocaleAndSize
`0028f320 `5df1d5c2 python35!time_strftime+0x28a
0:000> g
''

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com