[Python-Dev] Should asyncio ignore KeyboardInterrupt?

2015-07-05 Thread drekin 
> Once long ago in Internet time (issue 581232) time.sleep on windows was
> not interruptible and this was fixed.  Is it possible the work on EINTR
> has broken that fix?
> 
> (I don't currently have 3.5 installed on windows to test that theory...)

It is no problem to interrupt time.sleep() with Ctrl-C for me (Python 3.5.0b2, 
Windows Vista x64).

Regards, Adam Bartoš
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] [RELEASED] Python 3.5.0b3 is now available

2015-07-05 Thread Larry Hastings 



On behalf of the Python development community and the Python 3.5 release 
team, I'm relieved to announce the availability of Python 3.5.0b3.


Python 3.5 has now entered "feature freeze".  By default new features 
may no longer be added to Python 3.5.


This is a preview release, and its use is not recommended for production 
settings.


An important reminder for Windows users about Python 3.5.0b3: if 
installing Python 3.5.0b2 as a non-privileged user, you may need to 
escalate to administrator privileges to install an update to your C 
runtime libraries.



You can find Python 3.5.0b2 here:

   https://www.python.org/downloads/release/python-350b3/

Happy hacking,


//arry/
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] What's New editing

2015-07-05 Thread R. David Murray 
Just so people aren't caught unawares, it is very unlikely that I will have
time to be the final editor on "What's New for 3.5" they way I was for 3.3 and
3.4.  I've tried to encourage people to keep What's New up to date, but
*someone* should make a final editing pass.  Ideally they'd do at least the
research Serhiy did last year on checking that there's a mention for all of the
versionadded and versionchanged 3.5's in the docs.  Even better would be to
review the NEWS and/or commit history...but *that* is a really big job these
days

--David
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Importance of "async" keyword

2015-07-05 Thread Sven R. Kunze 

Thanks, Nick, for you reasoned response.

On 03.07.2015 11:40, Nick Coghlan wrote:

On 3 July 2015 at 06:55, Sven R. Kunze  wrote:

My understanding of coloring is "needs special treatment".

Being special or not (containing an 'await' or not), as long as I don't need
to care, I can call them either way (with 'await' or not) and each case
works sensibly that's fine with me.

I'm afraid you're going to be disappointed in that regard, as wishing
that event driven programs behaved more like synchronous programs is
like wishing that complex numbers behaved like real numbers.
Seems like we stick to this example once again. So, let me get this 
straight:


1) I can add, subtract, multiply and divide real numbers.
2) I can add, subtract, multiply and divide complex numbers.
3) I can even add, subtract, multiply and divide complex numbers AND 
real numbers altogether in a single expression.
4) Granted, complex numbers can do more but that basically follows from 
their definition and does not jeopardize ease of usage.


A) I can call a function and might get a return value.
B) I can await an awaitable and might get a return value.
C) I cannot use them interchangeably. Why? Because people think we 
cannot have the best things of both worlds.
D) Granted, awaitables can do more but that basically follows from their 
definition and does not need to jeopardize ease of usage.



There's
an extra level of complexity that is being deliberately introduced in
order to improve Python's ability to express certain kinds of
algorithms, and it isn't effective to just try to wish that complexity
away.

The payoff is that code that otherwise needs to be written as a long
series of disconnected callback chains (as has long been possible with
Twisted) can instead be written to look more like comparable
synchronous code


That is great. So, let's do the next step.


(and this approach should bring with it much improved
introspection support, at least in 3.5+ now that gi_yieldfrom and
cr_await are being exposed at the Python level).


Sensible would be something similar to:
await function: suspension point and runs the function until completion
call awaitable: runs the awaitable until completion

These both fail, and deliberately so: we don't know what they're
supposed to mean, and we refuse the temptation to guess.


Where do we guess here? It is a definition.


They're also
quite likely to indicate a bug (e.g. forgetting to call a native
coroutine function to get the coroutine out of it, forgetting to wait
for an awaitable) rather than something folks have done deliberately.

It's possible shorthand adapters may emerge over time, like:

 # Call awaitable from synchronous code
 def wait_for_result(awaitable):
 """Usage: result = asyncio.wait_for_result(awaitable)"""
return 
asyncio.get_event_loop().run_until_complete(awaitable.__await__())

 # Call blocking operation from asynchronous code
def blocking_call(f, *args, **kwds):
 """Usage: result = await asyncio.blocking_call(f, *args, **kwds))"""
 cb = functools.partial(f, *args, **kwds)
 return asyncio.get_event_loop().run_in_executor(cb)

However, those can be written already as utility functions, so we can
wait and see how strong the demand is for them as adapters. (They may
also be potentially useful to have as recipes in the documentation)
Sure, that would be reasonable. You have the first guy asking explicitly 
for these types of adapters provided by the current syntax and removal 
of 'async'.


Until a solution, we return to watching and have one reason less to 
switch to Python 3. :-/

___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Importance of "async" keyword

2015-07-05 Thread Chris Angelico 
On Mon, Jul 6, 2015 at 7:50 AM, Sven R. Kunze  wrote:
> Seems like we stick to this example once again. So, let me get this
> straight:
>
> 1) I can add, subtract, multiply and divide real numbers.
> 2) I can add, subtract, multiply and divide complex numbers.
> 3) I can even add, subtract, multiply and divide complex numbers AND real
> numbers altogether in a single expression.
> 4) Granted, complex numbers can do more but that basically follows from
> their definition and does not jeopardize ease of usage.

Until you run into a TypeError: unorderable types: complex() >
complex(), at which point you realize that they aren't a simple
superset of reals with all the same operations supported.

ChrisA
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What's New editing

2015-07-05 Thread Nick Coghlan 
On 6 July 2015 at 03:52, R. David Murray  wrote:
> Just so people aren't caught unawares, it is very unlikely that I will have
> time to be the final editor on "What's New for 3.5" they way I was for 3.3 and
> 3.4.

And thank you again for your work on those!

> I've tried to encourage people to keep What's New up to date, but
> *someone* should make a final editing pass.  Ideally they'd do at least the
> research Serhiy did last year on checking that there's a mention for all of 
> the
> versionadded and versionchanged 3.5's in the docs.  Even better would be to
> review the NEWS and/or commit history...but *that* is a really big job these
> days

What would your rough estimate of the scope of work be? As you note,
the amount of effort involved in doing a thorough job of that has
expanded beyond what can reasonably be expected of volunteer
contributors, so I'm wondering if it might make sense for the PSF to
start offering a contract technical writing gig to finalise the What's
New documentation for each new release.

After all, the What's New doc is an essential component of
communicating changes in recommended development practices to Python
educators, so ensuring we do a good job with that can have a big
multiplier effect on all the other work that goes into creating each
new release.

Regards,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Importance of "async" keyword

2015-07-05 Thread Steven D'Aprano 
On Sun, Jul 05, 2015 at 11:50:00PM +0200, Sven R. Kunze wrote:

> Seems like we stick to this example once again. So, let me get this 
> straight:
> 
> 1) I can add, subtract, multiply and divide real numbers.
> 2) I can add, subtract, multiply and divide complex numbers.

I don't think that this is a complelling analogy for calling regular 
functions and awaitable functions. Doing arithmetic is a bit more 
complicated than just the four elementary operators you mention. 
Contrast:

10 < 20

10+20j < 20+10j

It is not just that complex numbers can do more than real numbers. They 
can also do *less*.

But aside from that, your analogy looks to me like this:

(1) I can do arithmetic on reals; 
(2) I can do arithmetic on complex numbers;
(3) and I can do arithmetic on mixed real + complex expressions;

(A) I can travel in a car to the shops down the road;
(B) I can travel in a space ship to the moon;

(C) so why can't I travel in a car to the moon? Or use a space ship to 
fly to the shops down the road? Because people think we cannot have 
the best things of both worlds!

It seems to me that if "people think we cannot have the best things of 
both worlds" (your words), it is because there are solid reasons for 
that belief.

Could somebody build a car that can fly to the moon? Probably, but it 
would cost *more* than the combination of separate space ship plus car, 
it would require as much maintenance and support as a space ship, and 
the fuel economy when driving to work would be terrible. Not to mention 
all the complaints about the noise and the pollution.

I think that the distinction between regular and concurrent routines is 
practical and necessary, *not* just because of people's closed minds, 
but because of decades of collective experience with them.

To convince me differently, you will need more than some rather dubious 
analogies or arguments from theoretical purity that sequential 
syncronous code is just a special case of concurrent asyncronous code. 
An actual working implementation speaks most loudly of all, but at the 
very least you will need to stick to concrete arguments, not analogies.

Are you aware of any other languages which have eliminated the 
distinction between regular and concurrent functions? If it has already 
been done, that would make a good argument in favour of your idea.


-- 
Steven
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Importance of "async" keyword

2015-07-05 Thread Nick Coghlan 
On 6 July 2015 at 10:27, Chris Angelico  wrote:
> On Mon, Jul 6, 2015 at 7:50 AM, Sven R. Kunze  wrote:
>> Seems like we stick to this example once again. So, let me get this
>> straight:
>>
>> 1) I can add, subtract, multiply and divide real numbers.
>> 2) I can add, subtract, multiply and divide complex numbers.
>> 3) I can even add, subtract, multiply and divide complex numbers AND real
>> numbers altogether in a single expression.
>> 4) Granted, complex numbers can do more but that basically follows from
>> their definition and does not jeopardize ease of usage.
>
> Until you run into a TypeError: unorderable types: complex() >
> complex(), at which point you realize that they aren't a simple
> superset of reals with all the same operations supported.

Exactly. While complex numbers are a superset of the real numbers from
a value perspective, from a behavioural perspective, there are things
you can do when working only with real numbers that you can't do when
you have to account for the fact that here might be complex numbers
are in the mix. In a Python context, the essential operations specific
to real numbers are listed at
https://docs.python.org/3/library/numbers.html#numbers.Real

There's also a difference between the scope of the math and cmath modules:

>>> import math, cmath
>>> set(dir(math)) - set(dir(cmath))
{'floor', 'pow', 'erf', 'trunc', 'copysign', 'expm1', 'ldexp',
'fsum', 'erfc', 'lgamma', 'frexp', 'gamma', 'factorial', 'log2',
'fabs', 'log1p', 'atan2', 'hypot', 'modf', 'radians', 'degrees',
'fmod', 'ceil'}

It's a general principle of design that expanding the kinds of values
you accept (e.g. from real numbers to complex numbers) means you
reduce the kinds of operations you can assume will work (e.g. to the
behaviours of 2D complex numbers, rather than the 1D real number
line). Similarly, as you step further down the numeric tower from real
numbers to rationals and integers, you reduce the set of supported
values, but you also increase the number of defined behaviours.

When it comes to coroutines and subroutines, coroutines are the
superset - a subroutine is just a coroutine that never suspends before
producing a result. You can thus make certain simplifying assumptions
when executing subroutines that you can't make when executing a
coroutine.

It also turns out that "never suspends" is actually problematic, which
is why nominally subroutine based code these days tends to instead
have *implicit* suspension points based on either operating system
level pre-emptive multithreading where suspension may occur at any
point or greenlet style state switching where suspension may occur as
part of any function call (whether an explicit call or as part of a
syntactic protocol). These approaches provide parallel execution at
the expense of the ability to reason locally about code correctness,
which then causes all sorts of *other* problems.

That said, I think there's definitely value in providing a very simple
answer to the "how do I make a blocking call from a coroutine?"
question, so I filed an RFE to add asyncio.blocking_call:
http://bugs.python.org/issue24571

I'm less convinced of the value of "asyncio.wait_for_result()", so I
haven't filed an RFE for that one.

Regards,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What's New editing

2015-07-05 Thread R. David Murray 
On Mon, 06 Jul 2015 11:06:41 +1000, Nick Coghlan  wrote:
> On 6 July 2015 at 03:52, R. David Murray  wrote:
> > Just so people aren't caught unawares, it is very unlikely that I will have
> > time to be the final editor on "What's New for 3.5" they way I was for 3.3 
> > and
> > 3.4.
> 
> And thank you again for your work on those!
> 
> > I've tried to encourage people to keep What's New up to date, but
> > *someone* should make a final editing pass.  Ideally they'd do at least the
> > research Serhiy did last year on checking that there's a mention for all of 
> > the
> > versionadded and versionchanged 3.5's in the docs.  Even better would be to
> > review the NEWS and/or commit history...but *that* is a really big job these
> > days
> 
> What would your rough estimate of the scope of work be? As you note,
> the amount of effort involved in doing a thorough job of that has
> expanded beyond what can reasonably be expected of volunteer
> contributors, so I'm wondering if it might make sense for the PSF to
> start offering a contract technical writing gig to finalise the What's
> New documentation for each new release.
> 
> After all, the What's New doc is an essential component of
> communicating changes in recommended development practices to Python
> educators, so ensuring we do a good job with that can have a big
> multiplier effect on all the other work that goes into creating each
> new release.

I can tell you that 3.4 took me approximately 67 hours according to my
time log.  That was going through the list prepared by Serhiy, and going
through pretty much all of the NEWS entries but not the commit log.  I'm
a precisionist, so I suspect someone less...ocd...about the details
could do it a bit faster, perhaps at the cost of some small amount of
accuracy :)

On the other hand, my knowledge of the code base and the development
that had been going on probably sped up my analysis and writeup of
the missing entries (and revision of existing entries, in many cases).

On gripping hand, I also did some small amount of documentation
rewriting and clarification along the way.

--David
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Importance of "async" keyword

2015-07-05 Thread Steve Dower 
"A) I can call a function and might get a return value.
B) I can await an awaitable and might get a return value.
C) I cannot use them interchangeably. Why?"

Function != awaitable - the answer is right there in the terminology. Different 
names, different things.

Given A, B and the fact that an awaitable is just an object, here's another 
important point:

* I can call a function and might get an awaitable.

If the function was decorated with async, the "might" becomes "will", but it's 
still just a function returning a value.

Because an awaitable is indistinguishable from any other value, the compiler 
doesn't know whether to await or not, so it has to be specified by the 
developer. If the language specifies it, then it's impossible to handle 
awaitable objects (for example, to put them in a list and await them later).

C# is another language that implemented this exact model a few years ago and it 
works well.

Cheers,
Steve

Top-posted from my Windows Phone

From: Sven R. Kunze
Sent: ‎7/‎5/‎2015 14:50
To: [email protected]
Subject: Re: [Python-Dev] Importance of "async" keyword

Thanks, Nick, for you reasoned response.

On 03.07.2015 11:40, Nick Coghlan wrote:
> On 3 July 2015 at 06:55, Sven R. Kunze  wrote:
>> My understanding of coloring is "needs special treatment".
>>
>> Being special or not (containing an 'await' or not), as long as I don't need
>> to care, I can call them either way (with 'await' or not) and each case
>> works sensibly that's fine with me.
> I'm afraid you're going to be disappointed in that regard, as wishing
> that event driven programs behaved more like synchronous programs is
> like wishing that complex numbers behaved like real numbers.
Seems like we stick to this example once again. So, let me get this
straight:

1) I can add, subtract, multiply and divide real numbers.
2) I can add, subtract, multiply and divide complex numbers.
3) I can even add, subtract, multiply and divide complex numbers AND
real numbers altogether in a single expression.
4) Granted, complex numbers can do more but that basically follows from
their definition and does not jeopardize ease of usage.

A) I can call a function and might get a return value.
B) I can await an awaitable and might get a return value.
C) I cannot use them interchangeably. Why? Because people think we
cannot have the best things of both worlds.
D) Granted, awaitables can do more but that basically follows from their
definition and does not need to jeopardize ease of usage.

> There's
> an extra level of complexity that is being deliberately introduced in
> order to improve Python's ability to express certain kinds of
> algorithms, and it isn't effective to just try to wish that complexity
> away.
>
> The payoff is that code that otherwise needs to be written as a long
> series of disconnected callback chains (as has long been possible with
> Twisted) can instead be written to look more like comparable
> synchronous code

That is great. So, let's do the next step.

> (and this approach should bring with it much improved
> introspection support, at least in 3.5+ now that gi_yieldfrom and
> cr_await are being exposed at the Python level).
>
>> Sensible would be something similar to:
>> await function: suspension point and runs the function until completion
>> call awaitable: runs the awaitable until completion
> These both fail, and deliberately so: we don't know what they're
> supposed to mean, and we refuse the temptation to guess.

Where do we guess here? It is a definition.

> They're also
> quite likely to indicate a bug (e.g. forgetting to call a native
> coroutine function to get the coroutine out of it, forgetting to wait
> for an awaitable) rather than something folks have done deliberately.
>
> It's possible shorthand adapters may emerge over time, like:
>
>  # Call awaitable from synchronous code
>  def wait_for_result(awaitable):
>  """Usage: result = asyncio.wait_for_result(awaitable)"""
> return 
> asyncio.get_event_loop().run_until_complete(awaitable.__await__())
>
>  # Call blocking operation from asynchronous code
> def blocking_call(f, *args, **kwds):
>  """Usage: result = await asyncio.blocking_call(f, *args, **kwds))"""
>  cb = functools.partial(f, *args, **kwds)
>  return asyncio.get_event_loop().run_in_executor(cb)
>
> However, those can be written already as utility functions, so we can
> wait and see how strong the demand is for them as adapters. (They may
> also be potentially useful to have as recipes in the documentation)
Sure, that would be reasonable. You have the first guy asking explicitly
for these types of adapters provided by the current syntax and removal
of 'async'.

Until a solution, we return to watching and have one reason less to
switch to Python 3. :-/
___
Python-Dev mailing list
[email protected]

Re: [Python-Dev] Importance of "async" keyword

2015-07-05 Thread Nick Coghlan 
On 6 July 2015 at 10:49, Steven D'Aprano  wrote:
> On Sun, Jul 05, 2015 at 11:50:00PM +0200, Sven R. Kunze wrote:
>
>> Seems like we stick to this example once again. So, let me get this
>> straight:
>>
>> 1) I can add, subtract, multiply and divide real numbers.
>> 2) I can add, subtract, multiply and divide complex numbers.
>
> I don't think that this is a complelling analogy for calling regular
> functions and awaitable functions.

I actually really like the analogy, as the "Why can't complex numbers
be just like real numbers?" reaction is pretty common when folks are
first learning to use them for things like signal analysis. I know it
didn't really sink in for me at university - it was only a couple of
years into doing digital signal processing full time that the concepts
involved in switching back and forth between linear real number based
time domain analysis and cyclical complex number based frequency
domain analysis really started to make sense to me.

There's a wonderful page at
http://betterexplained.com/articles/a-visual-intuitive-guide-to-imaginary-numbers/
which not only does a great job of providing a relatively intuitive
explanation of the behaviour of complex numbers as an answer to the
question "What is the square root of negative 1?", it also compares
them to the original reactions to the "absurd" notion of negative
numbers as an answer to the question "What is the result of
subtracting a larger number from a smaller one?".

"Why can't coroutines be just like subroutines?" strikes me as being a
similar case where the answer is "because not everything can be
appropriately modelled as a subroutine", but that answer isn't going
to make intuitive sense if you've never personally encountered the
limits of subroutine based algorithm design.

I also think the analogy helps provide good design guidance, as folks
are already familiar with the notion of "use real numbers if you can,
complex numbers if you need to", and extending that to an attitude of
"use subroutines if you can, coroutines if you need to" would be a
*very* good thing in terms of encouraging maintainable designs.

"Are complex numbers better than real numbers?" is hopefully a
self-evidently nonsensical question - some things are better modelled
as complex numbers, others as real numbers, so the only reasonable
answer is to ask "What are you trying to model?". "Are coroutines
better than subroutines?" is the same kind of question, just applied
to algorithm design rather than numeric modelling.

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What's New editing

2015-07-05 Thread David Mertz 
On Sun, Jul 5, 2015 at 6:06 PM, Nick Coghlan  wrote:

> On 6 July 2015 at 03:52, R. David Murray  wrote:
> > Just so people aren't caught unawares, it is very unlikely that I will
> have
> > time to be the final editor on "What's New for 3.5" they way I was for
> 3.3 and
> > 3.4.
>
> And thank you again for your work on those!
>
> > I've tried to encourage people to keep What's New up to date, but
> > *someone* should make a final editing pass.  Ideally they'd do at least
> the
> > research Serhiy did last year on checking that there's a mention for all
> of the
> > versionadded and versionchanged 3.5's in the docs.  Even better would be
> to
> > review the NEWS and/or commit history...but *that* is a really big job
> these
> > days
>
> What would your rough estimate of the scope of work be? As you note,
> the amount of effort involved in doing a thorough job of that has
> expanded beyond what can reasonably be expected of volunteer
> contributors, so I'm wondering if it might make sense for the PSF to
> start offering a contract technical writing gig to finalise the What's
> New documentation for each new release.
>

I think I might be able to "volunteer" for the task of writing/editing the
"What's New in 3.5" docs.  I saw David's comment on it today, so obviously
haven't yet had a chance to run it by my employer (Continuum Analytics),
but I have a hunch they would allow me to do it at least in large part as
paid time.  I am experienced as a technical writer, follow python-dev,
write about new features, but am *not*, however, my self an existing core
developer.

If there is interest in this, or at least it seems plausible, I can run it
by my employer tomorrow to see about getting enough time allocated (using
David Murray's past experience as a guideline for what's likely to be
needed).

Yours, David...

-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What's New editing

2015-07-05 Thread Nick Coghlan 
On 6 July 2015 at 12:42, David Mertz  wrote:
> I think I might be able to "volunteer" for the task of writing/editing the
> "What's New in 3.5" docs.  I saw David's comment on it today, so obviously
> haven't yet had a chance to run it by my employer (Continuum Analytics), but
> I have a hunch they would allow me to do it at least in large part as paid
> time.  I am experienced as a technical writer, follow python-dev, write
> about new features, but am *not*, however, my self an existing core
> developer.

I think the last point may be a positive rather than a negative when
it comes to effectively describing new features :)

> If there is interest in this, or at least it seems plausible, I can run it
> by my employer tomorrow to see about getting enough time allocated (using
> David Murray's past experience as a guideline for what's likely to be
> needed).

That would be very helpful! I'd definitely be able to find the time to
review and merge updates, it's the research-and-writing side that
poses a problem for me (appreciating a task is worth doing isn't the
same thing as wanting to do it myself!).

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 493: Redistributor guidance for Python 2.7 HTTPS

2015-07-05 Thread Nick Coghlan 
Hi folks,

As previously discussed on python-ideas, Red Hat has been looking at
ways to provide a smoother migration path for system administrators to
get to a point where system Python installations are verifying HTTPS
by default.

While we're not proposing that these changes be implemented upstream
(given that the change in default behaviour was already implemented
for 2.7.9), we *would* like to pursue an explicit upstream
recommendation regarding how deviations from the upstream behaviour
should be handled. Otherwise we run the risk of different
redistributors pursuing mutually incompatible approaches to the
migration, which would be a rather unfortunate outcome.

The proposal in PEP 493 offers two recommendations:

* one option that leaves the default behaviour alone, but provides an
easy environment variable based way to revert to the legacy behaviour
on a per-application basis (based on a design suggested by MAL)
* a second option designed for backporting the changes to versions
that advertise themselves as older than 2.7.9 that permits opting in
to the new behaviour on a system wide basis (based on a design
suggested by Robert Kuska)

The main change from the last version discussed on python-ideas is
that in both cases there's now a required attribute that
redistributors are expected to add to the SSL module to signal the
presence of the feature (and to provide some useful information about
its implementation).

I think this is mature enough now for me to request pronouncement (if
Guido's interested in doing that himself), or volunteers to be
BDFL-Delegate (if Guido's would prefer someone else handle it).

Regards,
Nick.

==
PEP: 493
Title: HTTPS verification recommendations for Python 2.7 redistributors
Version: $Revision$
Last-Modified: $Date$
Author: Nick Coghlan ,
Robert Kuska ,
Marc-André Lemburg 
Status: Draft
Type: Informational
Content-Type: text/x-rst
Created: 10-May-2015
Post-History: 06-Jul-2015


Abstract


PEP 476 updated Python's default handling of HTTPS certificates to be
appropriate for communication over the public internet. The Python 2.7 long
term maintenance series was judged to be in scope for this change, with the
new behaviour introduced in the Python 2.7.9 maintenance release.

This PEP provides recommendations to downstream redistributors wishing to
provide a smoother migration experience when helping their users to manage
this change in Python's default behaviour.

*Note that this PEP is not currently accepted, so it is a *proposed*
recommendation, rather than an active one.*


Rationale
=

PEP 476 changed Python's default behaviour to better match the needs and
expectations of developers operating over the public internet, a category
which appears to include most new Python developers. It is the position of
the authors of this PEP that this was a correct decision.

However, it is also the case that this change *does* cause problems for
infrastructure administrators operating private intranets that rely on
self-signed certificates, or otherwise encounter problems with the new default
certificate verification settings.

The long term answer for such environments is to update their internal
certificate management to at least match the standards set by the public
internet, but in the meantime, it is desirable to offer these administrators
a way to continue receiving maintenance updates to the Python 2.7 series,
without having to gate that on upgrades to their certificate management
infrastructure.

PEP 476 did attempt to address this question, by covering how to revert the
new settings process wide by monkeypatching the ``ssl`` module to restore the
old behaviour. Unfortunately, the ``sitecustomize.py`` based technique proposed
to allow system administrators to disable the feature by default in their
Standard Operating Environment definition has been determined to be
insufficient in at least some cases. The specific case of interest to the
authors of this PEP is the one where a Linux distributor aims to provide
their users with a
`smoother migration path
`__
than the standard one provided by consuming upstream CPython 2.7 releases
directly, but other potential challenges have also been pointed out with
updating embedded Python runtimes and other user level installations of Python.

Rather than allowing a plethora of mutually incompatibile migration techniques
to bloom, this PEP proposes two alternative approaches that redistributors
may take when addressing these problems. Redistributors may choose to implement
one, both, or neither of these approaches based on their assessment of the
needs of their particular userbase.

These designs are being proposed as a recommendation for redistributors, rather
than as new upstream features, as they are needed purely to support legacy
environments migrating from older versions of Python 2.7. Neither approach
is being proposed as an up