Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators
On 20 November 2014 06:48, MRAB wrote:
> On 2014-11-19 20:10, Guido van Rossum wrote:
>
>> There's a new PEP proposing to change how to treat StopIteration
>> bubbling up out of a generator frame (not caused by a return from
>> the frame). The proposal is to replace such a StopIteration with a
>> RuntimeError (chained to the original StopIteration), so that only
>> *returning* from a generator (or falling off the end) causes the
>> iteration to terminate.
>>
>> The PEP says """any generator that depends on an implicitly-raised
> StopIteration to terminate it will have to be rewritten to either catch
> that exception or use a for-loop"""
>
> Shouldn't that be "... explicitly-raised ...", because returning raises
> StopIteration implicitly? ("raise StopIteration" is explicit)
The ways a generator can currently be terminated:
return [value]
raise StopIteration[([value])]
The first case is unchanged, and (as Chris noted), the second case can be
trivially converted to the first case.
It's the third implicit case which would no longer be allowed, which means
no more "or stop()" tricks to get generator expressions to terminate early.
(If folks wanted to restore that capability, they'd instead need to propose
new syntax that works the same way in both comprehensions and generator
expressions, rather than relying on the lazy evaluation of generator
expressions)
Cheers,
Nick.
--
Nick Coghlan | [email protected] | Brisbane, Australia
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators
On 20 November 2014 06:15, Benjamin Peterson wrote: > > On Wed, Nov 19, 2014, at 15:10, Guido van Rossum wrote: > > There's a new PEP proposing to change how to treat StopIteration bubbling > > up out of a generator frame (not caused by a return from the frame). The > > proposal is to replace such a StopIteration with a RuntimeError (chained > > to > > the original StopIteration), so that only *returning* from a generator > > (or > > falling off the end) causes the iteration to terminate. > > > > The proposal unifies the behavior of list comprehensions and generator > > expressions along the lines I had originally in mind when they were > > introduced. It renders useless/illegal certain hacks that have crept into > > some folks' arsenal of obfuscated Python tools. > > > > In Python 3.5 the proposed change is conditional on: > > > > from __future__ import replace_stopiteration_in_generators > > Drive-by comment: This seems like a terribly awkward name. Could a > shorter and sweeter name not be found? > I think my suggestion was something like "from __future__ import generator_return". I saw that style as somewhat similar to "from __future__ import division" - it just tells you what the change affects (in this case, returning from generators), while requiring folks to look up the documentation to find out the exact details of the old behaviour and the new behaviour. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators
I've made some updates to the PEP: - added 19-Nov-2014 to Post-History - removed "implicitly-raised" from the abstract - changed the __future__ thing to generator_return - added a clarifying paragraph that Chris added to his own draft - added a link to http://bugs.python.org/issue22906 which has a proof-of-concept patch There's still a lively discussion on python-ideas; Steven D'Aprano has dug up quite a bit of evidence that StopIteration is used quite a bit in ways that will break under the new behavior, and there also seems to be quite a bit of third-party information that recommends StopIteration over return to terminate a generator early. However I don't see much evidence that the current behavior is *better* than the proposal -- I see the current behavior as a definite wart, and I have definitely seen people struggle to debug silent early loop termination due to an "escaped" StopIteration. That said, I think for most people the change won't matter, some people will have to apply one of a few simple fixes, and a rare few will have to rewrite their code in a non-trivial way (sometimes this will affect "clever" libraries). I wonder if the PEP needs a better transition plan, e.g. - right now, start an education campaign - with Python 3.5, introduce "from __future__ import generator_return", and silent deprecation warnings - with Python 3.6, start issuing non-silent deprecation warnings - with Python 3.7, make the new behavior the default (subject to some kind of review) It would also be useful if we could extend the PEP with some examples of the various categories of fixes that can be applied easily, e.g. a few examples of "raise StopIteration" directly in a generator that can be replaced with "return" (or omitted, if it's at the end); a few examples of situations where "yield from" can supply an elegant fix (and an alternative for code that needs to be backward compatible with Python 3.2 or 2.7); and finally (to be honest) an example of code that will require being made more complicated. Oh, and it would also be nice if the PEP included some suggested words that 3rd party educators can use to explain the relationship between StopIteration and generators in a healthier way (preferably a way that also applies to older versions). Chris, are you up to drafting these additions? On Thu, Nov 20, 2014 at 2:05 AM, Nick Coghlan wrote: > On 20 November 2014 06:15, Benjamin Peterson wrote: > >> >> On Wed, Nov 19, 2014, at 15:10, Guido van Rossum wrote: >> > There's a new PEP proposing to change how to treat StopIteration >> bubbling >> > up out of a generator frame (not caused by a return from the frame). The >> > proposal is to replace such a StopIteration with a RuntimeError (chained >> > to >> > the original StopIteration), so that only *returning* from a generator >> > (or >> > falling off the end) causes the iteration to terminate. >> > >> > The proposal unifies the behavior of list comprehensions and generator >> > expressions along the lines I had originally in mind when they were >> > introduced. It renders useless/illegal certain hacks that have crept >> into >> > some folks' arsenal of obfuscated Python tools. >> > >> > In Python 3.5 the proposed change is conditional on: >> > >> > from __future__ import replace_stopiteration_in_generators >> >> Drive-by comment: This seems like a terribly awkward name. Could a >> shorter and sweeter name not be found? >> > > I think my suggestion was something like "from __future__ import > generator_return". > > I saw that style as somewhat similar to "from __future__ import division" > - it just tells you what the change affects (in this case, returning from > generators), while requiring folks to look up the documentation to find out > the exact details of the old behaviour and the new behaviour. > > Cheers, > Nick. > > -- > Nick Coghlan | [email protected] | Brisbane, Australia > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators
On Thu, 20 Nov 2014 11:36:54 -0800 Guido van Rossum wrote: > I've made some updates to the PEP: > > - added 19-Nov-2014 to Post-History > - removed "implicitly-raised" from the abstract > - changed the __future__ thing to generator_return To me "generator_return" sounds like the addition to generator syntax allowing for return statements (which was done as part of the "yield from" PEP). How about "generate_escape"? Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators
On 20.11.14 21:58, Antoine Pitrou wrote: To me "generator_return" sounds like the addition to generator syntax allowing for return statements (which was done as part of the "yield from" PEP). How about "generate_escape"? Or may be "generator_stop_iteration"? ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators
On Thu, Nov 20, 2014 at 12:13 PM, Serhiy Storchaka wrote: > On 20.11.14 21:58, Antoine Pitrou wrote: > >> To me "generator_return" sounds like the addition to generator syntax >> allowing for return statements (which was done as part of the "yield >> from" PEP). How about "generate_escape"? >> > > Or may be "generator_stop_iteration"? > Or just "generator_stop"? -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators
On 11/20/2014 5:04 PM, Guido van Rossum wrote: On Thu, Nov 20, 2014 at 12:13 PM, Serhiy Storchaka mailto:[email protected]>> wrote: On 20.11.14 21:58, Antoine Pitrou wrote: To me "generator_return" sounds like the addition to generator syntax allowing for return statements (which was done as part of the "yield from" PEP). How about "generate_escape"? Or may be "generator_stop_iteration"? Or just "generator_stop"? +1 About as sort as we could get for the informed user reader and neutrally cryptic so a naive reader will know to look it up rather than guess (wrongly) -- Terry Jan Reedy ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators
On Thu, 20 Nov 2014 14:04:24 -0800 Guido van Rossum wrote: > On Thu, Nov 20, 2014 at 12:13 PM, Serhiy Storchaka > wrote: > > > On 20.11.14 21:58, Antoine Pitrou wrote: > > > >> To me "generator_return" sounds like the addition to generator syntax > >> allowing for return statements (which was done as part of the "yield > >> from" PEP). How about "generate_escape"? > >> > > > > Or may be "generator_stop_iteration"? > > > > Or just "generator_stop"? That sounds good. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators
On 11/20/2014 2:36 PM, Guido van Rossum wrote: There's still a lively discussion on python-ideas; Steven D'Aprano has dug up quite a bit of evidence that StopIteration is used quite a bit in ways that will break under the new behavior, and there also seems to be quite a bit of third-party information that recommends StopIteration over return to terminate a generator early. However I don't see much evidence that the current behavior is *better* than the proposal -- I see the current behavior as a definite wart, and I have definitely seen people struggle to debug silent early loop termination due to an "escaped" StopIteration. That said, I think for most people the change won't matter, some people will have to apply one of a few simple fixes, and a rare few will have to rewrite their code in a non-trivial way (sometimes this will affect "clever" libraries). I wonder if the PEP needs a better transition plan, e.g. - right now, start an education campaign For StackOverflow, someone with a high-enough rep (you, Guido?) should add a tag for StopIteration with text like "The Python `StopIteration` exception." (Copied from the current AttributeError tag, typeerror and NameError also exist.). If and when the PEP is accepted, I would be willing to add comments and the new tag to existing and future questions/answers. - with Python 3.5, introduce "from __future__ import generator_return", and silent deprecation warnings - with Python 3.6, start issuing non-silent deprecation warnings I agree that two deprecation periods are needed. - with Python 3.7, make the new behavior the default (subject to some kind of review) It would also be useful if we could extend the PEP with some examples of the various categories of fixes that can be applied easily, e.g. a few examples of "raise StopIteration" directly in a generator that can be replaced with "return" (or omitted, if it's at the end); a few examples of situations where "yield from" can supply an elegant fix (and an alternative for code that needs to be backward compatible with Python 3.2 or 2.7); and finally (to be honest) an example of code that will require being made more complicated. Agree that explicit fixes are needed. Deprecation message could refer to section of PEP. Oh, and it would also be nice if the PEP included some suggested words that 3rd party educators can use to explain the relationship between StopIteration and generators in a healthier way (preferably a way that also applies to older versions). Chris, are you up to drafting these additions? -- Terry Jan Reedy ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators
On Fri, Nov 21, 2014 at 6:36 AM, Guido van Rossum wrote: > It would also be useful if we could extend the PEP with some examples of the > various categories of fixes that can be applied easily, e.g. a few examples > of "raise StopIteration" directly in a generator that can be replaced with > "return" (or omitted, if it's at the end); a few examples of situations > where "yield from" can supply an elegant fix (and an alternative for code > that needs to be backward compatible with Python 3.2 or 2.7); and finally > (to be honest) an example of code that will require being made more > complicated. > > Oh, and it would also be nice if the PEP included some suggested words that > 3rd party educators can use to explain the relationship between > StopIteration and generators in a healthier way (preferably a way that also > applies to older versions). > > Chris, are you up to drafting these additions? Sure, no problem. Will knock something up shortly. ChrisA ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators
On Fri, Nov 21, 2014 at 9:04 AM, Guido van Rossum wrote: > On Thu, Nov 20, 2014 at 12:13 PM, Serhiy Storchaka > wrote: >> >> On 20.11.14 21:58, Antoine Pitrou wrote: >>> >>> To me "generator_return" sounds like the addition to generator syntax >>> allowing for return statements (which was done as part of the "yield >>> from" PEP). How about "generate_escape"? >> >> >> Or may be "generator_stop_iteration"? > > > Or just "generator_stop"? Unrelated to the GeneratorExit exception. I don't think that'll be a problem though. ChrisA ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators
On Thu, Nov 20, 2014 at 3:13 PM, Antoine Pitrou wrote: > On Thu, 20 Nov 2014 14:04:24 -0800 > Guido van Rossum wrote: > > > On Thu, Nov 20, 2014 at 12:13 PM, Serhiy Storchaka > > wrote: > > > > > On 20.11.14 21:58, Antoine Pitrou wrote: > > > > > >> To me "generator_return" sounds like the addition to generator syntax > > >> allowing for return statements (which was done as part of the "yield > > >> from" PEP). How about "generate_escape"? > > >> > > > > > > Or may be "generator_stop_iteration"? > > > > > > > Or just "generator_stop"? > > That sounds good. > OK, updated the PEP. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators
On Fri, Nov 21, 2014 at 10:34 AM, Chris Angelico wrote: > On Fri, Nov 21, 2014 at 6:36 AM, Guido van Rossum wrote: >> It would also be useful if we could extend the PEP with some examples of the >> various categories of fixes that can be applied easily, e.g. a few examples >> of "raise StopIteration" directly in a generator that can be replaced with >> "return" (or omitted, if it's at the end); a few examples of situations >> where "yield from" can supply an elegant fix (and an alternative for code >> that needs to be backward compatible with Python 3.2 or 2.7); and finally >> (to be honest) an example of code that will require being made more >> complicated. >> >> Oh, and it would also be nice if the PEP included some suggested words that >> 3rd party educators can use to explain the relationship between >> StopIteration and generators in a healthier way (preferably a way that also >> applies to older versions). >> >> Chris, are you up to drafting these additions? > > Sure, no problem. Will knock something up shortly. Examples and explanatory text added. https://raw.githubusercontent.com/Rosuav/GenStopIter/master/pep-0479.txt ChrisA ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
