Re: [Python-Dev] Move selected documentation repos to PSF BitBucket account?
On Tue, 25 Nov 2014 16:17:06 +1000 Nick Coghlan wrote: > > The subsequent discussion has made me realise that dissatisfaction > with the current state of the infrastructure amongst core developers > is higher than I previously realised, so I've re-evaluated my own > priorities, and will be spending more time on both PEP 474 > (forge.python.org) and PEP 462 (the far more complex proposal to > consider introducing OpenStack style merge gating for CPython). > > At present, it looks like significant workflow improvements for the > main CPython repos will require custom tooling - there's very little > out there that will adequately support a long term maintenance branch, > a short term maintenance branch, additional security fix only > branches, and a separate main line of development. If something constructive comes out of this surrealistic discussion thread then all the better. Thank you Nick. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Move selected documentation repos to PSF BitBucket account?
On Tue Nov 25 2014 at 1:17:49 AM Nick Coghlan wrote: > On 25 November 2014 at 13:18, Donald Stufft wrote: > > > > There’s also the social aspects of it as well which is a big concern too > IMO. If you want to attract new contributors, not just keep the ones you > already have sometimes that means going to where the new contributors are > instead of telling them that they need to come to you. > > Again, the bottleneck at the moment is *reviewing* contributions, not > getting more of them. The two aspects are not unrelated, but my key > concern at this point is to make the patch review and acceptance > process easier, moreso than to increase the rate of incoming patches. > > My short term proposal to consider BitBucket as an option for support > repo hosting purposes was mostly driven by my delays in getting the > end-to-end signing PEPs for PyPI updated in a timely fashion - that > would have been much easier if the authors had been able to submit > pull requests, and I just reviewed and accepted them. > And then people thought, "ooh, if we are going to open that can of worms we might as well get the better network effect of GitHub" along with Guido going "git >= hg". > > The subsequent discussion has made me realise that dissatisfaction > with the current state of the infrastructure amongst core developers > is higher than I previously realised, so I've re-evaluated my own > priorities, and will be spending more time on both PEP 474 > (forge.python.org) and PEP 462 (the far more complex proposal to > consider introducing OpenStack style merge gating for CPython). > Yay! > > At present, it looks like significant workflow improvements for the > main CPython repos will require custom tooling - there's very little > out there that will adequately support a long term maintenance branch, > a short term maintenance branch, additional security fix only > branches, and a separate main line of development. > Yes, we are unfortunately special. > > Having our own Kallithea installation would provide additional > implementation options on that front, so I'll be keeping that in mind > as I work to get the proof-of-concept forge instance online. > I think this is a reasonable summary of what came up. Short of Donald and maybe Guido really liking the GitHub idea because of their reach, most of us just want better tooling and we all have various compromises we are willing to make to gain that tooling. I suspect if we make sure we add Bitbucket and GitHub login support to the issue tracker then that would help go a fair distance to helping with the GitHub pull of reach (and if we make it so people can simply paste in their fork's URL into the issue tracker and we simply grab a new patch for review that would go even farther). ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators
On Mon, Nov 24, 2014 at 10:22:54AM +1100, Chris Angelico wrote: > My point is that doing the same errant operation on a list or a dict > will give different exceptions. In the same way, calling next() on an > empty iterator will raise StopIteration normally, but might raise > RuntimeError instead. It's still an exception, it still indicates a > place where code needs to be changed I wouldn't interpret it like that. Calling next() on an empty iterator raises StopIteration. That's not a bug indicating a failure, it's the protocol working as expected. Your response to that may be to catch the StopIteration and ignore it, or to allow it to bubble up for something else to deal with it. Either way, next() raising StopIteration is not a bug, it is normal behaviour. (Failure to deal with any such StopIteration may be a bug.) However, if next() raises RuntimeError, that's not part of the protocol for iterators, so it is almost certainly a bug to be fixed. (Probably coming from an explicit "raise StopIteration" inside a generator function.) Your fix for the bug may be to refuse to fix it and just catch the exception and ignore it, but that's kind of nasty and hackish and shouldn't be considered good code. Do you agree this is a reasonable way to look at it? -- Steven ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators
On Wed, Nov 26, 2014 at 2:20 AM, Steven D'Aprano wrote:
> I wouldn't interpret it like that.
>
> Calling next() on an empty iterator raises StopIteration. That's not a
> bug indicating a failure, it's the protocol working as expected. Your
> response to that may be to catch the StopIteration and ignore it, or to
> allow it to bubble up for something else to deal with it. Either way,
> next() raising StopIteration is not a bug, it is normal behaviour.
>
> (Failure to deal with any such StopIteration may be a bug.)
>
> However, if next() raises RuntimeError, that's not part of the protocol
> for iterators, so it is almost certainly a bug to be fixed. (Probably
> coming from an explicit "raise StopIteration" inside a generator
> function.) Your fix for the bug may be to refuse to fix it and just
> catch the exception and ignore it, but that's kind of nasty and hackish
> and shouldn't be considered good code.
>
> Do you agree this is a reasonable way to look at it?
Yes. Specifically, your parenthesis in the middle is the important
bit. If you have a csv.DictReader, KeyError might be an important part
of your protocol (maybe you have an optional column in the CSV file),
but it should be caught before it crosses the boundary of "part of
your protocol". At some point, it needs to be converted into
ValueError, perhaps, or replaced with a default value, or some other
coping mechanism is used. Failure to deal with StopIteration when
calling next() is failure to cope with all of that function's
protocol, and that is most likely to be a bug. (There are times, and
some of them have been mentioned in these discussion threads, where
calling next() can never raise StopIteration, so there need be no
try/except - eg it=iter(string.split(" ")) - but that just means that
a StopIteration from that call is an error somewhere else. I'm
definitely happy for that kind of "shouldn't happen" to turn into a
RuntimeError rather than being left as an unexpectedly-short
generator.)
ChrisA
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators
On Wed, Nov 26, 2014 at 4:45 AM, Isaac Schwabacher wrote: > Yield can also raise StopIteration, if it's thrown in. The current > interaction of generator.throw(StopIteration) with yield from can't be > emulated under the PEP's behavior, though it's not clear that that's a > problem. > Hrm. I have *absolutely* no idea when you would use that, and how you'd go about reworking it to fit this proposal. Do you have any example code (production or synthetic) which throws StopIteration into a generator? ChrisA ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators
On Tue, Nov 25, 2014 at 9:49 AM, Chris Angelico wrote: > On Wed, Nov 26, 2014 at 4:45 AM, Isaac Schwabacher > wrote: > > Yield can also raise StopIteration, if it's thrown in. The current > interaction of generator.throw(StopIteration) with yield from can't be > emulated under the PEP's behavior, though it's not clear that that's a > problem. > > > > Hrm. I have *absolutely* no idea when you would use that, and how > you'd go about reworking it to fit this proposal. Do you have any > example code (production or synthetic) which throws StopIteration into > a generator? > Sounds like a good one for the obfuscated Python contest. :-) Unless the generator has a try/except surrounding the yield point into which the exception is thrown, it will bubble right out, and PEP 479 will turn this into a RuntimeError. I'll clarify this in the PEP (even though it logically follows from the proposal) -- I don't think there's anything to worry about. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators
On 11/25/14, Guido van Rossum wrote: > On Tue, Nov 25, 2014 at 9:49 AM, Chris Angelico [email protected]> wrote: > > > On Wed, Nov 26, 2014 at 4:45 AM, Isaac Schwabacher > > > '[email protected]>> wrote: > > > Yield can also raise StopIteration, if its thrown in. The current > > > interaction of generator.throw(StopIteration) with yield from cant be > > > emulated under the PEPs behavior, though its not clear that thats a > > > problem. > > > > Hrm. I have *absolutely* no idea when you would use that, To close the innermost generator in a yield-from chain. No, I don't know why you'd want to do that, either. > > and how > > you'd go about reworking it to fit this proposal. Do you have any > > example code (production or synthetic) which throws StopIteration into > > a generator? No. > Sounds like a good one for the obfuscated Python contest. :-) I'm just playing with my food now. :) > Unless the generator has a try/except surrounding the yield point into which > the exception is thrown, it will bubble right out, and PEP 479 will turn this > into a RuntimeError. I'll clarify this in the PEP (even though it logically > follows from the proposal) -- I don't think there's anything to worry about. > > > > -- > --Guido van Rossum (https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators
On Tue, Nov 25, 2014 at 10:12 AM, Isaac Schwabacher wrote: > On 11/25/14, Guido van Rossum wrote: > > On Tue, Nov 25, 2014 at 9:49 AM, Chris Angelico [email protected]')" target="1">[email protected]> wrote: > > > > > On Wed, Nov 26, 2014 at 4:45 AM, Isaac Schwabacher > > > [email protected]>> wrote: > > > > Yield can also raise StopIteration, if its thrown in. The current > interaction of generator.throw(StopIteration) with yield from cant be > emulated under the PEPs behavior, though its not clear that thats a problem. > > > > > > Hrm. I have *absolutely* no idea when you would use that, > > To close the innermost generator in a yield-from chain. No, I don't know > why you'd want to do that, either. For that purpose you should call the generator's close() method. This throws a GeneratorExit into the generator to give the generator a chance of cleanup (typically using try/finally). Various reasonable things happen if the generator misbehaves at this point -- if you want to learn what, read the code or experiment a bit on the command line (that's what I usually do). -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators
On 11/25/14, Chris Angelico wrote:
> On Wed, Nov 26, 2014 at 2:20 AM, Steven D'Aprano wrote:
> > I wouldn't interpret it like that.
> >
> > Calling next() on an empty iterator raises StopIteration. That's not a
> > bug indicating a failure, it's the protocol working as expected. Your
> > response to that may be to catch the StopIteration and ignore it, or to
> > allow it to bubble up for something else to deal with it. Either way,
> > next() raising StopIteration is not a bug, it is normal behaviour.
> >
> > (Failure to deal with any such StopIteration may be a bug.)
> >
> > However, if next() raises RuntimeError, that's not part of the protocol
> > for iterators, so it is almost certainly a bug to be fixed. (Probably
> > coming from an explicit "raise StopIteration" inside a generator
> > function.) Your fix for the bug may be to refuse to fix it and just
> > catch the exception and ignore it, but that's kind of nasty and hackish
> > and shouldn't be considered good code.
> >
> > Do you agree this is a reasonable way to look at it?
>
> Yes. Specifically, your parenthesis in the middle is the important
> bit. If you have a csv.DictReader, KeyError might be an important part
> of your protocol (maybe you have an optional column in the CSV file),
> but it should be caught before it crosses the boundary of "part of
> your protocol". At some point, it needs to be converted into
> ValueError, perhaps, or replaced with a default value, or some other
> coping mechanism is used. Failure to deal with StopIteration when
> calling next() is failure to cope with all of that function's
> protocol, and that is most likely to be a bug. (There are times, and
> some of them have been mentioned in these discussion threads, where
> calling next() can never raise StopIteration, so there need be no
> try/except - eg it=iter(string.split(" ")) - but that just means that
> a StopIteration from that call is an error somewhere else. I'm
> definitely happy for that kind of "shouldn't happen" to turn into a
> RuntimeError rather than being left as an unexpectedly-short
> generator.)
Yield can also raise StopIteration, if it's thrown in. The current interaction
of generator.throw(StopIteration) with yield from can't be emulated under the
PEP's behavior, though it's not clear that that's a problem.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (3.4): handle errors without a reason attribute
On Wed, 26 Nov 2014 00:06:06 + benjamin.peterson wrote: > https://hg.python.org/cpython/rev/e635c3ba75c8 > changeset: 93591:e635c3ba75c8 > branch: 3.4 > user:Benjamin Peterson > date:Tue Nov 25 15:43:58 2014 -0600 > summary: > handle errors without a reason attribute > > files: > Lib/test/support/__init__.py | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > > diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py > --- a/Lib/test/support/__init__.py > +++ b/Lib/test/support/__init__.py > @@ -698,7 +698,7 @@ > try: > f(*args, **kwargs) > except IOError as e: > -if e.reason == "CERTIFICATE_VERIFY_FAILED": > +if "CERTIFICATE_VERIFY_FAILED" in str(e): You should be able to keep the e.reason test if you only catch SSLError. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (3.4): handle errors without a reason attribute
On Tue, Nov 25, 2014, at 19:16, Antoine Pitrou wrote: > On Wed, 26 Nov 2014 00:06:06 + > benjamin.peterson wrote: > > > https://hg.python.org/cpython/rev/e635c3ba75c8 > > changeset: 93591:e635c3ba75c8 > > branch: 3.4 > > user:Benjamin Peterson > > date:Tue Nov 25 15:43:58 2014 -0600 > > summary: > > handle errors without a reason attribute > > > > files: > > Lib/test/support/__init__.py | 2 +- > > 1 files changed, 1 insertions(+), 1 deletions(-) > > > > > > diff --git a/Lib/test/support/__init__.py b/Lib/test/support/__init__.py > > --- a/Lib/test/support/__init__.py > > +++ b/Lib/test/support/__init__.py > > @@ -698,7 +698,7 @@ > > try: > > f(*args, **kwargs) > > except IOError as e: > > -if e.reason == "CERTIFICATE_VERIFY_FAILED": > > +if "CERTIFICATE_VERIFY_FAILED" in str(e): > > You should be able to keep the e.reason test if you only catch SSLError. Unfortunately, test_robotparser seems to manage to raise a cert validation error without a reason. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (3.4): handle errors without a reason attribute
On Tue, 25 Nov 2014 19:21:08 -0500 Benjamin Peterson wrote: > > > > You should be able to keep the e.reason test if you only catch SSLError. > > Unfortunately, test_robotparser seems to manage to raise a cert > validation error without a reason. Ahh... Perhaps it's urllib catching the SSLError and re-raising it as a URLError with the same message with no reason, then. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Please reconsider PEP 479.
I'm not particularly opposed to PEP 479, but the Abstract and Rationale could do with considerable clarification. They currently appear to promise things that are in disagreement with what the PEP actually delivers. The Abstract claims that the proposal will "unify the behaviour of list comprehensions and generator expressions", but it doesn't do that. What it actually does is provide special protection against escaped StopIteration exceptions in one particular context (the body of a generator). It doesn't prevent StopIteration from escaping anywhere else, including from list comprehensions, so if anything it actually *increases* the difference between generators and comprehensions. There may be merit in preventing rogue StopIterations escaping from generators, but the PEP should sell the idea on that basis, not on what sounds like a false promise that it will make comprehensions and generators behave identically. -- Greg ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Please reconsider PEP 479.
On Tue, Nov 25, 2014 at 4:58 PM, Greg wrote: > I'm not particularly opposed to PEP 479, but the Abstract and > Rationale could do with considerable clarification. I know. > They currently > appear to promise things that are in disagreement with what the PEP > actually delivers. > > The Abstract claims that the proposal will "unify the behaviour of > list comprehensions and generator expressions", but it doesn't do > that. What it actually does is provide special protection against > escaped StopIteration exceptions in one particular context (the > body of a generator). It doesn't prevent StopIteration from > escaping anywhere else, including from list comprehensions, so if > anything it actually *increases* the difference between generators > and comprehensions. > Hm, that sounds like you're either being contrarian or Chris and I have explained it even worse than I thought. Currently, there are cases where list(x for x in xs if P(x)) works while [x for x in xs if P(x)] fails (when P(x) raises StopIteration). With the PEP, both cases will raise some exception -- though you (and several others who've pointed this out) are right that the exception raised is different (RuntimeError vs. StopIteration) and if this occurs inside a __next__() method (not a generator) the StopIteration will cause the outer iteration to terminate silently. > There may be merit in preventing rogue StopIterations escaping > from generators, but the PEP should sell the idea on that basis, not > on what sounds like a false promise that it will make comprehensions > and generators behave identically. > I will weaken that language. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Please reconsider PEP 479.
On Wed, Nov 26, 2014 at 11:58 AM, Greg wrote: > The Abstract claims that the proposal will "unify the behaviour of > list comprehensions and generator expressions", but it doesn't do > that. I don't know that it completely unifies the behaviours, but it does merge them on the specific situation of a leaking StopIteration. With the original code examples that sparked this all off (see the first footnote in the PEP), current CPython has the list-comp form terminate cleanly, while the genexp form infinitely loops. With the POC patch on the issue tracker, both forms cause RuntimeError. Is there a better word than "unify" for that? ChrisA ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 479: Change StopIteration handling inside generators
On 26 November 2014 at 04:04, Guido van Rossum wrote: > On Tue, Nov 25, 2014 at 9:49 AM, Chris Angelico wrote: >> >> On Wed, Nov 26, 2014 at 4:45 AM, Isaac Schwabacher >> wrote: >> > Yield can also raise StopIteration, if it's thrown in. The current >> > interaction of generator.throw(StopIteration) with yield from can't be >> > emulated under the PEP's behavior, though it's not clear that that's a >> > problem. >> > >> >> Hrm. I have *absolutely* no idea when you would use that, and how >> you'd go about reworking it to fit this proposal. Do you have any >> example code (production or synthetic) which throws StopIteration into >> a generator? > > > Sounds like a good one for the obfuscated Python contest. :-) > > Unless the generator has a try/except surrounding the yield point into which > the exception is thrown, it will bubble right out, and PEP 479 will turn > this into a RuntimeError. I'll clarify this in the PEP (even though it > logically follows from the proposal) -- I don't think there's anything to > worry about. This is actually the edge case that needs to be handled in contextlib - a StopIteration raised by the with statement body gets thrown into the generator implementing the context manager. My current porting recommendation is to catch the RuntimeError & look at __cause__ to see if it's the StopIteration instance that was thrown in, but an alternative option would be to just call gen.close() in that case, rather than gen.throw(exc). Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
