Re: [Python-ideas] Membership of infinite iterators

On Thu, Nov 2, 2017 at 1:41 PM, Antoine Pitrou wrote: > On Thu, 2 Nov 2017 13:27:17 +0200 > Koos Zevenhoven wrote: > > ​There's a limit to how cheap the call to PyErr_CheckSignals() can be. > As I > > mentioned earlier, even just the fact that it's a C function call​ can be > > too much. > > > >

Re: [Python-ideas] Membership of infinite iterators

On Thu, 2 Nov 2017 13:27:17 +0200 Koos Zevenhoven wrote: > ​There's a limit to how cheap the call to PyErr_CheckSignals() can be. As I > mentioned earlier, even just the fact that it's a C function call​ can be > too much. > > That's why, in the above, I used a new name PyErr_PROBE_SIGNALS() inst

Re: [Python-ideas] Membership of infinite iterators

On Thu, Nov 2, 2017 at 3:39 AM, Nick Coghlan wrote: > On 2 November 2017 at 04:29, Koos Zevenhoven wrote: > >> SPOILER ALERT! At the moment, Nick's statement is in fact **always** >> true in **all** cases (at least when ignoring untypical cases and some >> inaccuracies in phrasing). ​Another que

Re: [Python-ideas] Membership of infinite iterators

On 2 November 2017 at 04:29, Koos Zevenhoven wrote: > SPOILER ALERT! At the moment, Nick's statement is in fact **always** true > in **all** cases (at least when ignoring untypical cases and some > inaccuracies in phrasing). ​Another question is whether the statement > **should** be true at all.​

Re: [Python-ideas] Membership of infinite iterators

Let me add something to the discussion that we had last month about code that cannot be interrupted with Ctrl-C or with the Jupyter Notebook interrupt command etc. I wasn't completely sure if anyone cared much about being able to interrupt long-running code and about things like atomic operations e

Re: [Python-ideas] Membership of infinite iterators

On Thu, Oct 19, 2017 at 3:42 AM, Nick Coghlan wrote: > On 19 October 2017 at 08:34, Greg Ewing > wrote: > >> Nick Coghlan wrote: >> >>> since breaking up the current single level loops as nested loops would >>> be a pre-requisite for allowing these APIs to check for signals while >>> they're run

Re: [Python-ideas] Membership of infinite iterators

Nick Coghlan wrote: having checks in both the producer & the consumer merely means that you'll be checking for signals twice every 65k iterations, rather than once. Here's a possible reason for wanting checks in the producers: If your producer happens to take a long time per iteration, and the

Re: [Python-ideas] Membership of infinite iterators

On 19 October 2017 at 08:34, Greg Ewing wrote: > Nick Coghlan wrote: > >> since breaking up the current single level loops as nested loops would be >> a pre-requisite for allowing these APIs to check for signals while they're >> running while keeping the per-iteration overhead low >> > > Is there

Re: [Python-ideas] Membership of infinite iterators

Nick Coghlan wrote: since breaking up the current single level loops as nested loops would be a pre-requisite for allowing these APIs to check for signals while they're running while keeping the per-iteration overhead low Is there really much overhead? Isn't it just checking a flag? -- Greg _

Re: [Python-ideas] Membership of infinite iterators

On Wed, Oct 18, 2017 at 10:30 PM, Serhiy Storchaka wrote: > 18.10.17 22:21, Koos Zevenhoven пише: > >> ​Nice! Though I'd really like a general ​solution that other code can >> easily adopt, even third-party extension libraries. >> > > What is the more general solution? For interrupting C code you

Re: [Python-ideas] Membership of infinite iterators

On Wed, Oct 18, 2017 at 10:21 PM, Koos Zevenhoven wrote: > On Wed, Oct 18, 2017 at 10:13 PM, Serhiy Storchaka > wrote: > >> 18.10.17 17:48, Nick Coghlan пише: >> >>> 1. It will make those loops slower, due to the extra overhead of >>> checking for signals (even the opcode eval loop includes all

Re: [Python-ideas] Membership of infinite iterators

18.10.17 22:21, Koos Zevenhoven пише: ​Nice! Though I'd really like a general ​solution that other code can easily adopt, even third-party extension libraries. What is the more general solution? For interrupting C code you need to check signals manually, either in every loop, or in every itera

Re: [Python-ideas] Membership of infinite iterators

On Wed, Oct 18, 2017 at 10:13 PM, Serhiy Storchaka wrote: > 18.10.17 17:48, Nick Coghlan пише: > >> 1. It will make those loops slower, due to the extra overhead of checking >> for signals (even the opcode eval loop includes all sorts of tricks to >> avoid actually checking for new signals, since

Re: [Python-ideas] Membership of infinite iterators

18.10.17 17:48, Nick Coghlan пише: 1. It will make those loops slower, due to the extra overhead of checking for signals (even the opcode eval loop includes all sorts of tricks to avoid actually checking for new signals, since doing so is relatively slow) 2. It will make those loops harder to m

Re: [Python-ideas] Membership of infinite iterators

On Wed, Oct 18, 2017 at 9:24 PM, MRAB wrote: > > The re module increments a counter on each iteration and checks for > signals when the bottom 12 bits are 0. > > The regex module increments a 16-bit counter on each iteration and checks > for signals when it wraps around to 0. > Then I​'d say tha

Re: [Python-ideas] Membership of infinite iterators

On 2017-10-18 15:48, Nick Coghlan wrote: On 18 October 2017 at 22:36, Koos Zevenhoven > wrote: On Wed, Oct 18, 2017 at 2:08 PM, Nick Coghlan mailto:ncogh...@gmail.com>> wrote: That one can only be fixed in count() - list already checks operator.leng

Re: [Python-ideas] Membership of infinite iterators

On Wed, Oct 18, 2017 at 6:56 PM, Paul Moore wrote: > OK, looks like I've lost track of what this thread is about then. > Sorry for the noise. > Paul > > ​No worries. I'm not sure I can tell what this thread is about either. Different people seem to have different ideas about that. My most recent

Re: [Python-ideas] Membership of infinite iterators

OK, looks like I've lost track of what this thread is about then. Sorry for the noise. Paul On 18 October 2017 at 16:40, Koos Zevenhoven wrote: > On Wed, Oct 18, 2017 at 6:36 PM, Paul Moore wrote: >> >> On 18 October 2017 at 16:27, Koos Zevenhoven wrote: >> > So you're talking about code that w

Re: [Python-ideas] Membership of infinite iterators

On Wed, Oct 18, 2017 at 6:36 PM, Paul Moore wrote: > On 18 October 2017 at 16:27, Koos Zevenhoven wrote: > > So you're talking about code that would make a C-implemented Python > iterable > > of strictly C-implemented Python objects and then pass this to something > > C-implemented like list(..)

Re: [Python-ideas] Membership of infinite iterators

On 18 October 2017 at 16:27, Koos Zevenhoven wrote: > So you're talking about code that would make a C-implemented Python iterable > of strictly C-implemented Python objects and then pass this to something > C-implemented like list(..) or sum(..), while expecting no Python code to be > run or sign

Re: [Python-ideas] Membership of infinite iterators

On Wed, Oct 18, 2017 at 5:48 PM, Nick Coghlan wrote: > On 18 October 2017 at 22:36, Koos Zevenhoven wrote: > >> On Wed, Oct 18, 2017 at 2:08 PM, Nick Coghlan wrote: >> >>> That one can only be fixed in count() - list already checks >>> operator.length_hint(), so implementing itertools.count.__l

Re: [Python-ideas] Membership of infinite iterators

On 18 October 2017 at 22:36, Koos Zevenhoven wrote: > On Wed, Oct 18, 2017 at 2:08 PM, Nick Coghlan wrote: > >> That one can only be fixed in count() - list already checks >> operator.length_hint(), so implementing itertools.count.__length_hint__() >> to always raise an exception would be enough

Re: [Python-ideas] Membership of infinite iterators

Hi all, FWIW, I just tried the list(count()) experiment on my phone (Termux Python interpreter under Android). Python 3.6.2 (default, Sep 16 2017, 23:55:07) [GCC 4.2.1 Compatible Android Clang 5.0.300080 ] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import

Re: [Python-ideas] Membership of infinite iterators

On 18 October 2017 at 22:53, Serhiy Storchaka wrote: > 18.10.17 13:22, Nick Coghlan пише: > >> 2.. These particular cases can be addressed locally using existing >> protocols, so the chances of negative side effects are low >> > > Only the particular case `count() in count()` can be addressed wit

Re: [Python-ideas] Membership of infinite iterators

18.10.17 13:22, Nick Coghlan пише: 2.. These particular cases can be addressed locally using existing protocols, so the chances of negative side effects are low Only the particular case `count() in count()` can be addressed without breaking the following examples: >>> class C: ... def __

Re: [Python-ideas] Membership of infinite iterators

On Wed, Oct 18, 2017 at 2:08 PM, Nick Coghlan wrote: > On 18 October 2017 at 20:39, Koos Zevenhoven wrote: > >> On Oct 18, 2017 13:29, "Nick Coghlan" wrote: >> >> On 18 October 2017 at 19:56, Koos Zevenhoven wrote: >> >>> I'm unable to reproduce the "uninterruptible with Ctrl-C"​ problem with

Re: [Python-ideas] Membership of infinite iterators

On 18 October 2017 at 20:39, Koos Zevenhoven wrote: > On Oct 18, 2017 13:29, "Nick Coghlan" wrote: > > On 18 October 2017 at 19:56, Koos Zevenhoven wrote: > >> I'm unable to reproduce the "uninterruptible with Ctrl-C"​ problem with >> infinite iterators. At least itertools doesn't seem to have

Re: [Python-ideas] Membership of infinite iterators

On Oct 18, 2017 13:29, "Nick Coghlan" wrote: On 18 October 2017 at 19:56, Koos Zevenhoven wrote: > I'm unable to reproduce the "uninterruptible with Ctrl-C"​ problem with > infinite iterators. At least itertools doesn't seem to have it: > > >>> import itertools > >>> for i in itertools.count():

Re: [Python-ideas] Membership of infinite iterators

On 18 October 2017 at 19:56, Koos Zevenhoven wrote: > I'm unable to reproduce the "uninterruptible with Ctrl-C"​ problem with > infinite iterators. At least itertools doesn't seem to have it: > > >>> import itertools > >>> for i in itertools.count(): > ... pass > ... > That's interrupting th

Re: [Python-ideas] Membership of infinite iterators

On 18 October 2017 at 10:56, Koos Zevenhoven wrote: > I'm unable to reproduce the "uninterruptible with Ctrl-C" problem with > infinite iterators. At least itertools doesn't seem to have it: > import itertools for i in itertools.count(): > ... pass > ... > ^CTraceback (most recent ca

Re: [Python-ideas] Membership of infinite iterators

On 18 October 2017 at 03:39, Koos Zevenhoven wrote: > On Tue, Oct 17, 2017 at 5:26 PM, Serhiy Storchaka > wrote: > >> 17.10.17 17:06, Nick Coghlan пише: >> >>> Keep in mind we're not talking about a regular loop you can break out of >>> with Ctrl-C here - we're talking about a tight loop inside

Re: [Python-ideas] Membership of infinite iterators

On Tue, Oct 17, 2017 at 9:44 PM, Brendan Barnwell wrote: > On 2017-10-17 07:26, Serhiy Storchaka wrote: > >> 17.10.17 17:06, Nick Coghlan пише: >> >>> >Keep in mind we're not talking about a regular loop you can break out of >>> >with Ctrl-C here - we're talking about a tight loop inside the >>>

Re: [Python-ideas] Membership of infinite iterators

On 2017-10-17 07:26, Serhiy Storchaka wrote: 17.10.17 17:06, Nick Coghlan пише: >Keep in mind we're not talking about a regular loop you can break out of >with Ctrl-C here - we're talking about a tight loop inside the >interpreter internals that leads to having to kill the whole host >process ju

Re: [Python-ideas] Membership of infinite iterators

On Tue, Oct 17, 2017 at 5:26 PM, Serhiy Storchaka wrote: > 17.10.17 17:06, Nick Coghlan пише: > >> Keep in mind we're not talking about a regular loop you can break out of >> with Ctrl-C here - we're talking about a tight loop inside the interpreter >> internals that leads to having to kill the w

Re: [Python-ideas] Membership of infinite iterators

17.10.17 17:06, Nick Coghlan пише: Keep in mind we're not talking about a regular loop you can break out of with Ctrl-C here - we're talking about a tight loop inside the interpreter internals that leads to having to kill the whole host process just to get out of it. And this is the root of t

Re: [Python-ideas] Membership of infinite iterators

On 17 October 2017 at 23:17, Koos Zevenhoven wrote: > On Tue, Oct 17, 2017 at 2:46 PM, Serhiy Storchaka > wrote: > >> 17.10.17 14:10, Nick Coghlan пише: >> >>> 1. It's pretty easy to write "for x in y in y" when you really meant to >>> write "for x in y", and if "y" is an infinite iterator, the

Re: [Python-ideas] Membership of infinite iterators

On 17 October 2017 at 21:46, Serhiy Storchaka wrote: > 17.10.17 14:10, Nick Coghlan пише: > >> 1. It's pretty easy to write "for x in y in y" when you really meant to >> write "for x in y", and if "y" is an infinite iterator, the "y in y" part >> will become an unbreakable infinite loop when exec

Re: [Python-ideas] Membership of infinite iterators

On Tue, Oct 17, 2017 at 2:46 PM, Serhiy Storchaka wrote: > 17.10.17 14:10, Nick Coghlan пише: > >> 1. It's pretty easy to write "for x in y in y" when you really meant to >> write "for x in y", and if "y" is an infinite iterator, the "y in y" part >> will become an unbreakable infinite loop when

Re: [Python-ideas] Membership of infinite iterators

17.10.17 14:10, Nick Coghlan пише: 1. It's pretty easy to write "for x in y in y" when you really meant to write "for x in y", and if "y" is an infinite iterator, the "y in y" part will become an unbreakable infinite loop when executed instead of the breakable one you intended (especially annoy

Re: [Python-ideas] Membership of infinite iterators

On 17 October 2017 at 19:19, Serhiy Storchaka wrote: > 17.10.17 09:42, Nick Coghlan пише: > >> On 17 October 2017 at 16:32, Nick Coghlan > ncogh...@gmail.com>> wrote: >> >> So this sounds like a reasonable API UX improvement to me, but you'd >> need to ensure that you don't inadvertently

Re: [Python-ideas] Membership of infinite iterators

On 17 October 2017 at 17:44, Steven D'Aprano wrote: > On Tue, Oct 17, 2017 at 04:42:35PM +1000, Nick Coghlan wrote: > > > I should also note that there's another option here beyond just returning > > "False": it would also be reasonable to raise an exception like > > "RuntimeError('Attempted nega

Re: [Python-ideas] Membership of infinite iterators

17.10.17 09:42, Nick Coghlan пише: On 17 October 2017 at 16:32, Nick Coghlan > wrote: So this sounds like a reasonable API UX improvement to me, but you'd need to ensure that you don't inadvertently change the external behaviour of *successful* containment

Re: [Python-ideas] Membership of infinite iterators

On Tue, Oct 17, 2017 at 04:42:35PM +1000, Nick Coghlan wrote: > I should also note that there's another option here beyond just returning > "False": it would also be reasonable to raise an exception like > "RuntimeError('Attempted negative containment check on infinite iterator')". I don't think

Re: [Python-ideas] Membership of infinite iterators

On 17 October 2017 at 16:32, Nick Coghlan wrote: > So this sounds like a reasonable API UX improvement to me, but you'd need > to ensure that you don't inadvertently change the external behaviour of > *successful* containment tests. > I should also note that there's another option here beyond ju

Re: [Python-ideas] Membership of infinite iterators

On 16 October 2017 at 11:12, Jason Campbell wrote: > I recently came across a bug where checking negative membership > (__contains__ returns False) of an infinite iterator will freeze the > program. > > It may seem a bit obvious, but you can check membership in range for > example without iterati

Re: [Python-ideas] Membership of infinite iterators

Given that: - it doesn't break anything; - the behavior makes sense; - it can avoid the machine freezing when one is not careful; It can be a good idea. Those generators could have iterators that are not themselves and have a __contains__ method behaving accordingly. I only did the mistake of u

Re: [Python-ideas] Membership of infinite iterators

בתאריך יום ג׳, 17 באוק׳ 2017, 00:13, מאת Terry Reedy ‏: > On 10/15/2017 9:12 PM, Jason Campbell wrote: > ... > > itertools.cycle could use membership from the underlying iterable > > If the underlying iterable is an iterator, this does not work. You > could define a Cycle class that requires that

Re: [Python-ideas] Membership of infinite iterators

On 10/15/2017 9:12 PM, Jason Campbell wrote: I recently came across a bug where checking negative membership (__contains__ returns False) of an infinite iterator will freeze the program. It may seem a bit obvious, but you can check membership in range for example without iterating over the e

[Python-ideas] Membership of infinite iterators

I recently came across a bug where checking negative membership (__contains__ returns False) of an infinite iterator will freeze the program. It may seem a bit obvious, but you can check membership in range for example without iterating over the entire range. `int(1e100) in range(int(1e101))`