Re: Nested loops is strangely slow, totally at a loss.

2014-12-10 Thread Ian Kelly
On Wed, Dec 10, 2014 at 9:01 AM, Ian Kelly wrote: > This also seems perfectly natural: > > def len(iterable): > return sum(1 for item in iterable) > > My observation is that seems strange to me that one standard sequence operation should be supported for arbitrary iterators and the other not.

Re: Nested loops is strangely slow, totally at a loss.

2014-12-10 Thread Ian Kelly
On Wed, Dec 10, 2014 at 1:21 AM, Peter Otten <__pete...@web.de> wrote: > > Ian Kelly wrote: > > Huh, I wasn't even aware that membership tests worked on iterables with no > > __contains__ method. Seems odd to me that 'x in y' should be supported but > > not 'len(y)'. > > To me > > def contains(iter

Re: Nested loops is strangely slow, totally at a loss.

2014-12-10 Thread Steven D'Aprano
Shiyao Ma wrote: > Thanks guys. > > I was only aware of a limited iterables which themselves are iterators, > e.g., the generator. > > Seems like its really a pitfall. Any glossary, list on the iterables that > *might* exhaust themselves? Iterables include: - iterators - sequences (e.g. lists,

Re: Nested loops is strangely slow, totally at a loss.

2014-12-10 Thread Peter Otten
Shiyao Ma wrote: > Thanks guys. > > I was only aware of a limited iterables which themselves are iterators, > e.g., the generator. > > Seems like its really a pitfall. Any glossary, list on the iterables that > *might* exhaust themselves? Usually the test iterable is iter(iterable) returns Tr

Re: Nested loops is strangely slow, totally at a loss.

2014-12-10 Thread Shiyao Ma
Thanks guys. I was only aware of a limited iterables which themselves are iterators, e.g., the generator. Seems like its really a pitfall. Any glossary, list on the iterables that *might* exhaust themselves? Regards. -- https://mail.python.org/mailman/listinfo/python-list

Re: Nested loops is strangely slow, totally at a loss.

2014-12-10 Thread Peter Otten
Ian Kelly wrote: > On Tue, Dec 9, 2014 at 11:30 PM, Chris Angelico wrote: >> Are you sure it isn't? Your 'space' is an iterable cubic >> cross-product. Your first loop checks (0,0,0) which is the first >> element returned, and is thus fast... but it also *consumes* that >> first element. The next

Re: Nested loops is strangely slow, totally at a loss.

2014-12-09 Thread Steven D'Aprano
On Wed, 10 Dec 2014 17:53:05 +1100, Chris Angelico wrote: > On Wed, Dec 10, 2014 at 5:44 PM, Steven D'Aprano > wrote: >> It would be nice if product iterators behaved like xrange() objects and >> could perform "in" tests without exhausting the iterator, but they >> don't. That's sad. > > It'd be

Re: Nested loops is strangely slow, totally at a loss.

2014-12-09 Thread Terry Reedy
On 12/10/2014 1:53 AM, Chris Angelico wrote: On Wed, Dec 10, 2014 at 5:44 PM, Steven D'Aprano wrote: It would be nice if product iterators behaved like xrange() objects and could perform "in" tests without exhausting the iterator, but they don't. That's sad. It'd be very difficult to do that

Re: Nested loops is strangely slow, totally at a loss.

2014-12-09 Thread Ian Kelly
On Tue, Dec 9, 2014 at 11:30 PM, Chris Angelico wrote: > Are you sure it isn't? Your 'space' is an iterable cubic > cross-product. Your first loop checks (0,0,0) which is the first > element returned, and is thus fast... but it also *consumes* that > first element. The next time you test it, the e

Re: Nested loops is strangely slow, totally at a loss.

2014-12-09 Thread Chris Angelico
On Wed, Dec 10, 2014 at 5:44 PM, Steven D'Aprano wrote: > It would be nice if product iterators behaved like xrange() objects and > could perform "in" tests without exhausting the iterator, but they don't. > That's sad. It'd be very difficult to do that in the general sense. But it should be poss

Re: Nested loops is strangely slow, totally at a loss.

2014-12-09 Thread Steven D'Aprano
On Wed, 10 Dec 2014 13:20:25 +0800, Shiyao Ma wrote: > When doing nested loop, the very first iteration of the innermost loop > ends ultimately slow. > > Let's the code speak. > > The following code is quite contrived. Actually it's derived from my > 3d-dct script. The actual difference is way m

Re: Nested loops is strangely slow, totally at a loss.

2014-12-09 Thread Chris Angelico
On Wed, Dec 10, 2014 at 4:20 PM, Shiyao Ma wrote: > from itertools import product > space_len = 580 > space = product(xrange(space_len), xrange(space_len), xrange(space_len)) > > sparse_cloud = product(xrange(1000), xrange(1000), xrange(1000)) > for i, j, k in sparse_cloud: > ts = timeit.defau

Re: Nested loops is strangely slow, totally at a loss.

2014-12-09 Thread Shiyao Ma
One thing to note, the logic of using "in" is not of concern here. This is a *contrived* example, the problem is the slowness of the first iteration. -- https://mail.python.org/mailman/listinfo/python-list

Nested loops is strangely slow, totally at a loss.

2014-12-09 Thread Shiyao Ma
When doing nested loop, the very first iteration of the innermost loop ends ultimately slow. Let's the code speak. The following code is quite contrived. Actually it's derived from my 3d-dct script. The actual difference is way more significant than this example. In case of any evil of gmail,