On Dec 12, 5:56 pm, Jason Grout <jason-s...@creativetrax.com> wrote:
> There's apparently a lot more overhead than I would have imagined:
>
> sage: %timeit [v for v in (x for x in range(10) ) if v > 0]
> 100000 loops, best of 3: 5.4 s per loop
> sage: %timeit [v for v in range(10) if v>0]
> 100000 loops, best of 3: 2.98 s per loop
I don't find that timing surprising. Hopefully, the main cost in these
constructs is the actual call to the "next" method on the iterator. In
the first example, the outer iterator has to call an iterator itself,
so you end up with 20 iterator calls rather than 10. So roughly twice
as slow seems about right.
> Interestingly, the builtin filter function isn't much faster than the
> slow version above:
>
> sage: %timeit filter(lambda x: x>0, range(10))
> 100000 loops, best of 3: 5.3 s per loop
I guess a call to a "lambda" closure incurs about the same cost as an
iterator invocation? In that case, the "double iterator" construction
is indeed not so bad for the original construction, because the
overhead of the second iterator is virtually cancelled by the two
calls to f already. Indeed:
def f(x):
return x
sage: %timeit [v for v in (f(x) for x in range(10)) if v > 0]
100000 loops, best of 3: 4.94 µs per loop
sage: %timeit [f(x) for x in range(10) if f(x) > 0]
100000 loops, best of 3: 4.34 µs per loop
As soon as f has a real cost and the "if" clause is reasonably often
true, the double iterator will be faster. In fact, a bit longer range
seems to tip the scale already:
sage: %timeit [f(x) for x in range(1000) if f(x) > 0]
1000 loops, best of 3: 387 µs per loop
sage: %timeit [v for v in (f(x) for x in range(1000)) if v > 0]
1000 loops, best of 3: 345 µs per loop
I guess this establishes that the nested iterator construction is
actually quite efficient in python. The overhead seems to be on the
order of a function call, which seems a reasonable price. I don't like
the syntax, because it does not read like English at all, but at least
it is unambiguous.
--
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org
