On Thu, 2007-09-20 at 18:55 +0000, John J. Lee wrote:
> Functions are never generators, senso stricto. There are "generator
> functions", which *return* (or yield) generators when you call them.
Actually, a generator function is a function that returns a generator.
The generator, in turn, is an object that wraps the iterator protocol
around a resumable function. When the generator's next() method is
called, it resumes its wrapped function that runs until it yields or
finishes. When the wrapped function yields a value, it is suspended, and
the yielded value is returned as the result of next(). If the wrapped
function finished, next() raises StopIteration.
Generators are a special case of iterators. Any object that implements
the iterator protocol (which among other less important things mandates
a next() method that returns the next value or raises StopIteration) is
an iterator. Generators simply have their special way (by calling into a
resumable function) of implementing the iterator protocol, but any
stateful object that returns a value or raises StopIteration in next()
is an iterator.
> It's true sometimes people refer to generator functions as simply
> "generators", but in examples like the above, it's useful to remember
> that they are two different things.
>
> In this case, frange isn't a generator function, because it doesn't
> yield. Instead, it returns the result of evaluating a generator
> expression (a generator). The generatator expression plays the same
> role as a generator function -- calling a generator function gives you
> a generator object; evaluating a generator expression gives you a
> generator object. There's nothing to stop you returning that
> generator object, which makes this function behave just like a regular
> generator function.
Generator expressions still "yield", but you don't see the yield in the
Python source code. Observe:
>>> def gf1():
... for i in (1,2,3):
... yield i
...
>>> def gf2():
... return (i for i in (1,2,3))
...
>>> g1 = gf1()
>>> g2 = gf2()
>>> g1
<generator object at 0xb7f66d8c>
>>> g2
<generator object at 0xb7f66f4c>
>>> dir(g1)
['__class__', '__delattr__', '__doc__', '__getattribute__', '__hash__',
'__init__', '__iter__', '__new__', '__reduce__', '__reduce_ex__',
'__repr__', '__setattr__', '__str__', 'close', 'gi_frame', 'gi_running',
'next', 'send', 'throw']
>>> dir(g2)
['__class__', '__delattr__', '__doc__', '__getattribute__', '__hash__',
'__init__', '__iter__', '__new__', '__reduce__', '__reduce_ex__',
'__repr__', '__setattr__', '__str__', 'close', 'gi_frame', 'gi_running',
'next', 'send', 'throw']
>>> import dis
>>> dis.dis(g1.gi_frame.f_code.co_code)
0 SETUP_LOOP 19 (to 22)
3 LOAD_CONST 4 (4)
6 GET_ITER
>> 7 FOR_ITER 11 (to 21)
10 STORE_FAST 0 (0)
13 LOAD_FAST 0 (0)
16 YIELD_VALUE
17 POP_TOP
18 JUMP_ABSOLUTE 7
>> 21 POP_BLOCK
>> 22 LOAD_CONST 0 (0)
25 RETURN_VALUE
>>> dis.dis(g2.gi_frame.f_code.co_code)
0 SETUP_LOOP 18 (to 21)
3 LOAD_FAST 0 (0)
>> 6 FOR_ITER 11 (to 20)
9 STORE_FAST 1 (1)
12 LOAD_FAST 1 (1)
15 YIELD_VALUE
16 POP_TOP
17 JUMP_ABSOLUTE 6
>> 20 POP_BLOCK
>> 21 LOAD_CONST 0 (0)
24 RETURN_VALUE
As you can see, except for a minor optimization in the byte code, there
is no discernible difference between the generator that's returned from
a generator expression and the generator that's returned from the
generator function. Note in particular that the byte code for g2
contains a YIELD_VALUE operation just like g1 does.
In fact, generator expressions are merely a short-hand notation for
simple generator functions of a particular form. The generator
expression
g = (expr(x) for x in some_iterable)
produces a generator that is almost indistinguishable from, and
operationally identical to, the generator produced by the following
code:
def __anonymous():
for x in some_iterable:
yield expr(x)
g = __anonymous()
del __anonymous
Hope this helps,
--
Carsten Haese
http://informixdb.sourceforge.net
--
http://mail.python.org/mailman/listinfo/python-list
