Hi
Using Python 2.4 I am trying to procduce a generator which will return
the results of dbi SQL statement using fetchmany for performance.
So instead of fetching one record for each call, I will fetch batches
of X (eg 100) and yeild each record in turn.
For reasons of pure asthetics and my own learning I wanted it to look
like this:
</B>
def resultsetbatchgen(cursor, size=100):
for results in (recordbatch for recordbatch in
cursor.fetchmany(size)):
for rec in results:
yield rec
</B>
Problem is this this gives spurious results. To understand the problem
if I excute the following in the console I get the correct results:
</B>
>>> cur.execute(<QUERY WITH MOER THAN 1000 records>)
>>> recs = (recordbatch for recordbatch in cur.fetchmany(1000))
>>> sum(map(lambda x: 1, (rec for rec in recs)))
1000
</B>
This is PERFECT! 1000 is what I expected. but now lets nest this inside
another for construct like so
</B>
>>> cur.execute(<QUERY WITH MOER THAN 1000 records>)
>>> for results in (recordbatch for recordbatch in
cur.fetchmany(1000)):
... print sum(map(lambda x: 1, (rec for rec in results)))
76
76
76
76
76
76
76
76
...........
</B>
Now it thinks each batch size is 76 ... ? This completly wrong, and
baffling.
The commands are exactly the same as far as I can tell, the only
difference is that it is now nested wihtin another for loop?
Any help would be greatly aprpeciated.
PS a working but far less elegant version of this below but I would
like to understand why teh above doesnt work and doesnt work
consitantly:
def resultsetbatch(cursor, size=100):
results = cursor.fetchmany(size)
while results <> []:
for rec in results:
yield rec
results = cursor.fetchmany(size)
--
http://mail.python.org/mailman/listinfo/python-list
