Hi,
I'm pretty new to Python (2.6) and I've run into a problem I just
can't seem to solve.
I'm using dbfpy to access DBF tables as part of a little test project.
I've programmed two separate functions, one that reads the DBF in main
thread and the other which reads the DBF asynchronously in a separate
thread.
Here's the code:
def demo_01():
'''DBF read speed only'''
dbf1 = Dbf('D:\\python\\testdbf\\promet.dbf', readOnly=1)
for i1 in xrange(len(dbf1)):
rec = dbf1[i1]
dbf1.close()
def demo_03():
'''DBF read speed into a FIFO queue'''
class mt(threading.Thread):
q = Queue.Queue(64)
def run(self):
dbf1 = Dbf('D:\\python\\testdbf\\promet.dbf', readOnly=1)
for i1 in xrange(len(dbf1)):
self.q.put(dbf1[i1])
dbf1.close()
del dbf1
self.q.join()
t = mt()
t.start()
while t.isAlive():
try:
rec = t.q.get(False, 0.2)
t.q.task_done();
except:
pass
del t
However I'm having serious issues with the second method. It seems
that as soon as I start accessing the queue from both threads, the
reading speed effectively halves.
I have tried the following:
1. using deque instead of queue (same speed)
2. reading 10 records at a time and inserting them in a separate loop
(hoped the congestion would help)
3. Increasing queue size to infinite and waiting 10 seconds in main
thread before I started reading - this one yielded full reading speed,
but the waiting took away all the threading benefits
I'm sure I'm doing something very wrong here, I just can't figure out
what.
Can anyone help me with this?
Thanks,
Jure
--
http://mail.python.org/mailman/listinfo/python-list
