On 29 Oct, 05:58 pm, tobias.oberst...@tavendo.de wrote: >Hi, > >I've got a question regarding > >deferToThread / runWithConnection > >I have a network server that accepts RPCs and forwards those to a >relational database calling stored procedures. > >The call of the stored procedures is happening via "runWithConnection" >.. that is on a background thread pool (since that DBs driver is >blocking). > >Now, when there is a large number of RPCs coming in roughly at once, >the latency of each call forwarded to the RDBMs increased into the 10's >of ms. > >I have created a self-contained test program without DB etc that I >think reproduces the "issue": > >https://github.com/oberstet/scratchbox/blob/master/python/twisted/threadpoollatency/test.py > >************** >Am I correct that "deferToThread" does not immediately forward the call >to a background thread, but only the next time the reactor loop runs? >**************
Hi Tobias, Sorry, I haven't had a chance to read your minimal example in any detail. However, I can direct you to the implementation of deferToThread: http://twistedmatrix.com/trac/browser/trunk/twisted/python/threadpool.py#L119 Notice the `self.q.put(o)`. This matches up with the call to `self.q.get` in the same module: http://twistedmatrix.com/trac/browser/trunk/twisted/python/threadpool.py#L158 Together, these bits of source should demonstrate that there's no waiting for a reactor iteration before the work is enqueued. The work goes into the Queue instance, and instantly any worker thread is free to grab it. Jean-Paul _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
