Hi Philip, multiprocessing.Queue is used to transfer data between processes, how it could be helpful for solving my problem? Thanks!
Sheng On Mar 8, 6:34 pm, Philip Semanchuk <phi...@semanchuk.com> wrote: > On Mar 8, 2011, at 3:25 PM, Sheng wrote: > > > This looks like a tornado problem, but trust me, it is almost all > > about the mechanism of multiprocessing module. > > [snip] > > > So the workflow is like this, > > > get() --> fork a subprocess to process the query request in > > async_func() -> when async_func() returns, callback_func uses the > > return result of async_func as the input argument, and send the query > > result to the client. > > > So the problem is the the query result as the result of sql_command > > might be too big to store them all in the memory, which in our case is > > stored in the variable "data". Can I send return from the async method > > early, say immediately after the query returns with the first result > > set, then stream the results to the browser. In other words, can > > async_func somehow notify callback_func to prepare receiving the data > > before async_func actually returns? > > Hi Sheng, > Have you looked at multiprocessing.Queue objects? > > HTH > Philip -- http://mail.python.org/mailman/listinfo/python-list
