I was describing a (much worse) solution where each request calls a capnproto RPC directly and calls `.wait()` to make sure the event loop is executed. This of course blocks the whole Python process (or thread), so you would have to ensure each new request gets serviced on a new process/thread. Thankfully, there's lots of wsgi runners that let you setup a potentially large number of workers (see https://uwsgi-docs.readthedocs.io/en/latest/Options.html?highlight=worker#workers ).
That being said, it sounds like you found a better solution, so don't worry about it. On Thu, Nov 10, 2016 at 7:26 AM, Andreas Stenius <[email protected]> wrote: > Phew. I got an initial poc working, where I have a client connect, and > register a handler, that gets called from the server end in another thread, > and have the result routed all the way back from the client. The "only" > loop to work around the event loop vs thread issue is that I have an extra > connection on the server between the capnp RPC server and the flask app. > > Looks promising. Had some initial hurdles to get over how to pass the > results back from the interface methods. > > > On Thu, Nov 10, 2016 at 11:06 AM Andreas Stenius <[email protected]> wrote: > >> Could you elaborate a little bit on how it would work with the request >> blocking on the capnp event loop. I'm not entirely sure I get how you see >> that working. >> >> Estimates are evil, by nature, I know ;) >> But it gives me a feel for what you _think_, and that say's a lot too.. >> regardless of how true it will eventually turn out to be. >> >> As it turns out, I took a slightly different approach to try out, where I >> have the flask process initiate it's own rpc client to the server (living >> in the same process), only using another interface to dispatch the events. >> That way, I use the network stack to communicate between the threads in a >> safe manner on the server, and without the hassle of exposing rpc servers >> on the clients end :) >> >> Thanks. >> >> >> On Thu, Nov 10, 2016 at 9:57 AM Jason Paryani <[email protected]> >> wrote: >> >> Another solution might be to use a multithreaded/multiprocess web server >> where each request is serviced by a single thread or process. That way, it >> wouldn't be a problem to block the request waiting on the capnp event loop. >> >> The event loop stuff is probably a few weeks worth of work, but I'm not >> super confident in that estimate :) >> >> >> On Nov 9, 2016, 11:45 PM -0800, Andreas Stenius <[email protected]>, wrote: >> >> Dang. Ok, thanks for the quick reply, Jason. >> >> I'm still reluctant to let capnp go, in favor of something else here, and >> it's use is going to be in-house only, so performance won't be a major >> issue. I'm thinking that it could work if instead of having the "server" >> end get a object from the client to call, it could make a new RPC >> connection to the client from the server instead, in a separate thread. >> That way, each server to client communication path would have it's own >> event loop, separate from the main server RPC event loop. Then each >> connection to the clients can use some python thread synchronization and >> call out to the RPC on events. Feels kind of a bloaty work-around, but I'm >> willing to give it a try. >> >> How much work would it be to implement the event loop integration you're >> suggesting ? I'm not that familiar with the low level python going into >> pycapnp, but I'm fairly fluent in C (and know some C++ but haven't used it >> as much), just thinking that if it's not a 6 month dev investment, I might >> be able to find some time to look into it, with some initial guidance as >> were it would fit. >> >> Cheers >> >> >> On Wed, Nov 9, 2016 at 9:09 PM Jason Paryani <[email protected]> >> wrote: >> >> Fulfilling a promise from another thread is *NOT* safe (see >> https://groups.google.com/forum/#!topic/capnproto/zSmTdNGdWg8). I'm not >> sure how to accomplish what you're trying to do with pycapnp today, sorry. >> >> Ideally, pycapnp would grow the ability to integrate with other event >> loops, but I'm not sure that's coming anytime soon. >> >> On Wed, Nov 9, 2016 at 3:26 AM, Andreas Stenius <[email protected]> wrote: >> >> Hi, >> >> I'm implementing a RPC server, which should be able to call a function on >> a client's object. But, I want to be able to initiate the call from the >> server outside of the event loop (or trigger something that will result in >> the call being made). >> >> The examples in pycapnp regarding threads and the event loop used the >> getTimer() to delay a call, and thinking along those lines, I tried to see >> if I simply could create my own promise, and fullfill that from another >> thread at a later time, but haven't figured out how to do it, if at all >> possible. >> >> To give a little more context, what I have is a flask app, and a rpc >> server, living in the same process. And I want to call a function on a >> client provided object over capnp-rpc from the flask app thread. >> >> Any hints/pointers ideas welcome :) >> >> Thanks, >> Andreas >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Cap'n Proto" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> Visit this group at https://groups.google.com/group/capnproto. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Cap'n Proto" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> Visit this group at https://groups.google.com/group/capnproto. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Cap'n Proto" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> Visit this group at https://groups.google.com/group/capnproto. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Cap'n Proto" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> Visit this group at https://groups.google.com/group/capnproto. >> >> -- > You received this message because you are subscribed to the Google Groups > "Cap'n Proto" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > Visit this group at https://groups.google.com/group/capnproto. > -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. Visit this group at https://groups.google.com/group/capnproto.
