Just wanted to close this thread off: I think I have what I need to unblock Go RPC improvements. My ramblings on implementation at the end didn't make much sense and were more complicated than what's needed. Don't mind me. :)
Time permitting, I'll try to collect my observations about backpressure in Cap'n Proto in some sort of sensible documentation. Perhaps this would be a good candidate for some of the non-normative docs of the RPC spec. I agree that being able to apply backpressure to a single capability without blocking the whole connection would be a boon. One thing I'm currently curious about in the C++ implementation: does the RPC system provide any backpressure for sending calls to the remote vat? AFAICT there's no bound on the EventLoop queue. On Wed, Jul 26, 2017 at 10:38 AM Kenton Varda <[email protected]> wrote: > On Wed, Jul 26, 2017 at 9:16 AM, Ross Light <[email protected]> wrote: >> >> Cap'n Proto extends the model by making request/response pairings >>> explicit, but it doesn't require that a response be sent before a new >>> request arrives. >>> >> >> Good point; I'm not arguing for that restriction. I'm fine with this >> sequence (which conceptually only requires one actor): >> >> 1. Alice sends Bob foo1() >> 2. Bob starts working on foo1() >> 3. Alice sends Bob foo2(). Bob queues it. >> 4. Alice sends Bob foo3(). Bob queues it. >> 5. Bob finishes foo1() and returns foo1()'s response to Alice >> 6. Bob starts working on foo2() >> 7. Bob finishes foo2() and returns foo2()'s response to Alice >> 8. Bob starts working on foo3() >> 9. Bob finishes foo3() and returns foo3()'s response to Alice >> > > In this example, you're saying Bob can't start working on a new request > until after sending a response for the last request. That's what I'm saying > is *not* a constraint imposed by Cap'n Proto. > > >> Here's the harder sequence (which IIUC, C++ permits. *If it doesn't*, >> then it simplifies everything.): >> >> 1. Alice sends Bob foo1() >> 2. Bob starts working on foo1(). It's going to do something that will >> take a long time (read as: requires a future), so it acknowledges delivery >> and keeps going. Bob now has has multiple conceptual actors for the same >> capability, although I can see how this can be also be thought of as a >> single actor receiving request messages and sending response messages. >> 3. Alice sends Bob foo2() >> 4. Bob starts working on foo2(). >> 5. foo2() is short, so Bob returns a result to Alice. >> 6. foo1()'s long task completes. Bob returns foo1()'s result to Alice. >> > > This does not create "multiple conceptual actors". I think you may be > mixing up actors with threads. The difference between a (conceptual) thread > and an (conceptual) actor is that a thread follows a call stack (possibly > crossing objects) while an actor follows an object (sending asynchronous > messages to other objects). > > In step 2, when Bob initiates "something that will take a long time", in > your threaded approach in Go, he makes a blocking call of some sort. But in > the actor model, blocking calls aren't allowed. Bob would initiate a > long-running operation by sending a message. When the operation completes, > a message is sent back to Bob with the results. In between these messages, > Bob is free to process other messages. The important thing is that only one > message handler is executing in Bob at a time, therefore Bob's state does > not need to be protected by a mutex. However, message handlers cannot block > -- they always complete immediately. > > Concretely speaking, in C++, the implementation of Bob.foo() will call > some other function that returns a promise, and then foo() will return a > promise chained off of it. As soon as foo() returns that promise, then a > new method on Bob can be invoked immediately, without waiting for the > returned promise to resolve. > > This of course suffers from the "function coloring problem" you referenced > earlier. All Cap'n Proto methods are colored red (asynchronous). > > I think what the function coloring analogy misses, though, is that > permitting functions to block doesn't really avoid the function-coloring > problem, it only sweeps the problem under the rug. Even in a multi-threaded > program, it is incredibly important to know which functions might block. > Because, in a multi-threaded program, you almost certainly don't want to > call a blocking functions while holding a mutex lock. If you do, you risk > blocking not only your own thread, but all other threads that might need to > take that lock. And in the case of bidirectional communication, you risk > deadlock. > > This is, I think, exactly the problem I think you're running into here. > > Alternatively, what if making a new call implicitly acknowledged the >>> current call? This avoids cognitive overhead and probably produces the >>> desired behavior? >>> >> >> I don't think this is a good idea, since it seems common to want to start >> off a call (or multiple) before acknowledging delivery. >> > > I guess I meant: *Waiting* on results of a sub call should implicitly > acknowledge the super call / unblock concurrent calls. So you could *start* > multiple sub calls while still being protected from concurrent calls, but > as soon as you *wait* on one, you're no longer protected. > > >> I thought about this a bit more over the last couple of days and I >> think I have a way out (finally). Right now, operating on the connection >> acquires a mutex. I think I need to extend this to be a mutex+condition, >> where the condition is for is-connection-making-call. When the connection >> makes a call, it marks the is-connection-making-call bit, then plumbs the >> is-in-a-connection-call info through the Context (think of as thread-local >> storage, except explicit). When the connection acquires the mutex, >> non-send-RPC operations will block on the is-connection-making-call bit to >> be cleared and send-RPC operations will not block. I've examined the >> send-RPC path and that operation ought to be safe to be called. This would >> avoid the nasty queue idea that I had. >> > > Sorry, I don't follow this. > > -Kenton > -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. Visit this group at https://groups.google.com/group/capnproto.
