The C++ RPC implementation has one, limited, form of backpressure created specifically for Sandstorm sandboxing purposes: setFlowLimit().
https://github.com/capnproto/capnproto/blob/master/c++/src/capnp/rpc.h#L115 This simple approach works well enough to prevent buggy Sandstorm apps from filling up the front-end's memory. It can theoretically lead to deadlock, though, in the case where a recursive call bounces back and forth enough times to fill the limit, then gets stuck waiting. -Kenton On Fri, Sep 1, 2017 at 9:26 AM, Ross Light <[email protected]> wrote: > Just wanted to close this thread off: I think I have what I need to > unblock Go RPC improvements. My ramblings on implementation at the end > didn't make much sense and were more complicated than what's needed. Don't > mind me. :) > > Time permitting, I'll try to collect my observations about backpressure in > Cap'n Proto in some sort of sensible documentation. Perhaps this would be > a good candidate for some of the non-normative docs of the RPC spec. I > agree that being able to apply backpressure to a single capability without > blocking the whole connection would be a boon. > > One thing I'm currently curious about in the C++ implementation: does the > RPC system provide any backpressure for sending calls to the remote vat? > AFAICT there's no bound on the EventLoop queue. > > On Wed, Jul 26, 2017 at 10:38 AM Kenton Varda <[email protected]> > wrote: > >> On Wed, Jul 26, 2017 at 9:16 AM, Ross Light <[email protected]> wrote: >>> >>> Cap'n Proto extends the model by making request/response pairings >>>> explicit, but it doesn't require that a response be sent before a new >>>> request arrives. >>>> >>> >>> Good point; I'm not arguing for that restriction. I'm fine with this >>> sequence (which conceptually only requires one actor): >>> >>> 1. Alice sends Bob foo1() >>> 2. Bob starts working on foo1() >>> 3. Alice sends Bob foo2(). Bob queues it. >>> 4. Alice sends Bob foo3(). Bob queues it. >>> 5. Bob finishes foo1() and returns foo1()'s response to Alice >>> 6. Bob starts working on foo2() >>> 7. Bob finishes foo2() and returns foo2()'s response to Alice >>> 8. Bob starts working on foo3() >>> 9. Bob finishes foo3() and returns foo3()'s response to Alice >>> >> >> In this example, you're saying Bob can't start working on a new request >> until after sending a response for the last request. That's what I'm saying >> is *not* a constraint imposed by Cap'n Proto. >> >> >>> Here's the harder sequence (which IIUC, C++ permits. *If it doesn't*, >>> then it simplifies everything.): >>> >>> 1. Alice sends Bob foo1() >>> 2. Bob starts working on foo1(). It's going to do something that will >>> take a long time (read as: requires a future), so it acknowledges delivery >>> and keeps going. Bob now has has multiple conceptual actors for the same >>> capability, although I can see how this can be also be thought of as a >>> single actor receiving request messages and sending response messages. >>> 3. Alice sends Bob foo2() >>> 4. Bob starts working on foo2(). >>> 5. foo2() is short, so Bob returns a result to Alice. >>> 6. foo1()'s long task completes. Bob returns foo1()'s result to Alice. >>> >> >> This does not create "multiple conceptual actors". I think you may be >> mixing up actors with threads. The difference between a (conceptual) thread >> and an (conceptual) actor is that a thread follows a call stack (possibly >> crossing objects) while an actor follows an object (sending asynchronous >> messages to other objects). >> >> In step 2, when Bob initiates "something that will take a long time", in >> your threaded approach in Go, he makes a blocking call of some sort. But in >> the actor model, blocking calls aren't allowed. Bob would initiate a >> long-running operation by sending a message. When the operation completes, >> a message is sent back to Bob with the results. In between these messages, >> Bob is free to process other messages. The important thing is that only one >> message handler is executing in Bob at a time, therefore Bob's state does >> not need to be protected by a mutex. However, message handlers cannot block >> -- they always complete immediately. >> >> Concretely speaking, in C++, the implementation of Bob.foo() will call >> some other function that returns a promise, and then foo() will return a >> promise chained off of it. As soon as foo() returns that promise, then a >> new method on Bob can be invoked immediately, without waiting for the >> returned promise to resolve. >> >> This of course suffers from the "function coloring problem" you >> referenced earlier. All Cap'n Proto methods are colored red (asynchronous). >> >> I think what the function coloring analogy misses, though, is that >> permitting functions to block doesn't really avoid the function-coloring >> problem, it only sweeps the problem under the rug. Even in a multi-threaded >> program, it is incredibly important to know which functions might block. >> Because, in a multi-threaded program, you almost certainly don't want to >> call a blocking functions while holding a mutex lock. If you do, you risk >> blocking not only your own thread, but all other threads that might need to >> take that lock. And in the case of bidirectional communication, you risk >> deadlock. >> >> This is, I think, exactly the problem I think you're running into here. >> >> Alternatively, what if making a new call implicitly acknowledged the >>>> current call? This avoids cognitive overhead and probably produces the >>>> desired behavior? >>>> >>> >>> I don't think this is a good idea, since it seems common to want to >>> start off a call (or multiple) before acknowledging delivery. >>> >> >> I guess I meant: *Waiting* on results of a sub call should implicitly >> acknowledge the super call / unblock concurrent calls. So you could *start* >> multiple sub calls while still being protected from concurrent calls, but >> as soon as you *wait* on one, you're no longer protected. >> >> >>> I thought about this a bit more over the last couple of days and I >>> think I have a way out (finally). Right now, operating on the connection >>> acquires a mutex. I think I need to extend this to be a mutex+condition, >>> where the condition is for is-connection-making-call. When the connection >>> makes a call, it marks the is-connection-making-call bit, then plumbs the >>> is-in-a-connection-call info through the Context (think of as thread-local >>> storage, except explicit). When the connection acquires the mutex, >>> non-send-RPC operations will block on the is-connection-making-call bit to >>> be cleared and send-RPC operations will not block. I've examined the >>> send-RPC path and that operation ought to be safe to be called. This would >>> avoid the nasty queue idea that I had. >>> >> >> Sorry, I don't follow this. >> >> -Kenton >> > -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. Visit this group at https://groups.google.com/group/capnproto.
