On Fri, Sep 29, 2023 at 1:01 PM Alex <capnpr...@centromere.net> wrote:
> Currently when an EOF occurs, there is no way to discern between an
> exceptional circumstance and a normal/expected circumstance.
>
That may be, but all we really want to decide here is whether to send an
abort message back to the peer. In the case of an EOF that was the result
of an exceptional situation, it's almost certainly the case that the peer
can no longer receive messages anyway, and therefore sending a message back
to them is pointless. It's only when the peer carefully shut down the
socket only in a single direction that it'll be able to receive a reply at
all -- and in that case, it's not an error situation.
So I think we can safely say: If we receive an EOF, we might as well send
an EOF.
(I kind of wish that if you closed a socket without doing shutdown(SHUT_WR)
first, the recv() on the other end would fail instead of signaling EOF, but
alas...)
> > To extend the protocol in this way we would have to think about
> > backwards compatibility. If a peer running an older version of capnp
> > receives the "goodbye" message, it will respond with an
> > "unimplemented" message, which seems like it could make things worse?
> >
>
> It's unclear to me how it would make things worse, since the connection
> is in the process of being shut down anyway. I am not saying it
> can't/wouldn't make things worse, I am only saying that it is not clear
> to me how that could be so.
>
Just in that it's sending *another* unwanted message back to a peer that
has already disconnected. But I suppose that's not that much worse compared
to the status quo.
> In the CapTP/E/Vat
> paradigm, is it valid for a single RPC system to form multiple
> independent connections to the same VatId?
That's an interesting question.
Ideally, no more than one connection is formed between any two vats, and
this is especially helpful if the application needs to be able to compare
capabilities for equality. But in practice I think this gets difficult to
ensure if vats cannot all directly address each other or cannot use
asymmetric cryptography to authenticate each other. I think it'll be hard
to answer this question definitively without a specific real-world system
to talk about.
VatNetwork::connect(vatId) is currently designed to return the existing
connection if there is one but I know of at least one real-world
implementation where it doesn't actually work that way.
> In other words, if I call:
>
> connA = VatNetwork::connect(vatIdA);
>
> followed by:
>
> connB = VatNetwork::connect(vatIdB);
>
> where vatIdA == vatIdB, should connA and connB refer to the same object
> in memory -- thus only ever creating a single RpcConnectionState? Or,
> should connA and connB instead be two independent objects in memory,
> each with their own independent underlying connection and thus,
> independently evolving RpcConnectionState?
>
> > I think, then, it has to be up to the RPC system to shut down
> > connections that are idle. Probably RpcSystem could signal to the
> > underlying VatNetwork whenever a connection has reached an idle
> > state, meaning it has no outstanding RPCs nor capabilities. The
> > VatNetwork could choose to close such a connection if it feels like
> > it -- some transports may want to do this on a timeout, others may
> > decide it's better to keep the connection open.
> >
>
> I have no strong opinion on this.
>
> > But I'd suggest not worrying about that for now and focusing just on
> > rpc-twoparty, since that's what most people are using today.
> >
>
> Indeed, a PR is forthcoming.
>
> > -Kenton
> >
>
> Alex
>
> >
> > > I considered modifying the
> > > signature of the BootstrapFactory's createFor method in this manner:
> > >
> > > Before:
> > >
> > > capnp::Capability::Client createFor(VatId::Reader clientId)
> > >
> > > After:
> > >
> > > capnp::Capability::Client createFor(VatId::Reader clientId,
> > > kj::Own<kj::PromiseFulfiller<void>> shutdown)
> > >
> > > The PromiseFulfiller can then be passed to the constructor of the
> > > Server:
> > >
> > > class AdderImpl final: public Adder::Server {
> > > public:
> > > AdderImpl(kj::Own<kj::PromiseFulfiller<void>> shutdown) :
> > > shutdown(kj::mv(shutdown)) {}
> > >
> > > kj::Promise<void> add(AddContext context) override {
> > > auto params = context.getParams();
> > > context.getResults().setValue(params.getLeft() +
> > > params.getRight()); return kj::READY_NOW;
> > > }
> > >
> > > kj::Promise<void> cleanupGracefully(CleanupGracefullyContext
> > > context) override {
> > > this->shutdown->fulfill();
> > > }
> > >
> > > private:
> > > kj::Own<kj::PromiseFulfiller<void>> shutdown;
> > > };
> > >
> > > Another approach could be to add a shutdown() method to FooContext.
> > >
> > > On the client side, perhaps it is best to simply allow the
> > > rpcSystem to fall out of scope, at which point the destructors can
> > > invoke the necessary machinery to send the GOODBYE and FIN the TCP
> > > stream.
> > >
> > > What do you think? Are these approaches going to lead to a leaky
> > > abstraction? Do you know of an elegant way to design this?
> > >
> > > Regards,
> > > Alex
> > >
> > > On Wed, 27 Sep 2023 14:07:07 -0500
> > > "'Kenton Varda' via Cap'n Proto" <capnproto@googlegroups.com> wrote:
> > >
> > > > (Happy to accept a PR. The relevant code is in `messageLoop()` and
> > > > `RpcConnectionState::disconnect()` in `rpc.c++`.)
> > > >
> > > > On Wed, Sep 27, 2023 at 2:05 PM Kenton Varda
> > > > <ken...@cloudflare.com> wrote:
> > > >
> > > > > Indeed, there isn't really a clean shutdown mechanism right
> > > > > now. I guess it hasn't come up as a priority because in most
> > > > > use cases we just haven't really cared if there's a TCP RST
> > > > > triggered under the hood... since we're already killing the
> > > > > connection, we ignore that error anyway.
> > > > >
> > > > > I suppose what we should do is, in the case that we receive a
> > > > > clean EOF, inhibit the sending of an abort message back, just
> > > > > send EOF back.
> > > > >
> > > > > -Kenton
> > > > >
> > > > > On Thu, Sep 21, 2023 at 6:09 PM 'Alex' via Cap'n Proto <
> > > > > capnproto@googlegroups.com> wrote:
> > > > >
> > > > >> Hi all,
> > > > >>
> > > > >> I am designing an application (in C++) where, upon invocation
> > > > >> of a particular RPC call, both the server and the client agree
> > > > >> to cleanly disconnect from one another. By "cleanly", I mean
> > > > >> that both the server and the client send a TCP FIN/ACK and
> > > > >> nothing more (e.g. no RSTs). Unfortunately, in the current
> > > > >> design the receipt of a FIN will cause
> > > > >> AsyncIoMessageStream::tryReadMessage() to abort, whereupon it
> > > > >> will throw KJ_EXCEPTION(DISCONNECTED, "Peer
> > > > >> disconnected.")[0]. This exception is eventually written to
> > > > >> the client socket, and if the client is already gone, there
> > > > >> will be one or more RSTs in response:
> > > > >>
> > > > >> C -> S: "Goodbye" (RPC call)
> > > > >> C -> S: "I have nothing more to say" (TCP FIN)
> > > > >>
> > > > >> (the client does not expect the server to say anything more and
> > > > >> closes the socket)
> > > > >>
> > > > >> S -> C: "Exception! You disconnected from me" (RPC message)
> > > > >> C -> S: "Error: Connection reset by peer" (TCP RST)
> > > > >>
> > > > >> Given that both the server and client have agreed to shut down
> > > > >> the connection, this is not an exceptional circumstance.
> > > > >> Therefore, an exception should not be thrown.
> > > > >>
> > > > >> Unfortunately, there does not seem to be a way to indicate to
> > > > >> the RpcSystem that the DISCONNECTED exception ought to be
> > > > >> suppressed. Is there something I am missing? I appreciate any
> > > > >> assistance.
> > > > >>
> > > > >> Regards,
> > > > >> Alex
> > > > >>
> > > > >> [0]
> > > > >>
> > >
> https://github.com/capnproto/capnproto/blob/761aeb17563a59f43b3fe9bae93df83c6bd57d06/c%2B%2B/src/capnp/rpc.c%2B%2B#L2775
> > >
> > > > >>
> > > > >> --
> > > > >> You received this message because you are subscribed to the
> > > > >> Google Groups "Cap'n Proto" group.
> > > > >> To unsubscribe from this group and stop receiving emails from
> > > > >> it, send an email to capnproto+unsubscr...@googlegroups.com.
> > > > >> To view this discussion on the web visit
> > > > >>
> > >
> https://groups.google.com/d/msgid/capnproto/20230921190853.115b911d%40centromere.net
> > >
> > > > >> .
> > > > >>
> > > > >
> > > >
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> > > Groups "Cap'n Proto" group.
> > > To unsubscribe from this group and stop receiving emails from it,
> > > send an email to capnproto+unsubscr...@googlegroups.com.
> > > To view this discussion on the web visit
> > >
> https://groups.google.com/d/msgid/capnproto/20230927173552.7ad4b38e%40centromere.net
> > > .
> > >
>
>
--
You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to capnproto+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/capnproto/CAJouXQ%3Du-8kq-9jahcgPw58Xa1BQy_paB%2Bzk9frUB8157Ap7Eg%40mail.gmail.com.
