The current RPC code is a mess, with bizarre even for TS compiling and linking. It won't be really bidirectional even if traffic_manager is removed. It's fragile, it's extremely difficult to extend or modify. You have to basically write custom serialization and deserialization logic for every message type. On top of that the serialization is very fragile. As I noted above, my view is getting the serialization robust and flexible is almost all the work in writing an RPC. That's the part we should leverage. Moving bytes in and out of sockets is what we do well, therefore we should do that part. Bringing in something like Avro or grpc is a big step and will have a lot of issues with integration. I strongly believe this approach will play to our strengths, give us a nice flexible and powerful RPC, with the minimum amount of work and additional footprint, yielding something that fits in well with our existing architecture.
On Thu, Aug 16, 2018 at 9:07 PM Leif Hedstrom <zw...@apache.org> wrote: > Then what is the motivation to change anything? Let’s just keep the > serialization the same as what we have, Peach cleaned that up noticeably. > > The issue with bidirectional communication gets solved as part of the > elimination of traffic_manager. > > — Leif > > > On Aug 16, 2018, at 17:40, Alan Carroll <solidwallofc...@oath.com.INVALID> > wrote: > > > > Yeah, it's not like we have any expertise in asynchronous I/O on IPC > > sockets, that would let us leverage someone else's YAML based > serialization > > / deserialization library. We should definitely go with something that > has > > its own I/O mechanisms (which are sure to be compatible!) and quite > > possibly its own threading model too. That'll be *easy*! And I can't wait > > to play with all the super cool features and adjustable knobs such a > thing > > will have. > > > >> On Thu, Aug 16, 2018 at 6:17 PM Leif Hedstrom <zw...@apache.org> wrote: > >> > >> I think rolling our own defeats the purpose of replacing what we have. > We > >> should pick something where there is a large sets of client libraries > >> already. As you say, performance is not critical, so pick one that is > >> popular. I think gRPC would be fine too. > >> > >> — Leif > >> > >>> On Aug 16, 2018, at 11:59, Derek Dagit <der...@oath.com.INVALID> > wrote: > >>> > >>> I was not at Cork, so I am probably missing a lot of the details. > >>> > >>> It seems having true RPC here would greatly expand the possibilities > for > >>> management and monitoring. > >>> > >>> > >>> - What are the current and planned use cases for this RPC? > >>> > >>> - What if we want to add authentication/authorization for multi-tenant > >>> management and isolation? > >>> > >>> - What about security issues and the ongoing cost of maintaining a > unique > >>> RPC solution, versus using something already battle-tested? > >>> > >>> - Didn't we try this once already? [0] > >>> > >>> > >>> (I figure there are good answers to all of these questions, and so I am > >>> mostly asking for the benefit of those of us not familiar with the past > >>> discussions.) > >>> > >>> [0] > >> > https://github.com/apache/trafficserver/pull/3504#issuecomment-389927441 > >>> > >>> > >>> > >>> What are the current and planned use cases for this RPC? > >>> > >>> On Thu, Aug 16, 2018 at 2:39 PM, Alan M. Carroll < > >>> a...@network-geographics.com> wrote: > >>> > >>>> I've been looking at how to do the RPC replacement that was agreed on > at > >>>> Cork. After looking at grpc, Avro, and Thrift, I don't like any of > them > >> for > >>>> our use case. I've moved to the camp that says we should build our own > >> thin > >>>> wrapper around YAML messages, very similar to a REST style API. We do > >> not > >>>> need high performance for this case, the message rate is generally > less > >>>> than one per second and the messages are not large. Previously going > >> YAML > >>>> would have been a major effort but since we've already paid that price > >> in > >>>> the ongoing configuration conversion, building an RPC on top of > YAMLCPP > >>>> seems quite easy. In this scheme the data is sent via YAML map > >> instances. > >>>> In each map, the top level keys are the messages, each key being a tag > >> that > >>>> identifies the message handler. The handler is given the map entry, > key > >> and > >>>> value, and processes it. If parallelism is needed, the handlers can > >> attach > >>>> an "sequence" field inside the map value to match requests to > responses. > >>>> > >>>> The RPC driver is simple. It has a hash of tags to > >> handlers/continuations. > >>>> It accumulates data from the socket until it gets a successful YAML > >> parse. > >>>> Then it walks the top level keys, dispatching an event for each one to > >> the > >>>> handler associated with the tag. Sending is done handing a YAML map > >> object > >>>> to the RPC, which puts it in a queue to be written by a blocking write > >>>> thread. My view is this would be less work than integrating a full > >> featured > >>>> RPC with many features we don't need. > >>>> > >>>> > >>> > >>> > >>> -- > >>> Derek > >> > >> > > > > -- > > *Beware the fisherman who's casting out his line in to a dried up > riverbed.* > > *Oh don't try to tell him 'cause he won't believe. Throw some bread to > the > > ducks instead.* > > *It's easier that way. *- Genesis : Duke : VI 25-28 > > -- *Beware the fisherman who's casting out his line in to a dried up riverbed.* *Oh don't try to tell him 'cause he won't believe. Throw some bread to the ducks instead.* *It's easier that way. *- Genesis : Duke : VI 25-28
