> On Aug 17, 2018, at 06:46, Alan Carroll <solidwallofc...@oath.com.INVALID>
> wrote:
>
> 1) Yes. Two counter points - right now we're not reducing anything because
> no one is willing to actually bring in the third party RPC. This proposal
> would IMHO be definitely better than the current one. Second, bringing in
> complete third party RPC will require glue logic to adapt to our
All other things said, this is not accurate. There are definitely suggestions
on the table to bring in an existing serialization protocol (and willing
people). There are plenty well supported libraries, even CBOR was one of the
suggestions. I’m fine piggy backing such libraries on our own transport (say
HTTP), but why invent our own marshaling again?
Cheers,
— Leif
> infrastructure. My opinion is this proposal will be less work and
> maintenance than that. This is of course speculative because the prior
> point means we don't have a basis for comparison. The best I can say is
> that I have spent time looking at what it would take and that lead me to
> this proposal.
>
> 2) RPC communication by YAML or JSON encoded messages is not some brand new
> idea of mine. It's actually a common style, sometimes referred to as a
> "REST API"[1]. This proposal would make any implementation language that
> supports YAML easily able to communicate. I suspect YAML support is more
> widespread than gRPC or Avro support.
>
> [1] https://en.wikipedia.org/wiki/Representational_state_transfer
>
> On Fri, Aug 17, 2018 at 9:12 AM Aaron Canary <acan...@oath.com.invalid>
> wrote:
>
>> 1. I thought the goal of replacing RPC was to reduce code maintenance. Does
>> writing our own do that? How does that effect the other apps we are
>> communicating with?
>>
>> 2. We are discussing using a 3rd party health monitor app which is likely
>> to have frequent communication with ATS. I.e. ATS pushes request when
>> traffic is dropped, HealthMon pushes when origin looks healthy again.
>> (Simplification) this could be RPC or something else.
>>
>> On Aug 16, 2018 9:53 PM, "Alan Carroll" <solidwallofc...@oath.com.invalid>
>> wrote:
>>
>> The current RPC code is a mess, with bizarre even for TS compiling and
>> linking. It won't be really bidirectional even if traffic_manager is
>> removed. It's fragile, it's extremely difficult to extend or modify. You
>> have to basically write custom serialization and deserialization logic for
>> every message type. On top of that the serialization is very fragile. As I
>> noted above, my view is getting the serialization robust and flexible is
>> almost all the work in writing an RPC. That's the part we should leverage.
>> Moving bytes in and out of sockets is what we do well, therefore we should
>> do that part. Bringing in something like Avro or grpc is a big step and
>> will have a lot of issues with integration. I strongly believe this
>> approach will play to our strengths, give us a nice flexible and powerful
>> RPC, with the minimum amount of work and additional footprint, yielding
>> something that fits in well with our existing architecture.
>>
>>
>>> On Thu, Aug 16, 2018 at 9:07 PM Leif Hedstrom <zw...@apache.org> wrote:
>>>
>>> Then what is the motivation to change anything? Let’s just keep the
>>> serialization the same as what we have, Peach cleaned that up noticeably.
>>>
>>> The issue with bidirectional communication gets solved as part of the
>>> elimination of traffic_manager.
>>>
>>> — Leif
>>>
>>>> On Aug 16, 2018, at 17:40, Alan Carroll <solidwallofc...@oath.com
>> .INVALID>
>>> wrote:
>>>>
>>>> Yeah, it's not like we have any expertise in asynchronous I/O on IPC
>>>> sockets, that would let us leverage someone else's YAML based
>>> serialization
>>>> / deserialization library. We should definitely go with something that
>>> has
>>>> its own I/O mechanisms (which are sure to be compatible!) and quite
>>>> possibly its own threading model too. That'll be *easy*! And I can't
>> wait
>>>> to play with all the super cool features and adjustable knobs such a
>>> thing
>>>> will have.
>>>>
>>>>> On Thu, Aug 16, 2018 at 6:17 PM Leif Hedstrom <zw...@apache.org>
>> wrote:
>>>>>
>>>>> I think rolling our own defeats the purpose of replacing what we have.
>>> We
>>>>> should pick something where there is a large sets of client libraries
>>>>> already. As you say, performance is not critical, so pick one that is
>>>>> popular. I think gRPC would be fine too.
>>>>>
>>>>> — Leif
>>>>>
>>>>>> On Aug 16, 2018, at 11:59, Derek Dagit <der...@oath.com.INVALID>
>>> wrote:
>>>>>>
>>>>>> I was not at Cork, so I am probably missing a lot of the details.
>>>>>>
>>>>>> It seems having true RPC here would greatly expand the possibilities
>>> for
>>>>>> management and monitoring.
>>>>>>
>>>>>>
>>>>>> - What are the current and planned use cases for this RPC?
>>>>>>
>>>>>> - What if we want to add authentication/authorization for
>> multi-tenant
>>>>>> management and isolation?
>>>>>>
>>>>>> - What about security issues and the ongoing cost of maintaining a
>>> unique
>>>>>> RPC solution, versus using something already battle-tested?
>>>>>>
>>>>>> - Didn't we try this once already? [0]
>>>>>>
>>>>>>
>>>>>> (I figure there are good answers to all of these questions, and so I
>> am
>>>>>> mostly asking for the benefit of those of us not familiar with the
>> past
>>>>>> discussions.)
>>>>>>
>>>>>> [0]
>>>>>
>>> https://github.com/apache/trafficserver/pull/3504#issuecomment-389927441
>>>>>>
>>>>>>
>>>>>>
>>>>>> What are the current and planned use cases for this RPC?
>>>>>>
>>>>>> On Thu, Aug 16, 2018 at 2:39 PM, Alan M. Carroll <
>>>>>> a...@network-geographics.com> wrote:
>>>>>>
>>>>>>> I've been looking at how to do the RPC replacement that was agreed
>> on
>>> at
>>>>>>> Cork. After looking at grpc, Avro, and Thrift, I don't like any of
>>> them
>>>>> for
>>>>>>> our use case. I've moved to the camp that says we should build our
>> own
>>>>> thin
>>>>>>> wrapper around YAML messages, very similar to a REST style API. We
>> do
>>>>> not
>>>>>>> need high performance for this case, the message rate is generally
>>> less
>>>>>>> than one per second and the messages are not large. Previously going
>>>>> YAML
>>>>>>> would have been a major effort but since we've already paid that
>> price
>>>>> in
>>>>>>> the ongoing configuration conversion, building an RPC on top of
>>> YAMLCPP
>>>>>>> seems quite easy. In this scheme the data is sent via YAML map
>>>>> instances.
>>>>>>> In each map, the top level keys are the messages, each key being a
>> tag
>>>>> that
>>>>>>> identifies the message handler. The handler is given the map entry,
>>> key
>>>>> and
>>>>>>> value, and processes it. If parallelism is needed, the handlers can
>>>>> attach
>>>>>>> an "sequence" field inside the map value to match requests to
>>> responses.
>>>>>>>
>>>>>>> The RPC driver is simple. It has a hash of tags to
>>>>> handlers/continuations.
>>>>>>> It accumulates data from the socket until it gets a successful YAML
>>>>> parse.
>>>>>>> Then it walks the top level keys, dispatching an event for each one
>> to
>>>>> the
>>>>>>> handler associated with the tag. Sending is done handing a YAML map
>>>>> object
>>>>>>> to the RPC, which puts it in a queue to be written by a blocking
>> write
>>>>>>> thread. My view is this would be less work than integrating a full
>>>>> featured
>>>>>>> RPC with many features we don't need.
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Derek
>>>>>
>>>>>
>>>>
>>>> --
>>>> *Beware the fisherman who's casting out his line in to a dried up
>>> riverbed.*
>>>> *Oh don't try to tell him 'cause he won't believe. Throw some bread to
>>> the
>>>> ducks instead.*
>>>> *It's easier that way. *- Genesis : Duke : VI 25-28
>>>
>>>
>>
>> --
>> *Beware the fisherman who's casting out his line in to a dried up
>> riverbed.*
>> *Oh don't try to tell him 'cause he won't believe. Throw some bread to the
>> ducks instead.*
>> *It's easier that way. *- Genesis : Duke : VI 25-28
>>
>
>
> --
> *Beware the fisherman who's casting out his line in to a dried up riverbed.*
> *Oh don't try to tell him 'cause he won't believe. Throw some bread to the
> ducks instead.*
> *It's easier that way. *- Genesis : Duke : VI 25-28
