Re: RPC replacement

2018-08-20 Thread Chris Lemmons
Well, and this is a bit nit-picky, but the JSON-RPC spec does require that a compliant implementation return a parse error if given invalid JSON. We could relax that if we wish, but it feels like it invites bugs. Also, it looks like yaml-cpp does emit yaml that is not json: https://github.com/jbed

Re: RPC replacement

2018-08-20 Thread Alan Carroll
I spent some time discussing this with Jason. We agree the key question here is, given YAMLCPP as a serializer, are there any cases where it will produce non-JSON output? Any incoming requests that are pure JSON will be parsed without a problem by YAMLCPP. But is the reverse true, that any output f

Re: RPC replacement

2018-08-20 Thread Alan Carroll
That's one of the things that attracted me to this idea. Once you've got a JSON/YAML parser, everything else is easy. You frequently don't even need the serializer - for your examples, those are just as easily done with `printf` as an official serializer. I'm also unclear if there is a really a di

Re: RPC replacement

2018-08-20 Thread Chris Lemmons
In the past, I've used JSON-RPC to good effect: https://www.jsonrpc.org/specification There are a bunch of things I like about it, but the biggest is that it's exceedingly well-specified and very easy to implement. Requests look like this: {"jsonrpc": "2.0", "method": "subtract", "params": [42, 2

Re: RPC replacement

2018-08-18 Thread Leif Hedstrom
> On Aug 17, 2018, at 2:34 PM, Alan Carroll > wrote: > > As far as I can tell, JSON schemas work for YAML as well, since the parsed > forms are identical. Cool. Maybe next step then is to show at least some subset of how the schema would look, such that a client could implement the YAML se

Re: RPC replacement

2018-08-18 Thread Leif Hedstrom
> On Aug 17, 2018, at 8:19 AM, Jason Kenny wrote: > > There are a few items in my mind that could be fixed. > > 1) There is a massive complex build knot of depends in this code. Changing > it would clean this up Fwiw, a *lot* of the weirdness and complexity here comes from the “local manag

Re: RPC replacement

2018-08-17 Thread Jason Kenny
There are a few items in my mind that could be fixed. 1) There is a massive complex build knot of depends in this code. Changing it would clean this up 2) The current code is still hard to follow. the YAML change would make it easier to understand, follow and maintain 3) independent of the "protoc

Re: RPC replacement

2018-08-17 Thread Alan Carroll
As far as I can tell, JSON schemas work for YAML as well, since the parsed forms are identical. On Fri, Aug 17, 2018 at 4:47 PM Leif Hedstrom wrote: > > > > On Aug 17, 2018, at 09:51, Alan Carroll > wrote: > > > > We would not be inventing our own marshaling again, we would be using > YAML. > >

Re: RPC replacement

2018-08-17 Thread Leif Hedstrom
> On Aug 17, 2018, at 09:51, Alan Carroll > wrote: > > We would not be inventing our own marshaling again, we would be using YAML. Is there schema definitions for yaml? I think that should be a minimum requirement at least. — Leif > >> On Fri, Aug 17, 2018 at 12:45 PM Leif Hedstrom wro

Re: RPC replacement

2018-08-17 Thread Alan Carroll
We would not be inventing our own marshaling again, we would be using YAML. On Fri, Aug 17, 2018 at 12:45 PM Leif Hedstrom wrote: > > > > On Aug 17, 2018, at 06:46, Alan Carroll > wrote: > > > > 1) Yes. Two counter points - right now we're not reducing anything > because > > no one is willing t

Re: RPC replacement

2018-08-17 Thread Leif Hedstrom
> On Aug 17, 2018, at 06:46, Alan Carroll > wrote: > > 1) Yes. Two counter points - right now we're not reducing anything because > no one is willing to actually bring in the third party RPC. This proposal > would IMHO be definitely better than the current one. Second, bringing in > complete

Re: RPC replacement

2018-08-17 Thread Alan Carroll
1) Yes. Two counter points - right now we're not reducing anything because no one is willing to actually bring in the third party RPC. This proposal would IMHO be definitely better than the current one. Second, bringing in complete third party RPC will require glue logic to adapt to our infrastruct

Re: RPC replacement

2018-08-17 Thread Aaron Canary
1. I thought the goal of replacing RPC was to reduce code maintenance. Does writing our own do that? How does that effect the other apps we are communicating with? 2. We are discussing using a 3rd party health monitor app which is likely to have frequent communication with ATS. I.e. ATS pushes req

Re: RPC replacement

2018-08-16 Thread Alan Carroll
The current RPC code is a mess, with bizarre even for TS compiling and linking. It won't be really bidirectional even if traffic_manager is removed. It's fragile, it's extremely difficult to extend or modify. You have to basically write custom serialization and deserialization logic for every messa

Re: RPC replacement

2018-08-16 Thread Leif Hedstrom
Then what is the motivation to change anything? Let’s just keep the serialization the same as what we have, Peach cleaned that up noticeably. The issue with bidirectional communication gets solved as part of the elimination of traffic_manager. — Leif > On Aug 16, 2018, at 17:40, Alan Carroll

Re: RPC replacement

2018-08-16 Thread Alan Carroll
Yeah, it's not like we have any expertise in asynchronous I/O on IPC sockets, that would let us leverage someone else's YAML based serialization / deserialization library. We should definitely go with something that has its own I/O mechanisms (which are sure to be compatible!) and quite possibly it

Re: RPC replacement

2018-08-16 Thread Leif Hedstrom
> On Aug 16, 2018, at 3:17 PM, Leif Hedstrom wrote: > > I think rolling our own defeats the purpose of replacing what we have. We > should pick something where there is a large sets of client libraries > already. As you say, performance is not critical, so pick one that is > popular. I thin

Re: RPC replacement

2018-08-16 Thread Leif Hedstrom
I think rolling our own defeats the purpose of replacing what we have. We should pick something where there is a large sets of client libraries already. As you say, performance is not critical, so pick one that is popular. I think gRPC would be fine too. — Leif > On Aug 16, 2018, at 11:59, De

Re: RPC replacement

2018-08-16 Thread Derek Dagit
Thanks for clarifying the scope. Good notes. I do like the idea of having a schema. Hopefully if we enable external registration we could provide a way for those also to have schemas with validation. What makes this more plausible now is the integration of YAML support to ATS, whereas before we m

Re: RPC replacement

2018-08-16 Thread Alan Carroll
This is for replacing the existing internal RPC that is used to communicate between traffic_ctl, traffic_manager, and traffic_server. This current runs on a UNIX domain socket and that would not change. Because it's UNIX domain, it has the same security requirements as any of the configuration file

Re: RPC replacement

2018-08-16 Thread Derek Dagit
I was not at Cork, so I am probably missing a lot of the details. It seems having true RPC here would greatly expand the possibilities for management and monitoring. - What are the current and planned use cases for this RPC? - What if we want to add authentication/authorization for multi-tenant