-1 I think a good set of HTTP APIs can simplify client implementations and allow for easier integration with Kafka as well as simpler ops/admin tools. Elasticsearch is a great example. However, I think it's too early to put this into the core. The existing Kafka REST proxies (that I know of) are based on HTTP 1.1 and require base64 encoding of binary data. I hope/suspect that we will see HTTP2-based Kafka-proxies but they should be external to the core project for rapid experimentation. Maybe someday, we can standardize on one. As an example, even if we decide to support HTTP 2, we still need a message framing spec on top of it. gRPC is an option but who knows what else will emerge. Then there's metadata serialization (Protobuf for gRPC). It will be hard to get everyone to agree on these and we probably don't need to all agree on them at this point.
For reference, - this is the PubSub gRPC spec: https://github.com/ googleapis/googleapis/blob/master/google/pubsub/v1/pubsub.proto - this is the message framing spec for gRPC on top of HTTP2 - http://www.grpc.io/docs/guides/wire.html Delimited-Message → Compressed-Flag Message-Length Message Compressed-Flag → 0 / 1 # encoded as 1 byte unsigned integer Message-Length → {length of Message} # encoded as 4 byte unsigned integer Message → *{binary octet} Cheers, Roger On Wed, Oct 26, 2016 at 10:21 PM, Jaikiran Pai <jai.forums2...@gmail.com> wrote: > -1. > > I would personally like Kafka core to be limited to the core aspects. I > consider REST server like this more a tooling thing than a core aspect of > Kafka and would like it to be outside of Kafka. > > -Jaikiran > > > On Wednesday 26 October 2016 02:46 AM, Harsha Chintalapani wrote: > >> Hi All, >> We are proposing to have a REST Server as part of Apache >> Kafka >> to provide producer/consumer/admin APIs. We Strongly believe having >> REST server functionality with Apache Kafka will help a lot of users. >> Here is the KIP that Mani Kumar wrote >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-80:+Ka >> fka+Rest+Server. >> There is a discussion thread in dev list that had differing opinions on >> whether to include REST server in Apache Kafka or not. You can read more >> about that in this thread >> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mb >> ox/%3CCAMVt_AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@ma >> il.gmail.com%3E >> >> This is a VOTE thread to check interest in the community for >> adding REST Server implementation in Apache Kafka. >> >> Thanks, >> Harsha >> >> >
