If we want to start a vote on this, I suggest starting a [VOTE] thread
instead of mentioning it in the middle of a very long [DISCUSS] thread. :)
Ismael
On Mon, Oct 24, 2016 at 9:46 PM, Ben Davison <ben.davi...@7digital.com>
wrote:
> This KIP has got muddy on differing opinions of what should be part of
> Kafka etc. I agree with Suresh, let's have a vote on if we actually want a
> REST client in Kafka core, and then we can work out what it actually looks
> like (personally I would like to reuse Confluents, great REST client if
> they donate it to ASF)
>
> For me +1 on REST client, this is a fundamental feature that's missing from
> Kafka.
>
> On Mon, Oct 24, 2016 at 9:22 PM, Jay Kreps <j...@confluent.io> wrote:
>
> > Hey Suresh,
> >
> > I think we agree that REST apis are useful. I don't think we agree that
> > they need to be part of the Kafka project. We've had this discussion a
> > dozen odd times for different protocols---AMQP, REST, MQTT. Going back
> the
> > last five years we've always rejected these. That doesn't mean they
> aren't
> > useful, I think having these as separate is fine, they don't have any
> > complex interaction with Kafka, they just use the vanilla APIs like any
> of
> > the dozens of things on the ecosystem page.
> >
> > In terms of how they're developed. I think there are two discussions
> here:
> > 1. Separate project or not
> > 2. Standalone Apache project or github
> >
> > The first of those I just talked about---I think this makes sense as an
> > independent project.
> >
> > For the second of these, I actually don't think that spawning off a bunch
> > of itty-bitty independent Apache projects is a good thing. I think you
> can
> > see this in the Hadoop ecosystem where the parts kind of all evolve off
> in
> > different directions and don't really work together as a whole. I think
> > that makes sense for deep infrastructure projects, but not for these
> little
> > helper projects. I think a hub and spoke model where you have a central
> > project (Kafka) and then a bunch of helper tools that strive to fit in
> with
> > it (in terms of config, monitoring, apis, and every other conventions) is
> > actually much better. In any case there are already so many of these
> tools
> > (we capture maybe 20% of them on the ecosystem page), made by so many
> > people, and virtually all developed in this style, it is a bit late to
> put
> > the cat back in the bag.
> >
> > Perhaps it's just a difference in background. For my part I had many
> years
> > successfully running github-style projects, and i think that model can
> work
> > quite well for small things. I do think it is important for these
> projects
> > to clarify governance, which we should absolutely do, but realistically
> it
> > is a pretty simple tool, there isn't a huge governance challenge for
> > something like this because its scope is so limited ("take http requests,
> > make Kafka requests").
> >
> > More specifically, I don't think there is an actual problem being solved
> > here. I haven't heard any complaint about direction or patches not
> getting
> > added. The only complaint I've heard is missing features where the normal
> > "patches accepted" rule applies. I would urge people to actually get
> > involved in contribution here. In the future if there is a situation
> where
> > people don't like the direction of a given tool, they can fork it and
> > either turn it into an independent Apache project or develop it
> > independently, trying to do that preemptively seems a bit hostile.
> >
> > -Jay
> >
> > On Mon, Oct 24, 2016 at 12:32 PM, Suresh Srinivas <
> sur...@hortonworks.com>
> > wrote:
> >
> > > I am dividing this discussion into two parts:
> > > 1. REST APIs as core Apache Kafka capability
> > > This should be a core Kafka functionality. Same view has been reflected
> > by
> > > others (users and developers) as well. While we can debate whether
> other
> > > capabilities are core Kafka (Streams, Connect), it would be good
> separate
> > > that out and to keep this discussion focussed on REST APIs as proposed
> by
> > > this KIP. If there is ambivalence about the need for this in core
> kafka,
> > > we could have voting in the mailing list.
> > >
> > > Can we get an agreement on this? I am +1 on REST API in Apache Kafka.
> > >
> > > 2. Community where Kafka REST APIs need to be collaborated on
> > > There is already a GitHub project that caters to Kafka REST APIs. But a
> > > company owned Github is less than ideal for collaboration due to lack
> of
> > > governance, open community and roadmap. This is reflected by many
> people
> > > interested in this functionality and still not contributing to and
> > > adopting the code base in the GitHub. I think trying overlay the ASF
> > > governance model on GitHub project, which is what the need is, seems
> > > unnecessary, if the code can be part of Apache Kafka.
> > >
> > > The question is, would Confluent be okay with licensing/contributing
> the
> > > code to Kafka project (assuming either Confluent or another contributor
> > > can work on it)? I see clear intent in collaborating with others on
> REST
> > > APIs from confluent. Why not do it in Kafka project under ASF? This
> takes
> > > away all the barrier to collaboration? Alternatively, if Confluent is
> not
> > > willing to contribute the code from GitHub, would anyone veto building
> a
> > > separate REST API functionality in ASF Kafka? There are enough people
> who
> > > want to work on this and maintain it.
> > >
> > > Regards,
> > > Suresh
> > >
> > >
> > >
> > > On 10/21/16, 9:41 PM, "Harsha Chintalapani" <ka...@harsha.io> wrote:
> > >
> > > >Sriram,
> > > > "Can the streaming platform exist without stream processing? - No.
> > > >Processing stream data again is a core part of streaming platform."
> > > >
> > > >Yes, it can. There are no.of Stream processing frameworks out there,
> and
> > > >they all have integration into Kafka.
> > > >It doesn't need to be developed within Kafka.
> > > >
> > > >
> > > >"Can the platform exist without the rest proxy? - Yes. The proxy does
> > not
> > > >complete the platform vision in anyway. It is just a good to have tool
> > > >that
> > > >might be required by quite a few users and there is an active project
> > that
> > > >works on this - https://github.com/confluentinc/kafka-rest";
> > > >
> > > >The rest proxy is as important as any API. The vision that shown here
> > > >http://kafka.apache.org/intro
> > > >require users to write the producers and consumers to get their data
> > into
> > > >and out of Kafka, without which having Streams or Connect won't help
> > > >anyone.
> > > >The rest proxy makes easier for users get their data into Kafka.
> > > >Adding the rest proxy to the project doesn't invalidate the current
> > > >vision,
> > > >it only strengthens it.
> > > >
> > > >Thanks,
> > > >Harsha
> > > >
> > > >
> > > >
> > > >
> > > >On Fri, Oct 21, 2016 at 2:31 PM Sriram Subramanian <r...@confluent.io>
> > > >wrote:
> > > >
> > > >FWIW, Apache Kafka has evolved a lot from where it started. It did
> start
> > > >as
> > > >a messaging system. Over time we realized that that the vision for
> Kafka
> > > >is
> > > >to build a streaming platform and not just a messaging system. You can
> > > >take
> > > >a look at the site for more description about what comprises the
> > streaming
> > > >platform http://kafka.apache.org/ and http://kafka.apache.org/intro.
> > > >
> > > >Can the streaming platform exist without Connect? - No. Data
> integration
> > > >is
> > > >fundamental to building an end to end platform
> > > >
> > > >Can the streaming platform exist without stream processing? - No.
> > > >Processing stream data again is a core part of streaming platform.
> > > >
> > > >Can the streaming platform exist without clients? - We at least need
> one
> > > >client library to complete the platform. Our Java clients help us to
> > > >complete the platform story. The rest of the clients are built and
> > > >maintained outside the project.
> > > >
> > > >Can the platform exist without the rest proxy? - Yes. The proxy does
> not
> > > >complete the platform vision in anyway. It is just a good to have tool
> > > >that
> > > >might be required by quite a few users and there is an active project
> > that
> > > >works on this - https://github.com/confluentinc/kafka-rest
> > > >
> > > >
> > > >
> > > >
> > > >On Fri, Oct 21, 2016 at 11:49 AM, Nacho Solis
> > > ><nso...@linkedin.com.invalid>
> > > >wrote:
> > > >
> > > >> Are you saying Kafka REST is subjective but Kafka Streams and Kafka
> > > >Connect
> > > >> are not subjective?
> > > >>
> > > >> > "there are likely places that can live without a rest proxy"
> > > >>
> > > >> There are also places that can live without Kafka Streams and Kafka
> > > >> Connect.
> > > >>
> > > >> Nacho
> > > >>
> > > >> On Fri, Oct 21, 2016 at 11:17 AM, Jun Rao <j...@confluent.io> wrote:
> > > >>
> > > >> > At the high level, I think ideally it makes sense to add a
> component
> > > >>to
> > > >> > Apache Kafka if (1) it's widely needed and (2) it needs tight
> > > >integration
> > > >> > with Kafka core. For Kafka Stream, we do expect stream processing
> > will
> > > >be
> > > >> > used widely in the future. Implementation wise, Kafka Stream only
> > > >> supports
> > > >> > getting data from Kafka and leverages quite a few of the core
> > > >> > functionalities in Kafka core. For example, it uses customized
> > > >>rebalance
> > > >> > callback in the consumer and uses the compacted topic heavily. So,
> > > >having
> > > >> > Kafka Stream in the same repo makes it easier for testing when
> those
> > > >core
> > > >> > functionalities evolve over time. Kafka Connect is in the same
> > > >situation.
> > > >> >
> > > >> > For rest proxy, whether it's widely used or not is going to be a
> bit
> > > >> > subjective. However, there are likely places that can live
> without a
> > > >rest
> > > >> > proxy. The rest proxy is just a proxy for the regular clients and
> > > >doesn't
> > > >> > need to be tightly integrated with Kafka core. So, the case for
> > > >including
> > > >> > rest proxy in Apache Kafka is probably not as strong as Kafka
> Stream
> > > >>and
> > > >> > Kafka Connect.
> > > >> >
> > > >> > Thanks,
> > > >> >
> > > >> > Jun
> > > >> >
> > > >> > On Thu, Oct 20, 2016 at 11:28 PM, Michael Pearce
> > > >><michael.pea...@ig.com>
> > > >> > wrote:
> > > >> >
> > > >> > > So from my reading essentially the first question needs to
> > > >answered/and
> > > >> > > voted on is:
> > > >> > >
> > > >> > > Is Apache Kafka Community only about the Core or does the apache
> > > >> > community
> > > >> > > also support some subprojects (and just we need some better way
> to
> > > >> manage
> > > >> > > this)
> > > >> > >
> > > >> > > If vote for Core only wins, then the following should be
> removed:
> > > >> > > Kafka Connect
> > > >> > > Kafka Stream
> > > >> > >
> > > >> > > If vote for Core only loses (aka we will support subprojects)
> > then:
> > > >> > > We should look to add Kafka Rest
> > > >> > >
> > > >> > > And we should look to see how we can manage better govern and
> > manage
> > > >> > > submodules.
> > > >> > >
> > > >> > > A good example which id propose here is how some other
> communities
> > > >>in
> > > >> > > Apache do this.
> > > >> > >
> > > >> > > Each Module has a Module Management Committee(MMC), this is like
> > > >almost
> > > >> > > the PMC but at a per module basis.
> > > >> > >
> > > >> > > This MMC should essentially hold the binding votes for that
> > module.
> > > >> > > The MMC should be made up of a single representative from each
> > > >> > > organisation (so no single organisation can fully veto the
> > > >>community
> > > >> it
> > > >> > > has to a genuine consenus)
> > > >> > > The MMC requires at least 3 members (so there cant be a tied
> vote
> > on
> > > >2)
> > > >> > > For a new Module to be added a MMC committee should be sought
> > > >> > > A new Module is only capable of being added if the above
> > > >>requirements
> > > >> can
> > > >> > > be met (e.g. 3 people wishing to step up, from 3 organisations)
> so
> > > >that
> > > >> > > only actively support modules would be added
> > > >> > >
> > > >> > > The PMC reviews each module every 6months or Year. If MMC is
> > > >>inactive,
> > > >> a
> > > >> > > vote/call to find replacements if raised, if none are
> forthcoming
> > > >> > dropping
> > > >> > > the MMC to less than 3 then the module moves to "the attic"
> (very
> > > >>much
> > > >> > like
> > > >> > > apache attic but a little more aggressively)
> > > >> > >
> > > >> > > This way the PMC does not need to micro manage every module
> > > >> > > We only add modules where some amount of active support and
> > > >maintenance
> > > >> > > and use is provided by the community
> > > >> > > We have an automatic way to retire old or inactive projects.
> > > >> > >
> > > >> > > Thoughts?
> > > >> > > Mike
> > > >> > >
> > > >> > >
> > > >> > > ________________________________________
> > > >> > > From: Harsha Ch <harsha...@gmail.com>
> > > >> > > Sent: Thursday, October 20, 2016 10:26 PM
> > > >> > > To: dev@kafka.apache.org
> > > >> > > Subject: Re: [DISCUSS] KIP-80: Kafka REST Server
> > > >> > >
> > > >> > > Jay,
> > > >> > > REST API is something every user is in need of. If the
> > > >>argument
> > > >> is
> > > >> > to
> > > >> > > clone and write your API, this will do a disservice to the
> users
> > as
> > > >> they
> > > >> > > now have to choose one vs. others instead of keeping one API
> that
> > is
> > > >> > > supported in Kafka community.
> > > >> > >
> > > >> > > "Pre-emptively re-creating another
> > > >> > > REST layer when it seems like we all quite agree on what needs
> to
> > be
> > > >> done
> > > >> > > and we have an existing code base for HTTP/Kafka access that is
> > > >heavily
> > > >> > > used in production seems quite silly."
> > > >> > >
> > > >> > > Exactly our point. Why can't we develop this in Apache
> > Kafka
> > > >> > > community? Instead of us open sourcing another GitHub project
> and
> > > >> > creating
> > > >> > > a divide in users and another version of API. Let's build this
> in
> > > >Kafka
> > > >> > > Community and use the governance model that is proven to provide
> > > >vendor
> > > >> > > free user driven consensus features. The argument that is adding
> > > >>this
> > > >> > REST
> > > >> > > server to Kafka will affect the agility of the project doesn't
> mak
> > > >> sense.
> > > >> > >
> > > >> > > It looks like your argument is either we develop all these small
> > > >>tools
> > > >> or
> > > >> > > none at all. We as a community need to look at supporting
> critical
> > > >> > > tools/API. Instead of dividing this project into individual
> > external
> > > >> > > communities. We should build this as part of Kafka which best
> > serves
> > > >> the
> > > >> > > needs of users.
> > > >> > > The Streams and Connect projects that were pushed into
> > Kafka
> > > >> > could
> > > >> > > have been left in their own Github projects based on your
> > arguments.
> > > >> What
> > > >> > > about the REST API is so different that such that it should stay
> > out
> > > >of
> > > >> > the
> > > >> > > Kafka project? From my experience, more users are asking for the
> > > >>REST
> > > >> > API.
> > > >> > >
> > > >> > > Thanks,
> > > >> > > Harsha
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > On Wed, Oct 12, 2016 at 8:03 AM Jay Kreps <j...@confluent.io>
> > wrote:
> > > >> > >
> > > >> > > > I think the questions around governance make sense, I think we
> > > >should
> > > >> > > > really clarify that to make the process more clear so it can
> be
> > > >fully
> > > >> > > > inclusive.
> > > >> > > >
> > > >> > > > The idea that we should not collaborate on what is there now,
> > > >though,
> > > >> > > > because in the future we might disagree about direction does
> not
> > > >> really
> > > >> > > > make sense to me. If in the future we disagree, that is the
> > beauty
> > > >of
> > > >> > > open
> > > >> > > > source, you can always fork off a copy of the code and start
> an
> > > >> > > independent
> > > >> > > > project either in Apache or elsewhere. Pre-emptively
> re-creating
> > > >> > another
> > > >> > > > REST layer when it seems like we all quite agree on what needs
> > to
> > > >>be
> > > >> > done
> > > >> > > > and we have an existing code base for HTTP/kafka access that
> is
> > > >> heavily
> > > >> > > > used in production seems quite silly.
> > > >> > > >
> > > >> > > > Let me give some background on how I at least think about
> these
> > > >> things.
> > > >> > > > I've participated in open source projects out of LinkedIn via
> > > >>github
> > > >> as
> > > >> > > > well as via the ASF. I don't think there is a "right" answer
> to
> > > >>how
> > > >> to
> > > >> > do
> > > >> > > > these but rather some tradeoffs. We thought about this quite a
> > lot
> > > >in
> > > >> > the
> > > >> > > > context of Kafka based on the experience with the Hadoop
> > ecosystem
> > > >as
> > > >> > > well
> > > >> > > > as from other open source communities.
> > > >> > > >
> > > >> > > > There is a rich ecosystem around Kafka. Many of the projects
> are
> > > >> quite
> > > >> > > > small--single clients or tools that do a single thing
> well--and
> > > >> almost
> > > >> > > none
> > > >> > > > of them are top level apache projects. I don't think trying to
> > > >>force
> > > >> > each
> > > >> > > > of these to turn into independent Apache projects is
> necessarily
> > > >>the
> > > >> > best
> > > >> > > > thing for the ecosystem.
> > > >> > > >
> > > >> > > > My observation of how this can go wrong is really what I think
> > has
> > > >> > > happened
> > > >> > > > to the Hadoop ecosystem. There you see quite a zoo of projects
> > > >>which
> > > >> > all
> > > >> > > > drift apart and don't quite work together well. Coordinating
> > even
> > > >> > simple
> > > >> > > > changes and standardization across these is exceptionally
> > > >>difficult.
> > > >> > The
> > > >> > > > result is a bit of a mess for users--the pieces just don't
> > really
> > > >> come
> > > >> > > > together very well. This makes sense for independent
> > > >>infrastructure
> > > >> > > systems
> > > >> > > > (Kudu vs HDFS) but I'm not at all convinced that doing this
> for
> > > >every
> > > >> > > > little tool or helper library has lead to a desirable state. I
> > > >>think
> > > >> > the
> > > >> > > > mode of operating where the Hadoop vendors spawn off a few new
> > > >Apache
> > > >> > > > projects for each new product initiative, especially since
> often
> > > >that
> > > >> > > > project is only valued by that vendor (and the other vendor
> has
> > a
> > > >> > > different
> > > >> > > > competing Apache project) doesn't necessarily do a better job
> at
> > > >> > > producing
> > > >> > > > high quality communities or high quality software.
> > > >> > > >
> > > >> > > > These tools/connects/clients/proxies and other integration
> > pieces
> > > >can
> > > >> > > take
> > > >> > > > many forms, but my take of what makes one of these things good
> > is
> > > >> that
> > > >> > it
> > > >> > > > remains simple, does its one thing well, and cleaves as
> closely
> > as
> > > >> > > possible
> > > >> > > > to the conventions for Kafka itself--i.e. doesn't invent new
> > ways
> > > >>of
> > > >> > > > monitoring, configuring, etc. For the tools we've contributed
> > > >>we've
> > > >> > tried
> > > >> > > > really hard to make them consistent with Kafka as well as with
> > > >>each
> > > >> > other
> > > >> > > > in how testing, configuration, monitoring, etc works.
> > > >> > > >
> > > >> > > > I think what Apache does superbly well is create a community
> for
> > > >> > > managing a
> > > >> > > > large infrastructure layer like Kafka in a vendor independent
> > way.
> > > >> > What I
> > > >> > > > think is less successful is attempting to form full and
> > > >>independent
> > > >> > > apache
> > > >> > > > communities around very simple single purpose tools,
> especially
> > if
> > > >> you
> > > >> > > hope
> > > >> > > > for these to come together into a cohesive toolset across
> > multiple
> > > >> such
> > > >> > > > tools. Much of what Apache does--create a collective decision
> > > >>making
> > > >> > > > process for resolving disagreement, help to trademark and
> > protect
> > > >the
> > > >> > > marks
> > > >> > > > of the project, etc just isn't that relevant for simple
> > > >> single-purpose
> > > >> > > > tools.
> > > >> > > >
> > > >> > > > So my take is there are a couple of options:
> > > >> > > >
> > > >> > > > 1. We can try to put all the small tools into the Apache
> > > >>Project.
> > > >> I
> > > >> > > > think this is not the right approach as there is simply too
> > > >>many
> > > >> of
> > > >> > > > them,
> > > >> > > > many in different languages, serving different protocols,
> > > >> > integrating
> > > >> > > > with
> > > >> > > > particular systems, and a single community can't
> effectively
> > > >> > maintain
> > > >> > > > them
> > > >> > > > all. Doing this would significantly slow the progress of
> the
> > > >Kafka
> > > >> > > > project.
> > > >> > > > As a protocol for messaging, I don't really see a case for
> > > >> including
> > > >> > > > REST
> > > >> > > > but not MQTT or AMQP which are technically much better
> suited
> > > >>to
> > > >> > > > messaging
> > > >> > > > and both are widely used for that.
> > > >> > > > 2. We can treat ecosystem projects that aren't top level
> > Apache
> > > >> > > projects
> > > >> > > > as invalid and try to recreate them all as Apache projects.
> > > >> > Honestly,
> > > >> > > > though, if you go to the Kafka ecosystem page virtually
> none
> > of
> > > >> the
> > > >> > > most
> > > >> > > > popular add-ons to Kafka are Apache projects. The most
> > > >>successful
> > > >> > > > things in
> > > >> > > > the Kafka ecosystem such as Yahoo Manager, librdkafka, a
> > number
> > > >of
> > > >> > > other
> > > >> > > > clients, as well as the existing REST layer have succeeded
> at
> > > >> > > developing
> > > >> > > > communities that actively contribute and use these pieces
> > and I
> > > >> > don't
> > > >> > > > know
> > > >> > > > that that is a bad thing unless that community proves to be
> > > >> > > uninclusive,
> > > >> > > > unresponsive, or goes in a bad technical direction--and
> those
> > > >>are
> > > >> > > > failure
> > > >> > > > modes that all open source efforts face.
> > > >> > > > 3. We can do what I think makes the most sense and try to
> > work
> > > >> with
> > > >> > > the
> > > >> > > > projects that exist in the ecosystem and if the project
> > doesn't
> > > >> > have a
> > > >> > > > responsive community or wants to go in a different
> direction
> > > >>fork
> > > >> or
> > > >> > > > recreate that work.
> > > >> > > >
> > > >> > > > Of course any person can choose whatever of these options they
> > > >>want.
> > > >> > But
> > > >> > > > from my point of view, option (3) has been the path of the
> > > >>community
> > > >> so
> > > >> > > far
> > > >> > > > and I think it has been quite successful.
> > > >> > > >
> > > >> > > > -Jay
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > > On Tue, Oct 11, 2016 at 10:33 PM, Harsha Chintalapani <
> > > >> ka...@harsha.io
> > > >> > >
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > > > Neha,
> > > >> > > > > "But I haven't seen any community emails or patches being
> > > >submitted
> > > >> > by
> > > >> > > > you
> > > >> > > > > guys, so I'm wondering why you are concerned about whether
> the
> > > >> > > community
> > > >> > > > is
> > > >> > > > > open to accepting patches or not."
> > > >> > > > >
> > > >> > > > > I think you are talking about contributing patches to this
> > > >> repository
> > > >> > > > > right? https://github.com/confluentinc/kafka-rest . All I
> am
> > > >> saying
> > > >> > > > > the guidelines/governance model is not clear on the project
> > and
> > > >>I
> > > >> > guess
> > > >> > > > its
> > > >> > > > > driven by opening a github issue request. Its the
> repository
> > > >owned
> > > >> > by
> > > >> > > > > confluent and as much I appreciate that the features we
> > > >>mentioned
> > > >> are
> > > >> > > in
> > > >> > > > > the roadmap and welcoming us to contribute to the project.
> It
> > > >> doesn't
> > > >> > > > > gurantee what we want to add in the furture will be in your
> > > >> roadmap.
> > > >> > > > >
> > > >> > > > > Hence the reason having it part of Kafka community will
> help a
> > > >>lot
> > > >> as
> > > >> > > > other
> > > >> > > > > users can participate in the discussions. We are happy to
> > drive
> > > >> any
> > > >> > > > > feature additions through KIPs which gives everyone a chance
> > to
> > > >> > > > participate
> > > >> > > > > and add to the discussions.
> > > >> > > > >
> > > >> > > > > Thanks,
> > > >> > > > > Harsha
> > > >> > > > >
> > > >> > > > > On Fri, Oct 7, 2016 at 11:52 PM Michael Pearce <
> > > >> > michael.pea...@ig.com>
> > > >> > > > > wrote:
> > > >> > > > >
> > > >> > > > > > +1
> > > >> > > > > >
> > > >> > > > > > I agree on the governance comments whole heartedly.
> > > >> > > > > >
> > > >> > > > > > Also i agree about the contribution comments made earlier
> in
> > > >>the
> > > >> > > > thread,
> > > >> > > > > i
> > > >> > > > > > personally am less likely to spend any of mine, or give
> > > >>project
> > > >> > time
> > > >> > > > > within
> > > >> > > > > > my internal projects to developers contributing to another
> > > >> > commercial
> > > >> > > > > > companies project even so technically open source, as then
> > > >>there
> > > >> is
> > > >> > > > that
> > > >> > > > > > commercial companies interest will always prevail and
> > > >essentially
> > > >> > can
> > > >> > > > > > always have a final vote where disagreement. Im sure they
> > > >>never
> > > >> > > intend
> > > >> > > > > to,
> > > >> > > > > > but there is that true reality. This is why we have
> > community
> > > >> open
> > > >> > > > source
> > > >> > > > > > projects.
> > > >> > > > > >
> > > >> > > > > > I can find many different implementations now of a rest
> > > >>endpoint
> > > >> on
> > > >> > > > > > GitHub, BitBucket etc. Each one has their benefits and
> > > >> > disadvantages
> > > >> > > in
> > > >> > > > > > their implementation. By making / providing one this would
> > > >>bring
> > > >> > > > together
> > > >> > > > > > these solutions, unifying those developers and also
> bringing
> > > >>the
> > > >> > best
> > > >> > > > of
> > > >> > > > > > all.
> > > >> > > > > >
> > > >> > > > > > I understand the concern on the community burden
> > > >> adding/supporting
> > > >> > > more
> > > >> > > > > > surface area for every client. But something like REST is
> > > >> universal
> > > >> > > and
> > > >> > > > > > worthy to be owned by the community.
> > > >> > > > > >
> > > >> > > > > > Mike
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > ________________________________________
> > > >> > > > > > From: Andrew Schofield <andrew_schofield_j...@outlook.com
> >
> > > >> > > > > > Sent: Saturday, October 8, 2016 1:19 AM
> > > >> > > > > > To: dev@kafka.apache.org
> > > >> > > > > > Subject: Re: [DISCUSS] KIP-80: Kafka REST Server
> > > >> > > > > >
> > > >> > > > > > There's a massive difference between the governance of
> Kafka
> > > >>and
> > > >> > the
> > > >> > > > > > governance of the REST proxy.
> > > >> > > > > >
> > > >> > > > > > In Kafka, there is a broad community of people
> contributing
> > > >their
> > > >> > > > > opinions
> > > >> > > > > > about future enhancements in the form of KIPs. There's
> some
> > > >> really
> > > >> > > deep
> > > >> > > > > > consideration that goes into some of the trickier KIPs.
> > There
> > > >are
> > > >> > > > people
> > > >> > > > > > outside Confluent deeply knowledgeable in Kafka and
> > building
> > > >the
> > > >> > > > > > reputations to become committers. I get the impression
> that
> > > >>the
> > > >> > > roadmap
> > > >> > > > > of
> > > >> > > > > > Kafka is not really community-owned (what's the big
> feature
> > > >>for
> > > >> > Kafka
> > > >> > > > > 0.11,
> > > >> > > > > > for example), but the conveyor belt of smaller features in
> > the
> > > >> form
> > > >> > > of
> > > >> > > > > KIPs
> > > >> > > > > > works nicely. It's a good example of open-source working
> > > >>well.
> > > >> > > > > >
> > > >> > > > > > The equivalent for the REST proxy is basically issues on
> > > >>GitHub.
> > > >> > The
> > > >> > > > > > roadmap is less clear. There's not really a community
> > properly
> > > >> > > engaged
> > > >> > > > in
> > > >> > > > > > the way that there is with Kafka. So, you could say that
> > it's
> > > >> clear
> > > >> > > > that
> > > >> > > > > > fewer people are interested, but I think the whole
> > governance
> > > >> > thing
> > > >> > > > is a
> > > >> > > > > > big barrier to engagement. And it's looking like it's
> > getting
> > > >out
> > > >> > of
> > > >> > > > > date.
> > > >> > > > > >
> > > >> > > > > > In technical terms, I can think of two big improvements to
> > the
> > > >> REST
> > > >> > > > > proxy.
> > > >> > > > > > First, it needs to use the new consumer API so that it's
> > > >possible
> > > >> > to
> > > >> > > > > secure
> > > >> > > > > > connections between the REST proxy and Kafka. I don't care
> > too
> > > >> much
> > > >> > > > which
> > > >> > > > > > method calls it uses actually uses to consume messages,
> > but I
> > > >do
> > > >> > > care
> > > >> > > > > that
> > > >> > > > > > I cannot secure connections because of network security
> > rules.
> > > >> > > Second,
> > > >> > > > > > there's an affinity between a consumer and the instance of
> > the
> > > >> REST
> > > >> > > > proxy
> > > >> > > > > > to which it first connected. Kafka itself avoids this kind
> > of
> > > >> > > affinity
> > > >> > > > > for
> > > >> > > > > > good reason, and in the name of availability the REST
> proxy
> > > >> should
> > > >> > > too.
> > > >> > > > > > These are natural KIPs.
> > > >> > > > > >
> > > >> > > > > > I think it would be good to have the code for the REST
> proxy
> > > >> > > > contributed
> > > >> > > > > > to Apache Kafka so that it would be able to be developed
> in
> > > >>the
> > > >> > same
> > > >> > > > way.
> > > >> > > > > >
> > > >> > > > > > Andrew Schofield
> > > >> > > > > >
> > > >> > > > > > From: Suresh Srinivas <sur...@hortonworks.com>
> > > >> > > > > > Sent: 07 October 2016 22:41:52
> > > >> > > > > > To: dev@kafka.apache.org
> > > >> > > > > > Subject: Re: [DISCUSS] KIP-80: Kafka REST Server
> > > >> > > > > >
> > > >> > > > > > ASF already gives us a clear framework and governance
> model
> > > >>for
> > > >> > > > community
> > > >> > > > > > development. This is already understood by the people
> > > >> contributing
> > > >> > to
> > > >> > > > > > Apache Kafka project, and they are the same people who
> want
> > to
> > > >> > > > contribute
> > > >> > > > > > to the REST server capability as well. Everyone is in
> > > >>agreement
> > > >> on
> > > >> > > the
> > > >> > > > > > need for collaborating on this effort. So why not
> contribute
> > > >>the
> > > >> > code
> > > >> > > > to
> > > >> > > > > > Apache Kafka. This will help avoid duplication of effort
> and
> > > >> forks
> > > >> > > that
> > > >> > > > > > may crop up, hugely benefitting the user community. This
> > will
> > > >> also
> > > >> > > > avoid
> > > >> > > > > > having to define a process similar to ASF on a GitHub
> > project
> > > >and
> > > >> > > > instead
> > > >> > > > > > there is a single community with clear understanding
> > community
> > > >> > > process
> > > >> > > > as
> > > >> > > > > > defined in ASF.
> > > >> > > > > >
> > > >> > > > > > As others have said, this is an important capability for
> > > >>Apache
> > > >> > > Kafka.
> > > >> > > > It
> > > >> > > > > > is worth maintaining this as a part of the project.
> > > >> > > > > >
> > > >> > > > > > Regards,
> > > >> > > > > > Suresh
> > > >> > > > > >
> > > >> > > > > > On 10/6/16, 8:32 AM, "Ofir Manor" <ofir.ma...@equalum.io>
> > > >>wrote:
> > > >> > > > > >
> > > >> > > > > > >I personally think it would be quite wasteful to
> > re-implement
> > > >> the
> > > >> > > REST
> > > >> > > > > > >gateway just because that an actively-maintained piece of
> > > >> > > > > Apache-licensed
> > > >> > > > > > >software is not governed directly by the Apache Kafka
> > > >> community...
> > > >> > > > While
> > > >> > > > > > >kafka-rest repo is owned by Confluent, the contributors
> > > >> including
> > > >> > > the
> > > >> > > > > main
> > > >> > > > > > >one are also part of the Apache Kafka community, so
> there
> > > >>is a
> > > >> > > chance
> > > >> > > > > to
> > > >> > > > > > >work this out.
> > > >> > > > > > >
> > > >> > > > > > >However, there are two valid concerns here that could be
> > > >> > addressed,
> > > >> > > > > around
> > > >> > > > > > >community and accessibility:
> > > >> > > > > > >>> What we are worried about is a project
> > > >> > > > > > >>> that's not maintained in a community. So the process
> of
> > > >> > accepting
> > > >> > > > > > >>>patches
> > > >> > > > > > >>> and priorities is not clear, and it's not developed in
> > > >Apache
> > > >> > > > > > >>>community.
> > > >> > > > > > >>> Not only that, existing REST API project doesn't
> support
> > > >>new
> > > >> > > client
> > > >> > > > > API
> > > >> > > > > > >and
> > > >> > > > > > >>> hence there is no security support either.
> > > >> > > > > > >
> > > >> > > > > > >This might be easy to fix. Maybe Confluent / kafka-rest
> > > >> community
> > > >> > > can
> > > >> > > > > > >clarify that - what is their contribution policy, dev
> > style,
> > > >> > roadmap
> > > >> > > > > etc.
> > > >> > > > > > >If they want, they can make an effort to encourage
> > > >participation
> > > >> > > from
> > > >> > > > > > >people outside Confluent (easily accept contributions,
> > invite
> > > >> > > external
> > > >> > > > > > >commiters or have open dev process similar to Apache
> Kafka
> > > >etc),
> > > >> > as
> > > >> > > > > there
> > > >> > > > > > >is definitely seems to be some interest on the list. That
> > > >>might
> > > >> > > clear
> > > >> > > > > the
> > > >> > > > > > >community concern and help kafka-rest project (but that
> is
> > a
> > > >> > > > calculation
> > > >> > > > > > >Confluent will have to make).
> > > >> > > > > > >
> > > >> > > > > > >The other, independent, concern is that REST is something
> > > >>that
> > > >> is
> > > >> > > > > expected
> > > >> > > > > > >to be available out of the box with Kafka. I personally
> > don't
> > > >> feel
> > > >> > > > > > >strongly
> > > >> > > > > > >about it (better use proper, efficient APIs from day
> one),
> > > >> though
> > > >> > it
> > > >> > > > is
> > > >> > > > > > >definitely way smaller than adding a stream processing
> > engine
> > > >to
> > > >> > the
> > > >> > > > > > >project :)
> > > >> > > > > > >Again,the kafka-rest "community" could take steps to make
> > it
> > > >> even
> > > >> > > > easier
> > > >> > > > > > >to
> > > >> > > > > > >install, configure and run kafka-rest for new users on
> > > >>vanilla
> > > >> > > Apache
> > > >> > > > > > >Kafka
> > > >> > > > > > >(outside the Confluent platform), if they wish that (or
> > > >>welcome
> > > >> > > > > > >contributions to that end), but that is up to them.
> > > >> > > > > > >Finally, if after the above steps were taken there would
> > > >>still
> > > >a
> > > >> > > > strong
> > > >> > > > > > >desire to include a great rest gateway with Apache
> Kafka, I
> > > >> assume
> > > >> > > the
> > > >> > > > > > >community could hypothetically fork the existing
> kafka-rest
> > > >into
> > > >> > an
> > > >> > > > > Apache
> > > >> > > > > > >Kafka subproject and maintain it "within Apache" instead
> of
> > > >> > > > implementing
> > > >> > > > > > >it
> > > >> > > > > > >from scratch (though I'm not a lawyer etc) - but I cannot
> > > >> imagine
> > > >> > it
> > > >> > > > > > >happen
> > > >> > > > > > >without Confluent blessing, and I think that is likely
> much
> > > >less
> > > >> > > > optimal
> > > >> > > > > > >(pulling in other Confluent / Apache licensed
> dependencies)
> > > >than
> > > >> > > > having
> > > >> > > > > a
> > > >> > > > > > >separate external community around kafka-rest.
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > >Just my two cents,
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > >Ofir Manor
> > > >> > > > > > >
> > > >> > > > > > >Co-Founder & CTO | Equalum
> > > >> > > > > > >
> > > >> > > > > > >Mobile: +972-54-7801286 <+972%2054-780-1286>
> > > ><+972%2054-780-1286>
> > > >> > <+972%2054-780-1286> |
> > > >> > > > Email:
> > > >> > > > > > ofir.ma...@equalum.io
> > > >> > > > > > >
> > > >> > > > > > >On Sun, Oct 2, 2016 at 11:23 PM, Harsha Chintalapani <
> > > >> > > ka...@harsha.io
> > > >> > > > >
> > > >> > > > > > >wrote:
> > > >> > > > > > >
> > > >> > > > > > >> Neha & Jay,
> > > >> > > > > > >> We did look at the open source
> > > >>alternatives.
> > > >> > Our
> > > >> > > > > > >>concern
> > > >> > > > > > >> is what's the patch acceptance and adding features/
> > > >>bug-fixes
> > > >> to
> > > >> > > the
> > > >> > > > > > >> existing project under a Github (although it's licensed
> > > >>under
> > > >> > > Apache
> > > >> > > > > > >>2.0).
> > > >> > > > > > >> It would be great if that project made available under
> > > >>Apache
> > > >> > and
> > > >> > > > > > >>driven by
> > > >> > > > > > >> the community. Adding to the above, not all Kafka
> users
> > > >>are
> > > >> > > > > interested
> > > >> > > > > > >>in
> > > >> > > > > > >> using the Java client API, they would like to have
> simple
> > > >REST
> > > >> > API
> > > >> > > > > where
> > > >> > > > > > >> they can code against using any language. I do believe
> > this
> > > >> adds
> > > >> > > > value
> > > >> > > > > > >>to
> > > >> > > > > > >> Apache Kafka in itself.
> > > >> > > > > > >>
> > > >> > > > > > >> "For 1, I don't think there is value in giving in to
> the
> > > >>NIH
> > > >> > > > syndrome
> > > >> > > > > > >>and
> > > >> > > > > > >> reinventing the wheel. What I'm looking for is a
> detailed
> > > >> > > comparison
> > > >> > > > > of
> > > >> > > > > > >>the
> > > >> > > > > > >> gaps and why those can't be improved in the REST proxy
> > that
> > > >> > > already
> > > >> > > > > > >>exists
> > > >> > > > > > >> and is actively maintained."
> > > >> > > > > > >>
> > > >> > > > > > >> We are not looking at this as NIH. What we are worried
> > > >>about
> > > >> > is a
> > > >> > > > > > >>project
> > > >> > > > > > >> that's not maintained in a community. So the process of
> > > >> > accepting
> > > >> > > > > > >>patches
> > > >> > > > > > >> and priorities is not clear, and it's not developed in
> > > >>Apache
> > > >> > > > > community.
> > > >> > > > > > >> Not only that, existing REST API project doesn't
> support
> > > >>new
> > > >> > > client
> > > >> > > > > API
> > > >> > > > > > >>and
> > > >> > > > > > >> hence there is no security support either.
> > > >> > > > > > >> We don't know the timeline when that's made available.
> We
> > > >> would
> > > >> > > like
> > > >> > > > > to
> > > >> > > > > > >>add
> > > >> > > > > > >> admin functionality into the REST API. So the Roadmap
> of
> > > >>that
> > > >> > > > project
> > > >> > > > > is
> > > >> > > > > > >> not driven by Apache.
> > > >> > > > > > >>
> > > >> > > > > > >>
> > > >> > > > > > >> "This doesn't materially have an impact on expanding
> the
> > > >> > usability
> > > >> > > > of
> > > >> > > > > > >> Kafka. In my experience, REST proxy + Java clients
> > only
> > > >> cover
> > > >> > > > ~50%
> > > >> > > > > of
> > > >> > > > > > >> all
> > > >> > > > > > >> Kafka users, and maybe 10% of those are the ones who
> > > >>will
> > > >> use
> > > >> > > the
> > > >> > > > > > >>REST
> > > >> > > > > > >> proxy. The remaining 50% are non-java client users
> (C,
> > > >> > python,
> > > >> > > > go,
> > > >> > > > > > >>node
> > > >> > > > > > >> etc)."
> > > >> > > > > > >>
> > > >> > > > > > >> REST API is most often asked feature in my interactions
> > > >>with
> > > >> > Kafka
> > > >> > > > > > >>users.
> > > >> > > > > > >> In an organization, There will be independent teams who
> > > >>will
> > > >> > write
> > > >> > > > > their
> > > >> > > > > > >> Kafka clients using different language libraries
> > available
> > > >> > today,
> > > >> > > > and
> > > >> > > > > > >> there is no way to standardize this. Instead of
> > supporting
> > > >> > several
> > > >> > > > > > >> different client libraries users will be interested in
> > > >>using
> > > >a
> > > >> > > REST
> > > >> > > > > API
> > > >> > > > > > >> server. The need for a REST API will only increase as
> > more
> > > >and
> > > >> > > more
> > > >> > > > > > >>users
> > > >> > > > > > >> start using Kafka.
> > > >> > > > > > >>
> > > >> > > > > > >> "More surface area means more work to keep things
> > > >>consistent.
> > > >> > > > Failure
> > > >> > > > > > >> to do that has, in fact, hurt the user experience."
> > > >> > > > > > >> Having myriad Kafka client GitHub projects that support
> > > >> > different
> > > >> > > > > > >>languages
> > > >> > > > > > >> hurts the user experience and pushes burden to maintain
> > > >>these
> > > >> > > > > libraries.
> > > >> > > > > > >> REST API is a simple code base that uses existing java
> > > >>client
> > > >> > > > > libraries
> > > >> > > > > > >>to
> > > >> > > > > > >> make life easier for the users.
> > > >> > > > > > >>
> > > >> > > > > > >> Thanks,
> > > >> > > > > > >> Harsha
> > > >> > > > > > >>
> > > >> > > > > > >> On Sat, Oct 1, 2016 at 10:41 AM Neha Narkhede <
> > > >> > n...@confluent.io>
> > > >> > > > > > wrote:
> > > >> > > > > > >>
> > > >> > > > > > >> > Manikumar,
> > > >> > > > > > >> >
> > > >> > > > > > >> > Thanks for sharing the proposal. I think there are 2
> > > >>parts
> > > >> to
> > > >> > > this
> > > >> > > > > > >> > discussion -
> > > >> > > > > > >> >
> > > >> > > > > > >> > 1. Should we rewrite a REST proxy given that there
> is a
> > > >> > > > > > >>feature-complete,
> > > >> > > > > > >> > open-source and actively maintained REST proxy in the
> > > >> > community?
> > > >> > > > > > >> > 2. Does adding a REST proxy to Apache Kafka make us
> > more
> > > >> agile
> > > >> > > and
> > > >> > > > > > >> maintain
> > > >> > > > > > >> > the high-quality experience that Kafka users have
> > today?
> > > >> > > > > > >> >
> > > >> > > > > > >> > For 1, I don't think there is value in giving in to
> the
> > > >>NIH
> > > >> > > > syndrome
> > > >> > > > > > >>and
> > > >> > > > > > >> > reinventing the wheel. What I'm looking for is a
> > detailed
> > > >> > > > comparison
> > > >> > > > > > >>of
> > > >> > > > > > >> the
> > > >> > > > > > >> > gaps and why those can't be improved in the REST
> proxy
> > > >>that
> > > >> > > > already
> > > >> > > > > > >> exists
> > > >> > > > > > >> > and is actively maintained. For example, we depend on
> > > >> zkClient
> > > >> > > and
> > > >> > > > > > >>have
> > > >> > > > > > >> > found as well as fixed several bugs by working
> closely
> > > >>with
> > > >> > the
> > > >> > > > > people
> > > >> > > > > > >> who
> > > >> > > > > > >> > maintain zkClient. This should be possible for REST
> > proxy
> > > >as
> > > >> > > well,
> > > >> > > > > > >>right?
> > > >> > > > > > >> >
> > > >> > > > > > >> > For 2, I'd like us to review our history of expanding
> > the
> > > >> > > surface
> > > >> > > > > > >>area to
> > > >> > > > > > >> > add more clients in the past. Here is a summary -
> > > >> > > > > > >> >
> > > >> > > > > > >> > - This doesn't materially have an impact on
> > expanding
> > > >the
> > > >> > > > > > >>usability of
> > > >> > > > > > >> > Kafka. In my experience, REST proxy + Java clients
> > > >>only
> > > >> > cover
> > > >> > > > > ~50%
> > > >> > > > > > >>of
> > > >> > > > > > >> > all
> > > >> > > > > > >> > Kafka users, and maybe 10% of those are the ones
> who
> > > >will
> > > >> > use
> > > >> > > > the
> > > >> > > > > > >>REST
> > > >> > > > > > >> > proxy. The remaining 50% are non-java client users
> > (C,
> > > >> > > python,
> > > >> > > > > go,
> > > >> > > > > > >> node
> > > >> > > > > > >> > etc).
> > > >> > > > > > >> > - People are a lot more excited about promising to
> > > >> > contribute
> > > >> > > > > while
> > > >> > > > > > >> > adding the surface area but not on an ongoing
> basis
> > > >>down
> > > >> > the
> > > >> > > > > line.
> > > >> > > > > > >> > - More surface area means more work to keep things
> > > >> > > consistent.
> > > >> > > > > > >>Failure
> > > >> > > > > > >> > to do that has, in fact, hurt the user experience.
> > > >> > > > > > >> > - More surface area hurts agility. We want to do a
> > few
> > > >> > things
> > > >> > > > > > >>really
> > > >> > > > > > >> > well as well as be agile to be able to build on
> our
> > > >>core
> > > >> > > > > > >>competency.
> > > >> > > > > > >> >
> > > >> > > > > > >> >
> > > >> > > > > > >> > On Sat, Oct 1, 2016 at 5:38 AM Manikumar <
> > > >> > > > manikumar.re...@gmail.com
> > > >> > > > > >
> > > >> > > > > > >> > wrote:
> > > >> > > > > > >> >
> > > >> > > > > > >> > > Hi Jay,
> > > >> > > > > > >> > >
> > > >> > > > > > >> > > Thanks for your reply.
> > > >> > > > > > >> > >
> > > >> > > > > > >> > > I agree that we can not add all the clients/tools
> > > >> available
> > > >> > in
> > > >> > > > > > >> ecosystem
> > > >> > > > > > >> > > page to Kafka repo itself. But we feel REST
> Interface
> > > >>is
> > > >> > > > different
> > > >> > > > > > >>from
> > > >> > > > > > >> > > other clients/tools. Since any language that can
> work
> > > >with
> > > >> > > HTTP
> > > >> > > > > can
> > > >> > > > > > >> > > easily integrate with this interface, Having an
> > > >"official"
> > > >> > > REST
> > > >> > > > > > >> > > interface helps user community. This also helps us
> to
> > > >> > > integrate
> > > >> > > > > well
> > > >> > > > > > >> > > with external management and provisioning tools.
> > > >>Apache
> > > >> > Kafka
> > > >> > > > > > >>release
> > > >> > > > > > >> > > with Java clients + REST interface is sufficient
> for
> > > >>most
> > > >> of
> > > >> > > the
> > > >> > > > > > >>user
> > > >> > > > > > >> > > deployments/requirements. This helps users to deal
> > with
> > > >> less
> > > >> > > > > number
> > > >> > > > > > >> > > of distributions/builds.
> > > >> > > > > > >> > >
> > > >> > > > > > >> > > Thanks,
> > > >> > > > > > >> > > Manikumar
> > > >> > > > > > >> > >
> > > >> > > > > > >> > >
> > > >> > > > > > >> > > On Sat, Oct 1, 2016 at 4:24 AM, Jay Kreps <
> > > >> j...@confluent.io
> > > >> > >
> > > >> > > > > wrote:
> > > >> > > > > > >> > >
> > > >> > > > > > >> > > > Hey guys,
> > > >> > > > > > >> > > >
> > > >> > > > > > >> > > > There's already a REST interface maintained as a
> > > >> separate
> > > >> > > > > > >> project--it's
> > > >> > > > > > >> > > > open source and apache licensed and actively
> > > >>maintained
> > > >> (
> > > >> > > > > > >> > > > https://github.com/confluentinc/kafka-rest).
> What
> > is
> > > >> > wrong
> > > >> > > > with
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > GitHub - confluentinc/kafka-rest: REST Proxy for Kafka
> > > >> > > > > > github.com
> > > >> > > > > > The Kafka REST Proxy provides a RESTful interface to a
> Kafka
> > > >> > cluster.
> > > >> > > > It
> > > >> > > > > > makes it easy to produce and consume messages, view the
> > state
> > > >>of
> > > >> > the
> > > >> > > > > > cluster, and ...
> > > >> > > > > >
> > > >> > > > > > >> that?
> > > >> > > > > > >> > > You
> > > >> > > > > > >> > > > mentioned that there was some compatibility
> > concern,
> > > >but
> > > >> > > > > > >> compatibility
> > > >> > > > > > >> > > has
> > > >> > > > > > >> > > > to do with the consumer protocol guarantees not
> the
> > > >repo
> > > >> > the
> > > >> > > > > code
> > > >> > > > > > >>is
> > > >> > > > > > >> > in,
> > > >> > > > > > >> > > > right? Not sure that concern makes sense.
> > > >> > > > > > >> > > >
> > > >> > > > > > >> > > > We could argue for adding pretty much anything
> and
> > > >> > > everything
> > > >> > > > in
> > > >> > > > > > >>the
> > > >> > > > > > >> > > > ecosystem page in Kafka itself but I'm not sure
> > that
> > > >> would
> > > >> > > > make
> > > >> > > > > > >>the
> > > >> > > > > > >> > > project
> > > >> > > > > > >> > > > more agile.
> > > >> > > > > > >> > > >
> > > >> > > > > > >> > > > -Jay
> > > >> > > > > > >> > > >
> > > >> > > > > > >> > > > On Wed, Sep 28, 2016 at 12:04 AM, Manikumar <
> > > >> > > > > > >> manikumar.re...@gmail.com
> > > >> > > > > > >> > >
> > > >> > > > > > >> > > > wrote:
> > > >> > > > > > >> > > >
> > > >> > > > > > >> > > > > Hi Kafka Devs,
> > > >> > > > > > >> > > > >
> > > >> > > > > > >> > > > > I created KIP-80 to add Kafka REST Server to
> > Kafka
> > > >> > > > Repository.
> > > >> > > > > > >> > > > >
> > > >> > > > > > >> > > > > There are already open-source alternatives are
> > > >> > available.
> > > >> > > > But
> > > >> > > > > > >>we
> > > >> > > > > > >> > would
> > > >> > > > > > >> > > > > like to add REST server that
> > > >> > > > > > >> > > > > many users ask for under Apache Kafka repo.
> Many
> > > >>data
> > > >> > > Infra
> > > >> > > > > > >>tools
> > > >> > > > > > >> > comes
> > > >> > > > > > >> > > > up
> > > >> > > > > > >> > > > > with Rest Interface.
> > > >> > > > > > >> > > > > It is useful to have inbuilt Rest API support
> for
> > > >> > Produce,
> > > >> > > > > > >>Consume
> > > >> > > > > > >> > > > messages
> > > >> > > > > > >> > > > > and admin interface for
> > > >> > > > > > >> > > > > integrating with external management and
> > > >>provisioning
> > > >> > > > > tools.This
> > > >> > > > > > >> will
> > > >> > > > > > >> > > > also
> > > >> > > > > > >> > > > > allow the maintenance of
> > > >> > > > > > >> > > > > REST server and adding new features makes it
> easy
> > > >> > because
> > > >> > > > > apache
> > > >> > > > > > >> > > > community.
> > > >> > > > > > >> > > > >
> > > >> > > > > > >> > > > > The KIP wiki is the following:
> > > >> > > > > > >> > > > > https://cwiki.apache.org/
> > > >> confluence/display/KAFKA/KIP-
> > > >> > > > > > >> > > > > 80%3A+Kafka+Rest+Server
> > > >> > > > > > >> > > > >
> > > >> > > > > > >> > > > > Your comments and feedback are welcome.
> > > >> > > > > > >> > > > >
> > > >> > > > > > >> > > > > Thanks,
> > > >> > > > > > >> > > > > Manikumar
> > > >> > > > > > >> > > > >
> > > >> > > > > > >> > > >
> > > >> > > > > > >> > >
> > > >> > > > > > >> > --
> > > >> > > > > > >> > Thanks,
> > > >> > > > > > >> > Neha
> > > >> > > > > > >> >
> > > >> > > > > > >>
> > > >> > > > > > The information contained in this email is strictly
> > > >>confidential
> > > >> > and
> > > >> > > > for
> > > >> > > > > > the use of the addressee only, unless otherwise indicated.
> > If
> > > >you
> > > >> > are
> > > >> > > > not
> > > >> > > > > > the intended recipient, please do not read, copy, use or
> > > >disclose
> > > >> > to
> > > >> > > > > others
> > > >> > > > > > this message or any attachment. Please also notify the
> > sender
> > > >>by
> > > >> > > > replying
> > > >> > > > > > to this email or by telephone (+44(020 7896 0011) and then
> > > >delete
> > > >> > the
> > > >> > > > > email
> > > >> > > > > > and any copies of it. Opinions, conclusion (etc) that do
> not
> > > >> relate
> > > >> > > to
> > > >> > > > > the
> > > >> > > > > > official business of this company shall be understood as
> > > >>neither
> > > >> > > given
> > > >> > > > > nor
> > > >> > > > > > endorsed by it. IG is a trading name of IG Markets Limited
> > (a
> > > >> > company
> > > >> > > > > > registered in England and Wales, company number 04008957)
> > and
> > > >>IG
> > > >> > > Index
> > > >> > > > > > Limited (a company registered in England and Wales,
> company
> > > >> number
> > > >> > > > > > 01190902). Registered address at Cannon Bridge House, 25
> > > >>Dowgate
> > > >> > > Hill,
> > > >> > > > > > London EC4R 2YA. Both IG Markets Limited (register number
> > > >195355)
> > > >> > and
> > > >> > > > IG
> > > >> > > > > > Index Limited (register number 114059) are authorised and
> > > >> regulated
> > > >> > > by
> > > >> > > > > the
> > > >> > > > > > Financial Conduct Authority.
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > > The information contained in this email is strictly confidential
> > and
> > > >> for
> > > >> > > the use of the addressee only, unless otherwise indicated. If
> you
> > > >>are
> > > >> not
> > > >> > > the intended recipient, please do not read, copy, use or
> disclose
> > to
> > > >> > others
> > > >> > > this message or any attachment. Please also notify the sender by
> > > >> replying
> > > >> > > to this email or by telephone (+44(020 7896 0011) and then
> delete
> > > >>the
> > > >> > email
> > > >> > > and any copies of it. Opinions, conclusion (etc) that do not
> > relate
> > > >>to
> > > >> > the
> > > >> > > official business of this company shall be understood as neither
> > > >>given
> > > >> > nor
> > > >> > > endorsed by it. IG is a trading name of IG Markets Limited (a
> > > >>company
> > > >> > > registered in England and Wales, company number 04008957) and IG
> > > >>Index
> > > >> > > Limited (a company registered in England and Wales, company
> number
> > > >> > > 01190902). Registered address at Cannon Bridge House, 25 Dowgate
> > > >>Hill,
> > > >> > > London EC4R 2YA. Both IG Markets Limited (register number
> 195355)
> > > >>and
> > > >> IG
> > > >> > > Index Limited (register number 114059) are authorised and
> > regulated
> > > >>by
> > > >> > the
> > > >> > > Financial Conduct Authority.
> > > >> > >
> > > >> >
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Nacho (Ignacio) Solis
> > > >> Kafka
> > > >> nso...@linkedin.com
> > > >>
> > >
> > >
> > >
> >
>
> --
>
>
> This email, including attachments, is private and confidential. If you have
> received this email in error please notify the sender and delete it from
> your system. Emails are not secure and may contain viruses. No liability
> can be accepted for viruses that might be transferred by this email or any
> attachment. Any unauthorised copying of this message or unauthorised
> distribution and publication of the information contained herein are
> prohibited.
>
> 7digital Limited. Registered office: 69 Wilson Street, London EC2A 2BB.
> Registered in England and Wales. Registered No. 04843573.
>
