Makes sense. -Jay
On Fri, Jul 19, 2013 at 5:25 PM, David Arthur <mum...@gmail.com> wrote: > I think there would be many advantages to having a REST API in Kafka, as a > core component. Not just for data services (producing/consuming messages), > but also for admin. I could imagine a Kafka web console that showed various > stats, allowed for maintenance operations, showed pretty viz of the > cluster, etc. > > The REST prototype I put together using Jetty was really a simplistic POC. > In reality, more modern web techniques like chunked transfer encoding and > streaming should be used for high throughput (in, and out). Netty would > probably be a good choice for a networking library. > > I'd be willing to bet that a sufficiently high performance and easy to use > REST API could put a lot of the 3rd party client libraries out of business, > so-to-speak. > > To more directly answer your question, Jay, I think if something like this > is done it should be done as a top-level component in Kafka. Is contrib > even maintained/updated anymore? Hasn't Camus supplanted the hadoop stuff? > > > On 7/19/13 4:24 PM, Jay Kreps wrote: > >> I think this is really cool and would be useful for a lot of people. >> David, >> do you think it would really benefit from being a contrib package in kafka >> versus a separately hosted github project? >> >> Here was our experience: contrib packages tend to get added but the >> authors >> and don't necessarily want involvement in kafka internals or necessarily >> follow the mailing list to answer questions. When we take then they end up >> kind of stagnating because there is overhead for the code review process, >> and the main kafka committers aren't generally qualified to fully maintain >> the code. >> >> What we have seen is that external projects that host add ons do really >> well. The author tends to get more credit for their work, directly >> responds >> to questions from users, etc. >> >> On the other hand maybe we feel this is central enough that it should just >> be a top-level package in kafka. I think this makes sense for things which >> are closely tied to internals, are general enough that there is only one >> good way to do it, and will be very common for kafka users to want. If >> this >> sounds better for the REST proxy the important thing is just that someone >> has bandwidth to be able to support it and ensure we we document it, have >> sufficient tests to keep it in sync, etc. >> >> Let me know what you think, if the preference is to add it to kafka then I >> can help with code reviews and stuff. >> >> -Jay >> >> >> On Thu, Jul 18, 2013 at 8:15 PM, David Arthur <mum...@gmail.com> wrote: >> >> I did some work on this a while ago: https://github.com/mumrah/** >>> kafka/commits/rest >>> <https://github.com/mumrah/**kafka/commits/rest<https://github.com/mumrah/kafka/commits/rest> >>> > >>> >>> >>> I think people would still be interested. I know I would :) >>> >>> -David >>> >>> >>> >>> On 7/18/13 6:19 PM, Yogita Bijani wrote: >>> >>> Hi, >>>> I am a newbie, I was just checking if there has been any updates on the >>>> REST proxy. >>>> I have tried to come up with a draft of REST API and can post it for >>>> review and discussion if it is still an open topic. >>>> >>>> Thanks >>>> Yogita >>>> >>>> Sent from my iPad >>>> >>>> >>> >
