Great idea Gwen! I think it would go a long way to making this work.  Issue
created :)

On Tue, Aug 19, 2014 at 4:33 PM, Gwen Shapira <gshap...@cloudera.com> wrote:

> Does it make sense to merge the Camus mailing list? (i.e. ask the
> Camus community to merge?) Its a fairly large and popular client.
>
> On Tue, Aug 19, 2014 at 1:27 PM, Joe Stein <joe.st...@stealth.ly> wrote:
> > I also opened issues on 3 of the clients on github that I frequently
> > use/involved in often enough.... would be great to get on the README as
> > such.
> >
> > Thanks to the community for driving things along!
> >
> > /*******************************************
> >  Joe Stein
> >  Founder, Principal Consultant
> >  Big Data Open Source Security LLC
> >  http://www.stealth.ly
> >  Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> > ********************************************/
> >
> >
> > On Tue, Aug 19, 2014 at 4:22 PM, Joe Stein <joe.st...@stealth.ly> wrote:
> >
> >> I just joined too, and tweeted.
> >>
> >> /*******************************************
> >>  Joe Stein
> >>  Founder, Principal Consultant
> >>  Big Data Open Source Security LLC
> >>  http://www.stealth.ly
> >>  Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> >> ********************************************/
> >>
> >>
> >> On Tue, Aug 19, 2014 at 4:08 PM, Jay Kreps <jay.kr...@gmail.com> wrote:
> >>
> >>> Cool. I just joined. I'll add it to the website so others can find it.
> >>> If someone was willing to ping some of the other client developers and
> >>> get them to join as well that would probably give us critical mass.
> >>>
> >>> -Jay
> >>>
> >>> On Tue, Aug 19, 2014 at 9:08 AM, Dana Powers <dana.pow...@rd.io>
> wrote:
> >>> > I created kafka-clie...@groups.google.com
> >>> >
> >>> > https://groups.google.com/forum/m/#!forum/kafka-clients
> >>> >
> >>> > No members and no guidelines yet, but it's a start.  Would love to
> get
> >>> this
> >>> > going.
> >>> >
> >>> > Dana
> >>> >  On Aug 19, 2014 9:03 AM, "Mark Roberts" <wiz...@gmail.com> wrote:
> >>> >
> >>> >> Did this mailing list ever get created? Was there consensus that it
> >>> did or
> >>> >> didn't need created?
> >>> >>
> >>> >> -Mark
> >>> >>
> >>> >> > On Jul 18, 2014, at 14:34, Jay Kreps <jay.kr...@gmail.com> wrote:
> >>> >> >
> >>> >> > A question was asked in another thread about what was an effective
> >>> way
> >>> >> > to contribute to the Kafka project for people who weren't very
> >>> >> > enthusiastic about writing Java/Scala code.
> >>> >> >
> >>> >> > I wanted to kind of advocate for an area I think is really
> important
> >>> >> > and not as good as it could be--the client ecosystem. I think our
> >>> goal
> >>> >> > is to make Kafka effective as a general purpose, centralized, data
> >>> >> > subscription system. This vision only really works if all your
> >>> >> > applications, are able to integrate easily, whatever language they
> >>> are
> >>> >> > in.
> >>> >> >
> >>> >> > We have a number of pretty good non-java producers. We have been
> >>> >> > lacking the features on the server-side to make writing non-java
> >>> >> > consumers easy. We are fixing that right now as part of the
> consumer
> >>> >> > work going on right now (which moves a lot of the functionality in
> >>> the
> >>> >> > java consumer to the server side).
> >>> >> >
> >>> >> > But apart from this I think there may be a lot more we can do to
> make
> >>> >> > the client ecosystem better.
> >>> >> >
> >>> >> > Here are some concrete ideas. If anyone has additional ideas
> please
> >>> >> > reply to this thread and share them. If you are interested in
> picking
> >>> >> > any of these up, please do.
> >>> >> >
> >>> >> > 1. The most obvious way to improve the ecosystem is to help work
> on
> >>> >> > clients. This doesn't necessarily mean writing new clients, since
> in
> >>> >> > many cases we already have a client in a given language. I think
> any
> >>> >> > way we can incentivize fewer, better clients rather than many
> >>> >> > half-working clients we should do. However we are working now on
> the
> >>> >> > server-side consumer co-ordination so it should now be possible to
> >>> >> > write much simpler consumers.
> >>> >> >
> >>> >> > 2. It would be great if someone put together a mailing list just
> for
> >>> >> > client developers to share tips, tricks, problems, and so on. We
> can
> >>> >> > make sure all the main contributors on this too. I think this
> could
> >>> be
> >>> >> > a forum for kind of directing improvements in this area.
> >>> >> >
> >>> >> > 3. Help improve the documentation on how to implement a client. We
> >>> >> > have tried to make the protocol spec not just a dry document but
> also
> >>> >> > have it share best practices, rationale, and intentions. I think
> this
> >>> >> > could potentially be even better as there is really a range of
> >>> options
> >>> >> > from a very simple quick implementation to a more complex highly
> >>> >> > optimized version. It would be good to really document some of the
> >>> >> > options and tradeoffs.
> >>> >> >
> >>> >> > 4. Come up with a standard way of documenting the features of
> >>> clients.
> >>> >> > In an ideal world it would be possible to get the same information
> >>> >> > (author, language, feature set, download link, source code, etc)
> for
> >>> >> > all clients. It would be great to standardize the documentation
> for
> >>> >> > the client as well. For example having one or two basic examples
> that
> >>> >> > are repeated for every client in a standardized way. This would
> let
> >>> >> > someone come to the Kafka site who is not a java developer, and
> click
> >>> >> > on the link for their language and view examples of interacting
> with
> >>> >> > Kafka in the language they know using the client they would
> >>> eventually
> >>> >> > use.
> >>> >> >
> >>> >> > 5. Build a Kafka Client Compatibility Kit (KCCK) :-) The idea is
> >>> this:
> >>> >> > anyone who wants to implement a client would implement a simple
> >>> >> > command line program with a set of standardized options. The
> >>> >> > compatibility kit would be a standard set of scripts that ran
> their
> >>> >> > client using this command line driver and validate its behavior.
> E.g.
> >>> >> > for a producer it would test that it correctly can send messages,
> >>> that
> >>> >> > the ordering is retained, that the client correctly handles
> >>> >> > reconnection and metadata refresh, and compression. The output
> would
> >>> >> > be a list of features that passed are certified, and perhaps basic
> >>> >> > performance information. This would be an easy way to help client
> >>> >> > developers write correct clients, as well as having a standardized
> >>> >> > comparison for the clients that says that they work correctly.
> >>> >> >
> >>> >> > -Jay
> >>> >>
> >>>
> >>
> >>
>

Reply via email to