I created kafka-clie...@groups.google.com https://groups.google.com/forum/m/#!forum/kafka-clients
No members and no guidelines yet, but it's a start. Would love to get this going. Dana On Aug 19, 2014 9:03 AM, "Mark Roberts" <wiz...@gmail.com> wrote: > Did this mailing list ever get created? Was there consensus that it did or > didn't need created? > > -Mark > > > On Jul 18, 2014, at 14:34, Jay Kreps <jay.kr...@gmail.com> wrote: > > > > A question was asked in another thread about what was an effective way > > to contribute to the Kafka project for people who weren't very > > enthusiastic about writing Java/Scala code. > > > > I wanted to kind of advocate for an area I think is really important > > and not as good as it could be--the client ecosystem. I think our goal > > is to make Kafka effective as a general purpose, centralized, data > > subscription system. This vision only really works if all your > > applications, are able to integrate easily, whatever language they are > > in. > > > > We have a number of pretty good non-java producers. We have been > > lacking the features on the server-side to make writing non-java > > consumers easy. We are fixing that right now as part of the consumer > > work going on right now (which moves a lot of the functionality in the > > java consumer to the server side). > > > > But apart from this I think there may be a lot more we can do to make > > the client ecosystem better. > > > > Here are some concrete ideas. If anyone has additional ideas please > > reply to this thread and share them. If you are interested in picking > > any of these up, please do. > > > > 1. The most obvious way to improve the ecosystem is to help work on > > clients. This doesn't necessarily mean writing new clients, since in > > many cases we already have a client in a given language. I think any > > way we can incentivize fewer, better clients rather than many > > half-working clients we should do. However we are working now on the > > server-side consumer co-ordination so it should now be possible to > > write much simpler consumers. > > > > 2. It would be great if someone put together a mailing list just for > > client developers to share tips, tricks, problems, and so on. We can > > make sure all the main contributors on this too. I think this could be > > a forum for kind of directing improvements in this area. > > > > 3. Help improve the documentation on how to implement a client. We > > have tried to make the protocol spec not just a dry document but also > > have it share best practices, rationale, and intentions. I think this > > could potentially be even better as there is really a range of options > > from a very simple quick implementation to a more complex highly > > optimized version. It would be good to really document some of the > > options and tradeoffs. > > > > 4. Come up with a standard way of documenting the features of clients. > > In an ideal world it would be possible to get the same information > > (author, language, feature set, download link, source code, etc) for > > all clients. It would be great to standardize the documentation for > > the client as well. For example having one or two basic examples that > > are repeated for every client in a standardized way. This would let > > someone come to the Kafka site who is not a java developer, and click > > on the link for their language and view examples of interacting with > > Kafka in the language they know using the client they would eventually > > use. > > > > 5. Build a Kafka Client Compatibility Kit (KCCK) :-) The idea is this: > > anyone who wants to implement a client would implement a simple > > command line program with a set of standardized options. The > > compatibility kit would be a standard set of scripts that ran their > > client using this command line driver and validate its behavior. E.g. > > for a producer it would test that it correctly can send messages, that > > the ordering is retained, that the client correctly handles > > reconnection and metadata refresh, and compression. The output would > > be a list of features that passed are certified, and perhaps basic > > performance information. This would be an easy way to help client > > developers write correct clients, as well as having a standardized > > comparison for the clients that says that they work correctly. > > > > -Jay >
