For that reasons I am now attempting to use your new, not released yet, 0.9 consumer API - I have an impression that it is more or less stable at the moment, am I right? Can you estimate when will you finish and release version 0.9?
The API is checked into trunk and I would not rule out minor changes to it before the 0.9 release. I suspect the 0.9 release is roughly 3 months away. Thanks, Neha On Sun, Jun 15, 2014 at 11:28 PM, Marta Kacperek < m.j.kacpe...@student.tudelft.nl> wrote: > Dear Kafka developers and users, > > I am currently working on creating a small library that would enable using > Reactive Extensions (Rx) with Kafka as data source. I started with what > seemed most reasonable for me: using Kafka's Java API and wrapping Kafka > producers and consumers with RxJava interfaces ( > https://github.com/Netflix/RxJava/wiki). > > One of my problems is version of your consumer API to use... Till now I > have written basic bindings using version 0.8.1.1 and "high level consumer > API", but then control over committing offsets is limited. On the other > hand, I'm not so keen to invest my time in designing and writing library > that would use this "simple/low level" API, dealing with partitioning, > offsets and leader election, while it is soon to be replaced. > > For that reasons I am now attempting to use your new, not released yet, > 0.9 consumer API - I have an impression that it is more or less stable at > the moment, am I right? Can you estimate when will you finish and release > version 0.9? > > One on the suggestions I found in this document: > https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Client+Re-Design > is that new consumer client should enable non-blocking, asynchronous data > access (and from what I've seen - it's not implemented yet). This is > exactly what Reactive Extensions are about. Rx wrappers/bindings for Kafka > API would give users very clean and convenient access to Kafka's data > asynchronously, using observables (push-based collections). All kinds of > operations on streams (modifications, filtering, merging and so on) are > also enabled by Rx (see https://github.com/Netflix/RxJava/wiki for > details). > > Maybe you can consider adding Rx bindings as a new feature for Kafka > consumer/producer API? Of course, that would require adding RxJava as > dependency to Kafka client library, but RxJava is lightweight and has no > dependencies of its own. And this would solve your design problem with > non-blocking kafka access. > > My library code is still under development, especially this using new > consumer API (I have some troubles with testing, but this is a different > thing...), and I would really appreciate some feedback from you at this > stage. Any advice or comments and are welcome. > > Best regards, > Marta Kacperek
