I read through the documentation however when I try to access Java API through the link posted on the design page I get "no page found"
http://people.apache.org/~nehanarkhede/kafka-0.9-consumer-javadoc/doc/kafka/clients/consumer/KafkaConsumer.html On Wed, Oct 21, 2015 at 9:59 AM, Mohit Anchlia <mohitanch...@gmail.com> wrote: > never mind, I found the documentation > > On Wed, Oct 21, 2015 at 9:50 AM, Mohit Anchlia <mohitanch...@gmail.com> > wrote: > >> Thanks. Where can I find new Java consumer API documentation with >> examples? >> >> On Tue, Oct 20, 2015 at 6:37 PM, Guozhang Wang <wangg...@gmail.com> >> wrote: >> >>> There are a bunch of new features added in 0.9 plus quite a lot of bug >>> fixes as well, a complete ticket list can be found here: >>> >>> >>> https://issues.apache.org/jira/browse/KAFKA-1686?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%200.9.0.0%20ORDER%20BY%20updated%20DESC >>> >>> In a short summary of the new features, 0.9 introduces: >>> >>> 1) security and quota management on the brokers. >>> 2) new Java consumer. >>> 3) copycat framework for ingress / egress of Kafka. >>> >>> Guozhang >>> >>> On Tue, Oct 20, 2015 at 4:32 PM, Mohit Anchlia <mohitanch...@gmail.com> >>> wrote: >>> >>> > Thanks. Are there any other major changes in .9 release other than the >>> > Consumer changes. Should I wait for .9 or go ahead and performance test >>> > with .8? >>> > >>> > On Tue, Oct 20, 2015 at 3:54 PM, Guozhang Wang <wangg...@gmail.com> >>> wrote: >>> > >>> > > We will have a release document for that on the release date, it is >>> not >>> > > complete yet. >>> > > >>> > > Guozhang >>> > > >>> > > On Tue, Oct 20, 2015 at 3:18 PM, Mohit Anchlia < >>> mohitanch...@gmail.com> >>> > > wrote: >>> > > >>> > > > Is there a wiki page where I can find all the major design changes >>> in >>> > > > 0.9.0? >>> > > > >>> > > > On Mon, Oct 19, 2015 at 4:24 PM, Guozhang Wang <wangg...@gmail.com >>> > >>> > > wrote: >>> > > > >>> > > > > It is not released yet, we are shooting for Nov. for 0.9.0. >>> > > > > >>> > > > > Guozhang >>> > > > > >>> > > > > On Mon, Oct 19, 2015 at 4:08 PM, Mohit Anchlia < >>> > mohitanch...@gmail.com >>> > > > >>> > > > > wrote: >>> > > > > >>> > > > > > Is 0.9.0 still under development? I don't see it here: >>> > > > > > http://kafka.apache.org/downloads.html >>> > > > > > >>> > > > > > On Mon, Oct 19, 2015 at 4:05 PM, Guozhang Wang < >>> wangg...@gmail.com >>> > > >>> > > > > wrote: >>> > > > > > >>> > > > > > > The links you are referring are for the old consumer. >>> > > > > > > >>> > > > > > > If you are using the ZooKeeper based high-level version of >>> the >>> > old >>> > > > > > consumer >>> > > > > > > which is described in the second link, then failures are >>> handled >>> > > and >>> > > > > > > abstracted from you so that if there is a failure in the >>> current >>> > > > > process, >>> > > > > > > its fetching partitions will be re-assigned to other >>> consumers >>> > > within >>> > > > > the >>> > > > > > > same group starting at the last checkpointed offset. And >>> offsets >>> > > can >>> > > > be >>> > > > > > > either checkpointed periodically or manually throw >>> > > consumer.commit() >>> > > > > > calls. >>> > > > > > > >>> > > > > > > BTW, in the coming 0.9.0 release there is a new consumer >>> written >>> > in >>> > > > > Java >>> > > > > > > which uses a poll() based API instead of a stream iterating >>> API. >>> > > More >>> > > > > > > details can be found here in case you are interested in >>> trying it >>> > > > out: >>> > > > > > > >>> > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > > >>> > >>> https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Client+Re-Design >>> > > > > > > >>> > > > > > > Guozhang >>> > > > > > > >>> > > > > > > On Mon, Oct 19, 2015 at 2:54 PM, Mohit Anchlia < >>> > > > mohitanch...@gmail.com >>> > > > > > >>> > > > > > > wrote: >>> > > > > > > >>> > > > > > > > By old consumer you mean version < .8? >>> > > > > > > > >>> > > > > > > > Here are the links: >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > >>> > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > > >>> > >>> https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example >>> > > > > > > > >>> > > > > > >>> > > > >>> > >>> https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example >>> > > > > > > > >>> > > > > > > > On Mon, Oct 19, 2015 at 12:52 PM, Guozhang Wang < >>> > > > wangg...@gmail.com> >>> > > > > > > > wrote: >>> > > > > > > > >>> > > > > > > > > Hi Mohit, >>> > > > > > > > > >>> > > > > > > > > Are you referring to the new Java consumer or the old >>> > consumer? >>> > > > Or >>> > > > > > more >>> > > > > > > > > specifically what examples doc are you referring to? >>> > > > > > > > > >>> > > > > > > > > Guozhang >>> > > > > > > > > >>> > > > > > > > > On Mon, Oct 19, 2015 at 10:01 AM, Mohit Anchlia < >>> > > > > > > mohitanch...@gmail.com> >>> > > > > > > > > wrote: >>> > > > > > > > > >>> > > > > > > > > > I see most of the consumer examples create a while/for >>> loop >>> > > and >>> > > > > > then >>> > > > > > > > > fetch >>> > > > > > > > > > messages iteratively. Is that the only way by which >>> clients >>> > > can >>> > > > > > > > consumer >>> > > > > > > > > > messages? If this is the preferred way then how do you >>> deal >>> > > > with >>> > > > > > > > > failures, >>> > > > > > > > > > exceptions such that messages are not lost. >>> > > > > > > > > > >>> > > > > > > > > > Also, please point me to examples that one would >>> consider >>> > as >>> > > a >>> > > > > > robust >>> > > > > > > > way >>> > > > > > > > > > of coding consumers. >>> > > > > > > > > > >>> > > > > > > > > >>> > > > > > > > > >>> > > > > > > > > >>> > > > > > > > > -- >>> > > > > > > > > -- Guozhang >>> > > > > > > > > >>> > > > > > > > >>> > > > > > > >>> > > > > > > >>> > > > > > > >>> > > > > > > -- >>> > > > > > > -- Guozhang >>> > > > > > > >>> > > > > > >>> > > > > >>> > > > > >>> > > > > >>> > > > > -- >>> > > > > -- Guozhang >>> > > > > >>> > > > >>> > > >>> > > >>> > > >>> > > -- >>> > > -- Guozhang >>> > > >>> > >>> >>> >>> >>> -- >>> -- Guozhang >>> >> >> >
