Slight off-topic, but is it also possible to replay a specific number of
messages? For example, using the simple consumer, can I go back/reset the
offset so that I always go read the last 10 messages assuming the size of
each individual message could be different. All I found in the simple
consumer example was that replaying needs a byte parameter, but maybe I
didn't look hard enough.
--
Sharninder
On Thu, Sep 25, 2014 at 10:15 PM, pankaj ojha <pankajojh...@gmail.com>
wrote:
> Thank You. I will try this out.
>
> On Thu, Sep 25, 2014 at 10:01 PM, Gwen Shapira <gshap...@cloudera.com>
> wrote:
>
> > Using high level consumer and assuming you already created an iterator:
> >
> > while (msgCount < maxMessages && it.hasNext()) {
> > bytes = it.next().message();
> > eventList.add(bytes);
> > }
> >
> > (See a complete example here:
> >
> >
> https://github.com/apache/flume/blob/trunk/flume-ng-sources/flume-kafka-source/src/main/java/org/apache/flume/source/kafka/KafkaSource.java
> > )
> >
> > Gwen
> >
> > On Thu, Sep 25, 2014 at 9:15 AM, pankaj ojha <pankajojh...@gmail.com>
> > wrote:
> > > Hi,
> > >
> > > My requirement is to read a specific number of messages from kafka
> topic
> > > which contains data in json format and after reading number of
> messges, i
> > > need to write that in a file and then stop. How can I count number of
> > > messages read by my consumer code(either simpleconsumer or high level)
> ?
> > >
> > > Please help.
> > >
> > > --
> > > Thanks,
> > > Pankaj Ojha
> >
>
>
>
> --
> Thanks,
> Pankaj Ojha
>
