You can use any offset you like to fetch (provided it is present on
the broker). Since the client-side FetchRequest and the server-side
Log API don't support reading a specific _number_ of messages you will
need to specify the bytes (size) to read. You can then extract as many
messages as you like
Slight off-topic, but is it also possible to replay a specific number of
messages? For example, using the simple consumer, can I go back/reset the
offset so that I always go read the last 10 messages assuming the size of
each individual message could be different. All I found in the simple
consumer
Thank You. I will try this out.
On Thu, Sep 25, 2014 at 10:01 PM, Gwen Shapira
wrote:
> Using high level consumer and assuming you already created an iterator:
>
> while (msgCount < maxMessages && it.hasNext()) {
> bytes = it.next().message();
> eventList.add(bytes);
> }
>
> (See a complete ex
Using high level consumer and assuming you already created an iterator:
while (msgCount < maxMessages && it.hasNext()) {
bytes = it.next().message();
eventList.add(bytes);
}
(See a complete example here:
https://github.com/apache/flume/blob/trunk/flume-ng-sources/flume-kafka-source/src/main/jav
Hi,
My requirement is to read a specific number of messages from kafka topic
which contains data in json format and after reading number of messges, i
need to write that in a file and then stop. How can I count number of
messages read by my consumer code(either simpleconsumer or high level) ?
Ple
