auto.offset.reset is honoured if the consumer group has not committed
offsets yet, or if the offsets expired (I think this is
offsets.retention.*).
Otherwise the last committed offsets should be read for that group.
Harald.
On 07.12.2016 18:48, Mohit Anchlia wrote:
Is auto.offset.reset honored just the first time consumer starts and
polling? In other words everytime consumer starts does it start from the
beginning even if it has already read those messages?
On Wed, Dec 7, 2016 at 1:43 AM, Harald Kirsch <harald.kir...@raytion.com>
wrote:
Have you defined
auto.offset.reset: earliest
or otherwise made sure (KafkaConsumer.position()) that the consumer does
not just wait for *new* messages to arrive?
Harald.
On 06.12.2016 20:11, Mohit Anchlia wrote:
I see this message in the logs:
[2016-12-06 13:54:16,586] INFO [GroupCoordinator 0]: Preparing to
restabilize group DemoConsumer with old generation 3
(kafka.coordinator.GroupCoordinator)
On Tue, Dec 6, 2016 at 10:53 AM, Mohit Anchlia <mohitanch...@gmail.com>
wrote:
I have a consumer polling a topic of Kafka 0.10. Even though the topic has
messages the consumer poll is not fetching the message. The thread dump
reveals:
"main" #1 prio=5 os_prio=0 tid=0x00007f3ba4008800 nid=0x798 runnable
[0x00007f3baa6c3000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.
java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0x00000006c6d1f8b8> (a sun.nio.ch.Util$3)
- locked <0x00000006c6d1f8a8> (a java.util.Collections$
UnmodifiableSet)
- locked <0x00000006c6d1f0b8> (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.apache.kafka.common.network.Selector.select(
Selector.java:470)
at org.apache.kafka.common.network.Selector.poll(
Selector.java:286)
at org.apache.kafka.clients.NetworkClient.poll(
NetworkClient.java:260)
at org.apache.kafka.clients.consumer.internals.
ConsumerNetworkClient.poll(ConsumerNetworkClient.java:232)
- locked <0x00000006c6d1eff8> (a org.apache.kafka.clients.
consumer.internals.ConsumerNetworkClient)
at org.apache.kafka.clients.consumer.KafkaConsumer.
pollOnce(KafkaConsumer.java:1031)
