It use the default one, which is uncomitted earliest
On Wed, Aug 23, 2017 at 7:34 AM, Dan Markhasin wrote:
> Is your storm consumer set to auto.offset.reset="earliest"?
>
> On 22 August 2017 at 10:05, Elyahou Ittah wrote:
>
> > I checked the __consumer_offsets topic and here is an extraction fr
I have question ,
if my service went down and i want to consume back whichever was not
consumed from a given consumer group ,
how to handle that ?
its either earliest or latest message , how about consuming all un-commited
messages ?
Keep learning keep moving .
On Wed, Aug 23, 2017 at 10:
Is your storm consumer set to auto.offset.reset="earliest"?
On 22 August 2017 at 10:05, Elyahou Ittah wrote:
> I checked the __consumer_offsets topic and here is an extraction from this
> log for the same consumer group, a specific topic (users) and specific
> partition (15):
>
> [storm_kafka_to
I checked the __consumer_offsets topic and here is an extraction from this
log for the same consumer group, a specific topic (users) and specific
partition (15):
[storm_kafka_topology,users,15]::[OffsetMetadata[8327,{topic-partition=users-15,
offset=8327, numFails=0, thread='Thread-11-kafkaSpout-e
attached log file
On Mon, Aug 21, 2017 at 11:06 AM, Elyahou Ittah
wrote:
> I am consuming from kafka using KafkaSpout of Storm and also in ruby using
> ruby-kafka gem (both use new consumer API).
>
> I noticed that after a rolling restart of the kafka cluster. The
> kafkaSpout reconsumed all kaf
I am consuming from kafka using KafkaSpout of Storm and also in ruby using
ruby-kafka gem (both use new consumer API).
I noticed that after a rolling restart of the kafka cluster. The kafkaSpout
reconsumed all kafka messages ignoring the committed offsets...
What can cause this behavior ?
Attach
