I successfully processed my data by resetting manually my topic offsets on
ZK.
If it may help someone, here's my steps :
Make sure you stop all your consumers before doing that, otherwise they
overwrite the new offsets you wrote
set /consumers/{yourConsumerGroup}/offsets/{yourFancyTopic}/{partit
Thanks for your quick answer.
If I set "auto.offset.reset" to "smallest" as for KafkaParams like this
val kafkaParams = Map[String, String](
"metadata.broker.list" -> brokers,
"group.id" -> groupId,
"auto.offset.reset" -> "smallest"
)
And then use :
val streams = KafkaUtils.createStream(ssc,
You could set this configuration "auto.offset.reset" through parameter
"kafkaParams" which is provided in some other overloaded APIs of
createStream.
By default Kafka will pick data from latest offset unless you explicitly
set it, this is the behavior Kafka, not Spark.
Thanks
Saisai
On Mon, Feb
