Re: Questions about manage offset in external storage and consumer failure detect

2018-09-19 Thread Matthias J. Sax
1. If you don't have a good reason to store offsets externally, I would not recommend it, but use the client's built-in mechanism. It will be more work (ie, code you need to write) if you store offsets externally. For some use-cases, it's beneficial to store the offsets in an external system to ge

Questions about manage offset in external storage and consumer failure detect

2018-09-18 Thread ????????????
Hi, I have two questions about Kafka consumer api: 1. I read the section "Storing Offsets Outside Kafka" in jdoc of KafkaConsumer. Is it safer or more efficient to manage offset with external storage compared to manage with Kafka(Zookeeper or topic _consumer_offset)? Or is there any benefit f