Hi Shushant, Write throughput on zoookeeper can be a problem depending on your commit policy. Typically you can handle this by just committing less frequently (with the obvious tradeoff). The consumer also supports storing offsets in Kafka itself through the "offsets.storage" option (see http://kafka.apache.org/documentation.html#consumerconfigs). With the new consumer (not released yet), this is actually the only choice.
By default, the consumer does not guarantee that duplicate messages won't be read. There is a good section in the consumer documentation that describes Kafka's message delivery semantics: http://kafka.apache.org/documentation.html#semantics. -Jason On Sat, Jun 27, 2015 at 7:51 AM, Shushant Arora <shushantaror...@gmail.com> wrote: > Hi > > Is zookeeper not fit for high writes? since write goes via leader > node then how kafka maintains offsets using zookeeper update operation very > efficiently. > > If there are a lot of subscribers you may feel a squeeze > though. > > zookeeper is also not strictly consistent then how its guranteed cosumers > won't read duplicate messages in high level api? >
