Currently Kafka depends on zookeeper 3.3.4 that doesn't have a batch write api. So if you commit after every message at a high rate, it will be slow and inefficient. Besides it will cause zookeeper performance to degrade.
Thanks, Neha On May 16, 2013 6:54 PM, "Rob Withers" <reefed...@gmail.com> wrote: > We are calling commitOffsets after every message consumption. It looks to > be ~60% slower, with 29 partitions. If a single KafkaStream thread is from > a connector, and there are 29 partitions, then commitOffsets sends 29 > offset updates, correct? Are these offset updates batched in one send to > zookeeper? > > thanks, > rob
