[
https://issues.apache.org/jira/browse/KAFKA-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13720886#comment-13720886
]
Neha Narkhede commented on KAFKA-987:
-------------------------------------
I'm trying to see if I understand what you are saying here.
1. The basic logic is that as long as the consumer rebalances before starting
consumption, the offset cache will be updated. This is true for the zookeeper
consumer behavior today. Now, it really doesn't matter much where the consumer
starts consuming from. If it hasn't read any messages, there is no need to
update offsets in zookeeper. If it reads messages, the offsets will be
different from what's in the cache, so they will get checkpointed.
2. I don't think this is worth doing since it only reduces one zookeeper write.
> Avoid checkpointing offsets in Kafka consumer that have not changed since the
> last commit
> -----------------------------------------------------------------------------------------
>
> Key: KAFKA-987
> URL: https://issues.apache.org/jira/browse/KAFKA-987
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Swapnil Ghike
> Assignee: Swapnil Ghike
> Labels: improvement
> Fix For: 0.8
>
> Attachments: kafka-987.patch, kafka-987-v2.patch
>
>
> We need to fix the Kafka zookeeper consumer to avoid checkpointing offsets
> that have not changed since the last offset commit. This will help reduce the
> write load on zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
