[
https://issues.apache.org/jira/browse/KAFKA-1012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13751802#comment-13751802
]
Tejas Patil commented on KAFKA-1012:
------------------------------------
Thanks a lot @guozhang for all your comments !!!
26. OffsetMetadataAndError
26.1 +1. Would include in next patch.
27. ZookeeperConsumerConnector
27.1 Nice one. Would include in next patch.
27.2 Would rename it as per your comment.
27.3 "... as offset backend storage is not fully started up, will try later"
hints that this would happen only on offset manager startup. The loading
happens not only on startup but also when there are leadership changes. To
cater to the 2nd possibility, it was phrased as "... as some offsets are being
loaded"
28. KafkaApis
28.1 Well, it seems like trunk has a mixup of both the styles:
$ grep -r --include=*.scala "} else {" . | wc -l
66
$ grep -r --include=*.scala "else {" . | grep -v "} else {" | grep -v "}else {"
| wc -l
19
The one you suggested is used frequently and I would follow your advice.
29. OffsetManager
29.1 +1. Would include in next patch.
29.2 Good catch :)
30. In some coming version down the line, we would be shifting to think client
so forwarding would go out of the picture. Offset fetch wont have a purgatory
at broker end but commits would. Its expected that offset fetch request would
have low latency as it would just fetch from hash table or ZK.
> Implement an Offset Manager and hook offset requests to it
> ----------------------------------------------------------
>
> Key: KAFKA-1012
> URL: https://issues.apache.org/jira/browse/KAFKA-1012
> Project: Kafka
> Issue Type: Sub-task
> Components: consumer
> Reporter: Tejas Patil
> Assignee: Tejas Patil
> Priority: Minor
> Attachments: KAFKA-1012.patch, KAFKA-1012-v2.patch
>
>
> After KAFKA-657, we have a protocol for consumers to commit and fetch offsets
> from brokers. Currently, consumers are not using this API and directly
> talking with Zookeeper.
> This Jira will involve following:
> 1. Add a special topic in kafka for storing offsets
> 2. Add an OffsetManager interface which would handle storing, accessing,
> loading and maintaining consumer offsets
> 3. Implement offset managers for both of these 2 choices : existing ZK based
> storage or inbuilt storage for offsets.
> 4. Leader brokers would now maintain an additional hash table of offsets for
> the group-topic-partitions that they lead
> 5. Consumers should now use the OffsetCommit and OffsetFetch API
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
