[
https://issues.apache.org/jira/browse/KAFKA-13612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482729#comment-17482729
]
Jason Gustafson commented on KAFKA-13612:
-----------------------------------------
[~dnwe] I agree with you that computing consumer lag is painful. However, keep
in mind that even the `Fetch` API uses polling, and the requests must be sent
to all partitions of __consumer_offsets (which for most clusters means fetching
on all brokers). It is fair to say though that this approach gives you only the
deltas, which is more efficient than always fetching all offsets. On the other
hand, you still need to periodically send ListOffsets to all of the partitions
that have committed offsets to get the latest end offset, so the difference
might not be quite so dramatic.
The problem is that reading from __consumer_offsets is not a supported API, so
users have to deal with breakage. There's no guarantee that we'll even continue
to use this topic for committed offsets in the future. Many users run into
hotspot issues with the current hashing approach, so there is some interest in
changing it. My take is that this there is indeed a gap in the consumer
administrative APIs and we should try to address it with an API that can
actually offer compatibility guarantees. For example, perhaps we could
introduce an OffsetQuery API or something like that to make it easier to get
the latest offset commits without respect to a specific group.
> internal topics won't be created in metadataRequest when
> auto.create.topics.enable=false
> ----------------------------------------------------------------------------------------
>
> Key: KAFKA-13612
> URL: https://issues.apache.org/jira/browse/KAFKA-13612
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 2.8.0, 3.0.0
> Reporter: Luke Chen
> Assignee: Luke Chen
> Priority: Major
>
> In KAFKA-9751, when create internal topics through FindCoordinator or
> Metadata request, we route the topic creation request to the controller
> instead of handling by itself. We change logic in
> `KafkaApis#getTopicMetadata`, and make the internal topic won't get created
> when "auto.create.topics.enable=false`.
> h4.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
