[
https://issues.apache.org/jira/browse/KAFKA-8821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17010252#comment-17010252
]
Sophie Blee-Goldman commented on KAFKA-8821:
--------------------------------------------
Disabling auto-topic creation is possible for brokers version 0.11.0.0+ as of
KAFKA-5291. As of 2.2.1/2.3.x the compatibility support for Streams client
changed, where the minimum broker version requirement is now also 0.11.x. This
means we can now safely go back to using regular subscription
Note, that users can still use pattern subscription themselves to subscribe to
input topics. We should consider adding broker-side regex support and in
general avoid fetching/sending metadata on every single topic in a cluster.
> Avoid pattern subscription to allow for stricter ACL settings
> -------------------------------------------------------------
>
> Key: KAFKA-8821
> URL: https://issues.apache.org/jira/browse/KAFKA-8821
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Matthias J. Sax
> Priority: Minor
>
> To avoid triggering auto topic creation (if `auto.create.topic.enable=true`
> on the brokers), Kafka Streams uses consumer pattern subscription. For this
> case, the consumer requests all metadata from the brokers and does client
> side filtering.
> However, if users want to set ACL to restrict a Kafka Streams application,
> this may results in broker side ERROR logs that some metadata cannot be
> provided. The only way to avoid those broker side ERROR logs is to grant
> corresponding permissions.
> As of 2.3 release it's possible to disable auto topic creation client side
> (via https://issues.apache.org/jira/browse/KAFKA-7320). Kafka Streams should
> use this new feature (note, that broker version 0.11 is required) to allow
> users to set strict ACLs without getting flooded with ERROR logs on the
> broker.
> The proposal is that by default Kafka Streams disables auto-topic create
> client side (optimistically) and uses regular subscription (not pattern
> subscription). If an older broker is used, users need to explicitly enable
> `allow.auto.create.topic` client side. If we detect this setting, we switch
> back to pattern based subscription.
> If users don't enable auto topic create client side and run with an older
> broker, we would just rethrow the exception to the user, adding some context
> information on how to fix the issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
