Hi John, I’m not a Kafka streams expert but have experimented a few times – I recall that Kafka Streams does need to create/use “internal topics” – and security has to be set on clients correctly from memory. This may help? https://kafka.apache.org/23/documentation/streams/developer-guide/manage-topics And this https://kafka.apache.org/23/documentation/streams/developer-guide/security.html#streams-developer-guide-security
Regards, Paul Brebner, NetApp From: John D. Ament <johndam...@apache.org> Date: Friday, 22 November 2024 at 8:46 am To: users@kafka.apache.org <users@kafka.apache.org> Subject: Explicitly creating topology topics in a streams app [You don't often get email from johndam...@apache.org. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] EXTERNAL EMAIL - USE CAUTION when clicking links or attachments Hi, When I look at most stream examples I notice that they tend to include the broker setting: auto.create.topics.enable=true My understanding is that this isn't usually recommended for production environments, so we have it off. We have started to play with Kafka Streams apps a bit but noticed that they fail in our production environments with errors around "UNKNOWN_TOPIC_OR_PARTITION". I suspect it's related to not having auto create enabled. Is there an option to force the client to create the topology topics that they use for Kafka Streams apps? Or is auto creation required? Thanks, John
