[
https://issues.apache.org/jira/browse/KAFKA-4633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16040795#comment-16040795
]
Andres Gomez Ferrer edited comment on KAFKA-4633 at 6/7/17 12:42 PM:
---------------------------------------------------------------------
Hi all,
I have some question about this issue. With this change, the Kafka Streams app
doesn't auto-create topics, so you need to create the topics before start
running the APP. This is ok if you create the input/output topics, but when you
use the *through* method to do some repartition .. Do you need to create the
topic before? Maybe this is a little strange, isn't?
I have seen the *through* method, and it works with *to(...)* and *stream(...)*
{code}
@Override
public KStream<K, V> through(Serde<K> keySerde, Serde<V> valSerde,
StreamPartitioner<? super K, ? super V> partitioner, String topic) {
to(keySerde, valSerde, partitioner, topic);
return topology.stream(keySerde, valSerde, topic);
}
{code}
What do you think? Maybe this is a new issue or Is normal that you need to
create the topic that later you use in the *through* method?
Regards,
Andrés
was (Author: agomez):
Hi all,
I have some question about this issue. With this change, the Kafka Streams app
doesn't auto-create topics, so you need to create the topics before start
running the APP. This is ok if you create the input/output topics, but when you
use the *through* method to do some repartition .. Do you need to create the
topic before? Maybe this is a little strange, isn't?
I have seen the *through* method, and it works with *to(...)* and *stream(...)*
{code}
@Override
public KStream<K, V> through(Serde<K> keySerde, Serde<V> valSerde,
StreamPartitioner<? super K, ? super V> partitioner, String topic) {
to(keySerde, valSerde, partitioner, topic);
return topology.stream(keySerde, valSerde, topic);
}
{code}
What do you think? Maybe this is a new issue or Is normal that you need to
create the topic that later you use in the * through* method?
Regards,
Andrés
> Always use regex pattern subscription to avoid auto create topics
> -----------------------------------------------------------------
>
> Key: KAFKA-4633
> URL: https://issues.apache.org/jira/browse/KAFKA-4633
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Reporter: Guozhang Wang
> Assignee: Guozhang Wang
> Labels: architecture
> Fix For: 0.10.2.0
>
>
> In {{KafkaConsumer}}, a metadata update is requested whenever
> {{subscribe(List<String> topics ..)}} is called. And when such a metadata
> request is sent to the broker upon the first {{poll}} call, it will cause the
> broker to auto-create any topics that do not exist if the broker-side config
> {{topic.auto.create}} is turned on.
> In order to work around this issue until the config is default to false and
> gradually be deprecated, we will let Streams to always use the other
> {{subscribe}} function with regex pattern, which will send the metadata
> request with empty topic list and hence won't trigger broker-side auto topic
> creation.
> The side-effect is that the metadata response will be larger, since it
> contains all the topic infos; but since we only refresh it infrequently this
> will add negligible overhead.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
