[
https://issues.apache.org/jira/browse/KAFKA-4936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16380868#comment-16380868
]
Blake Miller commented on KAFKA-4936:
-------------------------------------
IMO, such a feature does not need to support automatic topic creation in order
to be useful. As [~mjsax] pointed out, it adds significant complexity. I might
consider picking this up. I see from the Confluent Google Group that Damian Guy
suggested a workaround:
"If you don't know the set of topics then you would need to use a custom
Processor and you would also need to create an instance of the KafkaProducer.
Keeping in mind that in order to guarantee at-least-once each producer.send
would need to be synchronous."
[https://groups.google.com/forum/#!topic/confluent-platform/wnXLKw1-XQk]
I suppose a proper implementation in the KStreams API would do something
analogous under the hood. Does that sound reasonable?
Supporting exactly-once here sounds plausible but the details are a little
beyond me at present. It seems like the feature might be useful without that,
as well, since currently the only options seem to be a custom Processor +
Producer, or just fall back to using Producer & Consumer directly instead of
Kafka Streams, neither of which would support exactly-once.
> Allow dynamic routing of output records
> ---------------------------------------
>
> Key: KAFKA-4936
> URL: https://issues.apache.org/jira/browse/KAFKA-4936
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Matthias J. Sax
> Priority: Major
> Labels: needs-kip
>
> Currently, all used output topics must be know beforehand, and thus, it's not
> possible to send output records to topic in a dynamic fashion.
> There have been couple of request for this feature and we should consider
> adding it. There are many open questions however, with regard to topic
> creation and configuration (replication factor, number of partitions) etc.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
