[
https://issues.apache.org/jira/browse/KAFKA-7669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17032512#comment-17032512
]
John Roesler commented on KAFKA-7669:
-------------------------------------
This issue has been lurking for a while, and has been reported a number of
different ways. It seems to take two forms:
1. changing the topology at all (in apparently compatible ways) can renumber
operators and corrupt the application upon restart
2. changing the topology in combination with a rolling bounce results in
members executing a different topology than the leader, which leads to extra
problems (such as NPEs)
https://issues.apache.org/jira/browse/KAFKA-7669 is related, and seems to be
more about just changing the topology at all
https://issues.apache.org/jira/browse/KAFKA-8307 proposes a fix
https://issues.apache.org/jira/browse/KAFKA-8810 seems to be a duplicate of
KAFKA-8307
and https://issues.apache.org/jira/browse/KAFKA-9518 reports an exception that
results from a rolling-bounce topology change.
> Stream topology definition is not robust to the ordering changes
> ----------------------------------------------------------------
>
> Key: KAFKA-7669
> URL: https://issues.apache.org/jira/browse/KAFKA-7669
> Project: Kafka
> Issue Type: Wish
> Components: streams
> Affects Versions: 2.0.0
> Reporter: Mateusz Owczarek
> Priority: Major
>
> It seems that if the user does not guarantee the order of the stream topology
> definition, he may end up with multiple stream branches having the same
> internal changelog (and repartition, if created) topic.
> Let's assume:
> {code:java}
> val initialStream = new StreamsBuilder().stream(sth);
> val someStrings = (1 to 10).map(_.toString)
> val notGuaranteedOrderOfStreams: Map[String, KStream[...]] =
> someStrings.map(s => s -> initialStream.filter(...)).toMap{code}
> When the user defines now common aggregation logic for the
> notGuaranteedOrderOfStreams, and runs multiple instances of the application
> the KSTREAM-AGGREGATE-STATE-STORE topics names will not be unique and will
> contain results of the different streams from notGuaranteedOrderOfStreams map.
> All of this without a single warning that the topology (or just the order of
> the topology definition) differs in different instances of the Kafka Streams
> application.
> Also, I am concerned that ids in "KSTREAM-AGGREGATE-STATE-STORE-id-changelog
> " match so well for the different application instances (and different
> topologies).
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
