[
https://issues.apache.org/jira/browse/KAFKA-12743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338959#comment-17338959
]
Sergey Zyrianov commented on KAFKA-12743:
-----------------------------------------
Hi [~guozhang],
Looks like the workaround wont work for us due to latency in ktable updates
compared to put() on statestore. Order of the operations is reversed (now we
push topic and wait state appears in ktable, versus put state to statestore and
send it to changelog topic)
With latency come race conditions and we need to come up with something else.
I find KIP-545 is misleading when it claims "The offset translation is great
feature to serve the foundation of migrating or failing over downstream
consumers (*including Kafka stream applications*)"
As you pointed out there no good story for streams applications when it comes
to DR.
> [Kafka Streams] - regex name for state-store change-log topic
> -------------------------------------------------------------
>
> Key: KAFKA-12743
> URL: https://issues.apache.org/jira/browse/KAFKA-12743
> Project: Kafka
> Issue Type: Improvement
> Components: mirrormaker, streams
> Affects Versions: 2.8.0
> Reporter: Sergey Zyrianov
> Priority: Major
>
> Currently, when working with Kafka backed state stores in Kafka Streams,
> these log compacted topics are given a hardcoded name :
> _app_id-storename-changelog_
> {noformat}
> public static String storeChangelogTopic(String applicationId, String
> storeName) {
> return applicationId + "-" + storeName + STATE_CHANGELOG_TOPIC_SUFFIX;
> }{noformat}
>
> MirrorMaker2(mm2) copies these topics to remote cluster under the name
> _src-cluster-alias.app_id-storename-changelog_
>
> When streams app fails over to the remote cluster it has troubles to find
> changelog topic of its state store since it was renamed - given source
> cluster prefix by mm2.
> Whats the fix should be ? instruct mm2 to keep topic name or subscribe to
> regex *._app_id-storename-changelog_ topic name for the state's changelog.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
