Hi We are using kafka streams api. We have come across an issue which we need a short term fix for.
We host 2 independent kafka clusters in 2 different data centres and these clusters are mirrored using mirrormaker. The problem this brings is that we cannot use kTables backed by a rocksDB because when traffic flips from one datacentre to the other, the local state will not be populated with previous state from the other datacentre. This would not be an issue if we had one logical kafka cluster. We are moving to this setup, but in the short term need a solution. The kafka docs say that you can plug in a custom database for ktables, so we are exploring the possibility of using a shared database (e.g. mongo, or oracle) to store the state. This database is visible by both kafka clusters and acts as a single source of truth. I guess my question is does anybody see an reason why this would not work? Thanks Ade
