Looking for a replication scheme whereby a copy of my stream is replicated into another dc such that the same events appear in the same order with the same offsets in each dc.
This makes it easier for me to build replicated state machines as I get exactly the same data in each dc Is there any way to achieve this in Kafka?. When I looked at mirrormaker it did not work like this. I got dupes due to it's at least once nature (I note there are also issue reports of mm losing data). And also the offsets don't line up. I am aware of a commercial streaming product that provides this sort of guarantee, it is also the sort of thing one gets from commercial database replication schemes. But I'm interested in Kafka because it has a good story for HA within a DC. However it is the cross DC piece I find lacking. I note that Kafka's underlying LogManager api allows one to specify the offsets and wondered if a product could be built on that api on the remote end of a Kafka cross DC rep scheme that faithfully copied the data and offsets from primary dc to disaster site. Finally I would like consumers of my stream to only receive data that has been made globally durable ie only allow 'messages after' calls to return messages that have been replicated. One way to get this effect would be to serve consumers from the copy in the DR site, but could anything else achieve this. Thanks
