MirrorMaker 2.0 stores the offsets of one cluster in another. So, you can read the offsets from the same cluster once this KIP is implemented.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0#KIP-382:MirrorMaker2.0-RemoteClusterUtils On Fri, May 10, 2019 at 12:29 PM Emmanuel <emmanuel....@gmail.com> wrote: > Thanks! I wonder if this is a bit far-fetched as no one seems to do this at > the moment. > > On Fri, May 10, 2019 at 12:50 AM Guozhang Wang <wangg...@gmail.com> wrote: > > > Hello Emmanuel, > > > > Yes I think it is do-able technically. Note that it means the offsets of > > cluster A would be stored on cluster B and hence upon restarting one need > > to talk to cluster B in order to get the committed position in cluster A. > > > > > > Guozhang > > > > > > On Thu, May 9, 2019 at 11:58 AM Emmanuel <emmanuel....@gmail.com> wrote: > > > > > Hello, > > > > > > I would like to know if there is a Java client that would allow me to > > > consume from topics on a cluster A and produce to topics on a cluster B > > > with exactly-once semantics. My understanding of the Kafka transactions > > is > > > that on the paper it could work, but the kafka java client assumes both > > are > > > running on the same cluster. Is it something do-able? If it is and > > nothing > > > exists yet, would it be an interesting contribution? > > > > > > > > > -- > > -- Guozhang > > >
