Hello, Just to close this issue out. The 8 producer going to the 10 cluster was the root issue. The mirror maker by default was unable to produce the message to the destination cluster. The work around was to include a MirrorMakerMessageHandler that did nothing but repackage the message again. In the future it might be nice if the mirror handled this auto-magically, but at least the ability to alter the behavior provided an easy fix. Hope this helps someone else, thanks.
Craig Hello, > > I think we're misunderstanding the docs on some level and I need a little > clarification. We have the following setup: > > 1) 0.8.2 producer -> writing to Kafka 0.10.0.1 cluster w/ version 10 > message format (source cluster). > 2) 0.10.0.1 mirror using the 'new consumer' reading from the source > cluster and writing to Kafka 0.10.0.1 cluster w/version 0.8.2 message > format (destination cluster). We need some of the features like SSL, hence > using the new consumer. > 3) Lots of old 0.8.2 consumers reading from the destination cluster that > still need to be upgraded. > > We're seeing errors from the mirror maker when trying to produce to the > destination cluster like the following: > > java.lang.IllegalArgumentException: Invalid timestamp -1 > at org.apache.kafka.clients.producer.ProducerRecord.<init> > (ProducerRecord.java:60) > > Is the root problem the 0.8.2 producer sending data to the source cluster > or the new 10 mirror writing data to the destination cluster in 0.8.2 > format? From the docs we were under the impression that the data would be > stored in the source cluster in 10 format regardless of the producer and > the mirror could produce to the destination cluster regardless of it's > message format setting. > > Is this current setup non-functional or is there a way to make this work? > For example, if the mirror producing is the issue could we implement a > custom MirrorMakerMessageHandler? Any advice and clarification would be > helpful, thanks. > > Craig >
