Hello,
I think we're misunderstanding the docs on some level and I need a little
clarification. We have the following setup:
1) 0.8.2 producer -> writing to Kafka 0.10.0.1 cluster w/ version 10
message format (source cluster).
2) 0.10.0.1 mirror using the 'new consumer' reading from the source cluster
and writing to Kafka 0.10.0.1 cluster w/version 0.8.2 message format
(destination cluster). We need some of the features like SSL, hence using
the new consumer.
3) Lots of old 0.8.2 consumers reading from the destination cluster that
still need to be upgraded.
We're seeing errors from the mirror maker when trying to produce to the
destination cluster like the following:
java.lang.IllegalArgumentException: Invalid timestamp -1
at
org.apache.kafka.clients.producer.ProducerRecord.<init>(ProducerRecord.java:60)
Is the root problem the 0.8.2 producer sending data to the source cluster
or the new 10 mirror writing data to the destination cluster in 0.8.2
format? From the docs we were under the impression that the data would be
stored in the source cluster in 10 format regardless of the producer and
the mirror could produce to the destination cluster regardless of it's
message format setting.
Is this current setup non-functional or is there a way to make this work?
For example, if the mirror producing is the issue could we implement a
custom MirrorMakerMessageHandler? Any advice and clarification would be
helpful, thanks.
Craig
