Hi, We are trying to run an Kafka streams applications against a Kafka cluster and some of the incoming messages have negative timestamp as some of the producers are using older version of the Kafka.
Therefore we used WallclockTimeStampExtractor to patch those timestamps. But also read in the documentation that this will not have any effect for the internally created topics for the streams. (repartitioned topics.) It is strange that internal records have negative timestamps as we are using latest kafka versions for kafka stream application. Error message we get ====================== org.apache.kafka.streams.processor.FailOnInvalidTimestamp - Input record ConsumerRecord(topic = my-app-KSTREAM-TRANSFORM-0000000009-repartition, partition = 6, offset = 18, CreateTime = -1, serialized key size = 8, serialized value size = 104, headers = RecordHeaders(headers = [], isReadOnly = false), key = 13143752, value = <our message>) has invalid (negative) timestamp. Possibly because a pre-0.10 producer client was used to write this record to Kafka without embedding a timestamp, or because the input topic was created before upgrading the Kafka cluster to 0.10+. Use a different TimestampExtractor to process this data. Properties in the code ======================= kafkaStreamsProperties.put(StreamsConfig.DEFAULT_TIMESTAMP_EXTRACTOR_CLASS_CONFIG,WallclockTimestampExtractor.class); kafkaStreamsProperties.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass()); kafkaStreamsProperties.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, Serdes.String().getClass()); kafkaStreamsProperties.put(StreamsConfig.DEFAULT_DESERIALIZATION_EXCEPTION_HANDLER_CLASS_CONFIG, LogAndContinueExceptionHandler.class); Kafka cluster version : kafka_2.12-0.10.2.1 Kafka version in streams application : 1.0.0 We have same streams application running on two instances consuming the messages from a Kafka cluster. What could be the reason stream application fails on negative timestamps while we have used wallClockTimeStamp extractor ? Thank you, Best Regards, Shelan Perera
