Hi Kafka people...
After a bit of tuning and an upgrade to Kafka 0.10.1.2, this error starts
showing up and the whole thing kind of dies.
2017-05-14 18:51:52,342 | ERROR | hread-3-producer | RecordCollectorImpl
| 91 - com.dexels.kafka.streams - 0.0.115.201705131415 | task
[1_0] Error sending record to topic
KNBSB-test-generation-7-personcore-personcore-photopersongeneration-7-repartition.
No more offsets will be recorded for this task and the exception will
eventually be thrown
org.apache.kafka.common.errors.InvalidTimestampException: The timestamp of
the message is out of acceptable range.
2017-05-14 18:51:52,343 | INFO | StreamThread-3 | StreamThread
| 91 - com.dexels.kafka.streams - 0.0.115.201705131415 |
stream-thread [StreamThread-3] Flushing state stores of task 1_0
2017-05-14 18:51:52,345 | ERROR | StreamThread-3 | StreamThread
| 91 - com.dexels.kafka.streams - 0.0.115.201705131415 |
stream-thread [StreamThread-3] Failed while executing StreamTask 1_0 due to
flush state:
org.apache.kafka.streams.errors.StreamsException: task [1_0] exception
caught when producing
at
org.apache.kafka.streams.processor.internals.RecordCollectorImpl.checkForException(RecordCollectorImpl.java:121)[91:com.dexels.kafka.streams:0.0.115.201705131415]
at
org.apache.kafka.streams.processor.internals.RecordCollectorImpl.flush(RecordCollectorImpl.java:129)[91:com.dexels.kafka.streams:0.0.115.201705131415]
at
org.apache.kafka.streams.processor.internals.StreamTask.flushState(StreamTask.java:422)[91:com.dexels.kafka.streams:0.0.115.201705131415]
at
org.apache.kafka.streams.processor.internals.StreamThread$4.apply(StreamThread.java:555)[91:com.dexels.kafka.streams:0.0.115.201705131415]
at
org.apache.kafka.streams.processor.internals.StreamThread.performOnTasks(StreamThread.java:501)[91:com.dexels.kafka.streams:0.0.115.201705131415]
at
org.apache.kafka.streams.processor.internals.StreamThread.flushAllState(StreamThread.java:551)[91:com.dexels.kafka.streams:0.0.115.201705131415]
at
org.apache.kafka.streams.processor.internals.StreamThread.shutdownTasksAndState(StreamThread.java:449)[91:com.dexels.kafka.streams:0.0.115.201705131415]
at
org.apache.kafka.streams.processor.internals.StreamThread.shutdown(StreamThread.java:391)[91:com.dexels.kafka.streams:0.0.115.201705131415]
at
org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:372)[91:com.dexels.kafka.streams:0.0.115.201705131415]
Caused by: org.apache.kafka.common.errors.InvalidTimestampException: The
timestamp of the message is out of acceptable range.
What does this mean? How can I debug this?
Two observations:
- I only see this on *-repartition topics
- ... which are also the only topic with cleanup policy = 'delete'
