Jason Gustafson created KAFKA-12176:
---------------------------------------
Summary: Consider changing default
log.message.timestamp.difference.max.ms
Key: KAFKA-12176
URL: https://issues.apache.org/jira/browse/KAFKA-12176
Project: Kafka
Issue Type: Improvement
Reporter: Jason Gustafson
Fix For: 3.0.0
The default `log.message.timestamp.difference.max.ms` is Long.MaxValue, which
means the broker will accept arbitrary timestamps. The broker relies on
timestamps internally for deciding when a segments should be rolled and when
they should be deleted. A buggy client which is producing messages with
timestamps too far in the future or past can cause segments to roll frequently
which can lead to open file exhaustion, or it can cause segments to be retained
indefinitely which can lead to disk space exhaustion.
In https://issues.apache.org/jira/browse/KAFKA-4340, it was proposed to set the
default equal to `log.retention.ms`, which still seems to make sense. This was
rejected for two reasons as far as I can tell. First was compatibility, which
just means we would need a major upgrade. The second is that we previously did
not have a way to tell the user which record had violated the max timestamp
difference, but now we have
[KIP-467|https://cwiki.apache.org/confluence/display/KAFKA/KIP-467%3A+Augment+ProduceResponse+error+messaging+for+specific+culprit+records].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
