[
https://issues.apache.org/jira/browse/KAFKA-12176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17263020#comment-17263020
]
Jun Rao commented on KAFKA-12176:
---------------------------------
Another complication in the previous discussion was whether this default
behavior applies to compacted topic too since it can retain the data forever.
> Consider changing default log.message.timestamp.difference.max.ms
> -----------------------------------------------------------------
>
> Key: KAFKA-12176
> URL: https://issues.apache.org/jira/browse/KAFKA-12176
> Project: Kafka
> Issue Type: Improvement
> Reporter: Jason Gustafson
> Priority: Major
> Labels: needs-kip
> Fix For: 3.0.0
>
>
> The default `log.message.timestamp.difference.max.ms` is Long.MaxValue, which
> means the broker will accept arbitrary timestamps. The broker relies on
> timestamps internally for deciding when a segments should be rolled and when
> they should be deleted. A buggy client which is producing messages with
> timestamps too far in the future or past can cause segments to roll
> frequently which can lead to open file exhaustion, or it can cause segments
> to be retained indefinitely which can lead to disk space exhaustion.
> In https://issues.apache.org/jira/browse/KAFKA-4340, it was proposed to set
> the default equal to `log.retention.ms`, which still seems to make sense.
> This was rejected for two reasons as far as I can tell. First was
> compatibility, which just means we would need a major upgrade. The second is
> that we previously did not have a way to tell the user which record had
> violated the max timestamp difference, but now we have
> [KIP-467|https://cwiki.apache.org/confluence/display/KAFKA/KIP-467%3A+Augment+ProduceResponse+error+messaging+for+specific+culprit+records].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
