chia7712 commented on code in PR #22784:
URL: https://github.com/apache/kafka/pull/22784#discussion_r3626873896
##########
clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java:
##########
@@ -183,8 +183,22 @@ public class ConsumerConfig extends AbstractConfig {
"Negative duration is not allowed.</li>" +
"<li>none: throw exception to the consumer if no previous offset
is found for the consumer's group</li>" +
"<li>anything else: throw exception to the consumer.</li></ul>" +
- "<p>Note that altering partition numbers while setting this config
to latest may cause message delivery loss since " +
- "producers could start to send messages to newly added partitions
(i.e. no initial offsets exist yet) before consumers reset their offsets.";
+ "<p>Note that increasing a topic's partition count while this
config is set to <code>latest</code> may cause silent " +
+ "message loss: producers may begin appending records to a newly
created partition before the consumer discovers it, " +
+ "and <code>latest</code> resets the position to the log end
offset, skipping any records produced during that discovery gap.</p>" +
+ "<p>To avoid this, prefer
<code>by_duration:<duration></code>. When a partition has no committed
offset, " +
+ "<code>by_duration</code> determines the starting position by
issuing a <code>ListOffsets</code> lookup for " +
+ "<code>now() - duration</code>. If the target timestamp is earlier
than the partition's creation time, the lookup " +
+ "returns offset 0, ensuring that records produced during the
discovery window are still consumed. Size the duration to cover " +
Review Comment:
`offset 0` is not accurate if the partition's start offset is not 0, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
