fsk119 commented on a change in pull request #14111:
URL: https://github.com/apache/flink/pull/14111#discussion_r528648191



##########
File path: docs/dev/table/connectors/kafka.md
##########
@@ -249,6 +249,15 @@ Besides enabling Flink's checkpointing, you can also 
choose three different mode
 
 Please refer to [Kafka documentation]({% link dev/connectors/kafka.md 
%}#kafka-producers-and-fault-tolerance) for more caveats about delivery 
guarantees.
 
+### Per-partition-watermark Source
+
+Flink supports to emit per-partition-watermark for Kafka. Using this feature, 
watermarks are generated inside the Kafka consumer. The per-partition-watermark 
are merged in
+the same way as watermarks are merged on stream shuffles. The output watermark 
of the source is determined by the minimum watermark among the partitions it 
reads. Considering a watermark assigner
+advance the watermark according to the event-time on the records. If some 
partitions in the topics are idle, the watermark assigner will not advance. You 
can alleviate this problem by
+setting appropriate [idelness timeouts]({{ site.baseurl 
}}/dev/event_timestamps_watermarks.html#dealing-with-idle-sources).

Review comment:
       Could you give a bad case when this feature is enabled? With this 
feature, we can get more fine-grained watermark. Only some partition are idle, 
the watermark assigner doesn't advance itself. However, we can set idle timeout 
to solve this problem.
   
   If we need to introduce a option, I think the default behaviour is to enable 
this feature.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Reply via email to