twalthr commented on a change in pull request #14111:
URL: https://github.com/apache/flink/pull/14111#discussion_r528729420
##########
File path: docs/dev/table/connectors/kafka.md
##########
@@ -249,6 +249,15 @@ Besides enabling Flink's checkpointing, you can also
choose three different mode
Please refer to [Kafka documentation]({% link dev/connectors/kafka.md
%}#kafka-producers-and-fault-tolerance) for more caveats about delivery
guarantees.
+### Per-partition-watermark Source
+
+Flink supports to emit per-partition-watermark for Kafka. Using this feature,
watermarks are generated inside the Kafka consumer. The per-partition-watermark
are merged in
+the same way as watermarks are merged on stream shuffles. The output watermark
of the source is determined by the minimum watermark among the partitions it
reads. Considering a watermark assigner
+advance the watermark according to the event-time on the records. If some
partitions in the topics are idle, the watermark assigner will not advance. You
can alleviate this problem by
+setting appropriate [idelness timeouts]({{ site.baseurl
}}/dev/event_timestamps_watermarks.html#dealing-with-idle-sources).
Review comment:
Maybe there is some misunderstanding. The link that you put in the docs
helps DataStream API users:
```
WatermarkStrategy
.<Tuple2<Long,
String>>forBoundedOutOfOrderness(Duration.ofSeconds(20))
.withIdleness(Duration.ofMinutes(1));
```
But it should be updated to help Table/SQL API users.
I found
`org.apache.flink.table.api.config.ExecutionConfigOptions#TABLE_EXEC_SOURCE_IDLE_TIMEOUT`
we should link to this instead if it works for the Kafka connector.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
