[
https://issues.apache.org/jira/browse/FLINK-8144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16265251#comment-16265251
]
ASF GitHub Bot commented on FLINK-8144:
---------------------------------------
Github user fhueske commented on the issue:
https://github.com/apache/flink/pull/5063
I see. The bad performance is related to way that you generate the
watermarks. Generating so many watermarks is very inefficient because you
processing many more watermarks than actual records (watermarks are always
broadcasted).
I'm not convinced that this is a scenario that we should optimize for.
> Optimize the timer logic in RowTimeUnboundedOver
> ------------------------------------------------
>
> Key: FLINK-8144
> URL: https://issues.apache.org/jira/browse/FLINK-8144
> Project: Flink
> Issue Type: Bug
> Components: Table API & SQL
> Reporter: Dian Fu
> Assignee: Dian Fu
> Fix For: 1.5.0
>
>
> Currently the logic of {{RowTimeUnboundedOver}} is as follows:
> 1) When element comes, buffer it in MapState and and register a timer at
> {{current watermark + 1}}
> 2) When event timer triggered, scan the MapState and find the elements below
> the current watermark and process it. If there are remaining elements to
> process, register a new timer at {{current watermark + 1}}.
> Let's assume that watermark comes about 5 seconds later than the event on
> average, then we will scan about 5000 times the MapState before actually
> processing the events.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
