Re: apache flink: Why checkpoint coordinator takes long time to get completion

2019-07-23 Thread Xiangyu Su
Hi Zili, here is the release notes for 1.8.1 https://flink.apache.org/news/2019/07/02/release-1.8.1.html But I could not find any ticket related to the "unexpected time-consuming", I have just tested our application with both versions, this issue is be able to reproduce every time with version 1.8

Re: apache flink: Why checkpoint coordinator takes long time to get completion

2019-07-22 Thread Zili Chen
Hi Xiangyu, Could you share the corresponding JIRA that fixed this issue? Best, tison. Xiangyu Su 于2019年7月19日周五 下午8:47写道: > btw. it seems like this issue has been fixed in 1.8.1 > > On Fri, 19 Jul 2019 at 12:21, Xiangyu Su wrote: > >> Ok, thanks. >> >> and this time-consuming until now alway

Re: apache flink: Why checkpoint coordinator takes long time to get completion

2019-07-19 Thread Xiangyu Su
btw. it seems like this issue has been fixed in 1.8.1 On Fri, 19 Jul 2019 at 12:21, Xiangyu Su wrote: > Ok, thanks. > > and this time-consuming until now always happens after 3rd checkpointing, > and this unexpected time-consuming was always consistent (~ 4 min by under > 4G/min incoming traffi

Re: apache flink: Why checkpoint coordinator takes long time to get completion

2019-07-19 Thread Xiangyu Su
Ok, thanks. and this time-consuming until now always happens after 3rd checkpointing, and this unexpected time-consuming was always consistent (~ 4 min by under 4G/min incoming traffic). On Fri, 19 Jul 2019 at 11:06, Biao Liu wrote: > Hi Xiangyu, > > Just took a glance at the relevant codes. T

Re: apache flink: Why checkpoint coordinator takes long time to get completion

2019-07-19 Thread Biao Liu
Hi Xiangyu, Just took a glance at the relevant codes. There is a gap between calculating the duration and logging it out. I guess the checkpoint 4 is finished in 1 minute, but there is an unexpected time-consuming operation during that time. But I can't tell which part it is. Xiangyu Su 于2019年7

apache flink: Why checkpoint coordinator takes long time to get completion

2019-07-19 Thread Xiangyu Su
Dear flink community, We are POC flink(1.8) to process data in real time, and using global checkpointing(S3) and local checkpointing(EBS), deploy cluster on EKS. Our application is consuming data from Kinesis. For my test e.g I am using checkpointing interval 5min. and minimum pause 2min. The is