Re: Restore from a savepoint is very slow

2018-04-04 Thread Dongwon Kim
It was due to too low parallelism. I increase parallelism large enough (actually set it to the total number of task slots on the cluster) and it makes restore from a savepoint much faster. This is somewhat related to the previous discussion I had with Robert and Aljoscha. Having a standalone clu

Re: Restore from a savepoint is very slow

2018-04-01 Thread Dongwon Kim
Attached is a log file from a taskmanager. Please take a look at the log file considering the below events: - Around 01:10:47 : the job is submitted to the job manager. - Around 01:16:30 : suddenly source starts to read from and sink starts to write data to Kafka Any help would be greatly appreci

Restore from a savepoint is very slow

2018-04-01 Thread Dongwon Kim
Hi, While restoring from the latest checkpoint starts immediately after the job is restarted, restoring from a savepoint takes more than five minutes until the job makes progress. During the blackout, I cannot observe any resource usage over the cluster. After that period of time, I observe that