Re: Slow restart from savepoint with large broadcast state when increasing parallelism

2022-12-16 Thread Jun Qin
Hi Ken, > Broadcast state is weird in that it’s duplicated, apparently avoid “hot > spots” when restoring from state. So I’m wondering how Flink handles the case > of restoring broadcast state when the parallelism increases. The Flink doc is here: https://nightlies.apache.org/flink/flink-docs-

Re: Slow restart from savepoint with large broadcast state when increasing parallelism

2022-12-16 Thread Ken Krugler
Hi Jun, Thanks for following up. The state storage is internal at a client, and isn’t throttled. Also restoring from the savepoint when we didn’t change the parallelism was fine. I didn’t see any errors in the TM logs, but I didn’t carefully inspect them - I’ll do that when we give this anoth