Re: Question about Flink internals

2017-09-08 Thread Junguk Cho
Hi, Timo. Thank you for detailed replies. It helps me to understand flink a lot. However, there are misinterpreted points. 2. From a user's perspective you can only see the "real data". Internally, there are different types of records that flow through the topology (namely watermarks, checkpoint

Re: Question about Flink internals

2017-09-07 Thread Timo Walther
Hi Junguk, I try to answer your questions, but also loop in Ufuk who might now more about the network internals: 1. Yes, every operator/operator chain has a "setParallelism()" method do specify the parallelism. The overall parallelism of the job can be set when submitting a job. The parallel