You might also want to change the parallelism if the rate of your input
streams varies, e.g., you scale an application down over night or the
weekend.

2018-02-13 13:43 GMT+01:00 Xingcan Cui <xingc...@gmail.com>:

> Hi Max,
>
> Currently, the timers can only be used with keyed streams. As @Fabian
> suggested, you can “forge” a keyed stream with the special KeySelector,
> which maps all the records to the same key.
>
> IMO, Flink uses keyed streams/states as it’s a deterministic distribution
> mechanism. Here, “the parallelism changes” may also refer to a parallelism
> change after the job restarts (e.g., when a node crashes). Flink can make
> sure that all the processing tasks and states will be safely re-distributed
> across the new cluster.
>
> Hope that helps.
>
> Best,
> Xingcan
>
> > On 13 Feb 2018, at 5:18 PM, m@xi <makisnt...@gmail.com> wrote:
> >
> > OK Great!
> >
> > Thanks a lot for the super ultra fast answer Fabian!
> >
> > One intuitive follow-up question.
> >
> > So, keyed state is the most preferable one, as it is easy for the Flink
> > System to perform the re-distribution in case of change in parallelism,
> if
> > we have a scale-up or scale-down. Also, it is useful to use hash
> partition a
> > stream to different nodes/processors/PU (Processing Units) in general, by
> > Keyed State.
> >
> > Any other reasons for making Keyed State a must?
> >
> > Last but not least, can you elaborate further on the "when the
> parallelism
> > changes" part. I have read this in many topics in this forum, but I
> cannot
> > understand its essence. For example, I define the parallelism of each
> > operator in my Flink Job program based on the number of available PU.
> Maybe
> > the essence lies in the fast that the number of PU might change from
> time to
> > time, e.g. add more servers to the cluster where Flink runs and without
> > stopping the Flink Job that runs you may perform the rescaling.
> >
> > Thanks in advance.
> >
> > Best,
> > Max
> >
> >
> > --
> > Sent from: http://apache-flink-user-mailing-list-archive.2336050.
> n4.nabble.com/
> >
>
>

Reply via email to