Dear Stefan,
Thanks for the clarification.
How is however the state recovered in the case of a task failure? Assuming
there is a topology of 10 workers and a worker dies. The state in this case,
after restarting the entire execution, will how exactly be distributed across
the workers?
Domi
Hi Dominik,
as Gordon’s response only covers keyed-state, I will briefly explain what
happens for non-keyed operator state. In contrast to Flink 1.1, Flink 1.2
checkpointing does not write a single blackbox object (e.g. ONE object that is
a set of all kafka offsets is emitted), but a list of bl
Hi Dominik,
Do you mean how Flink redistributes an operator’s state when the parallelism of
the operator is changed?
If so, you can take a look at [1] and [2].
Cheers,
Gordon
[1] https://issues.apache.org/jira/browse/FLINK-3755
[2]
https://docs.google.com/document/d/1G1OS1z3xEBOrYD4wSu-LuBCyPU
Hi everyone,
In the case of scaling out a Flink cluster, how does Flink handle operator
state partitioning of a staged topology?
Regards,
Dominik
