Hi Krzysztof,
Sorry there are indeed no document said that the operator state is only kept in
memory, but based on the
current implementation it is indeed the case.
And I might also need to fix one point: the Split Enumerate should be executed
in the JM side inside the OperatorCoordinator,
and
Thank you both,
yes seems that the only option on a non keyed operate would be List State,
my bad.
Yun Gao,
I'm wondering from where you get the information that " Flink only support
in-memory operator state", can you point me to the documentation that says
that?
I cannot find any mention in the d
Hi Krzysztof,
If I understand right, I think managed operator state might not help here since
currently Flink
only support in-memory operator state.
Is it possible currently we first have a customized SplitEnumerator to skip the
processed files
in some other way? For example, if these files hav
Hi Krzysztof,
Non-keyed operator state only supports list-like state [1] as there exist no
primary key in operator state. That is to say you cannot use map state in
source operator.
[1]
https://nightlies.apache.org/flink/flink-docs-release-1.13/docs/dev/datastream/fault-tolerance/state/#using
