Hi, Flink supports two types of state: 1) Key-partitioned state 2) Non-partitioned operator state (Checkpointed interface)
Key-partitioned state is internally organized by key and can be "simply" rehashed. The actual implementation is more involved to make this efficient. This document contains details [1]. In general, non-partitioned state can not be split. However, there are special types of state which is not organized by key but which is splittable such as a list state which can be split into individual elements and redistributed. This is described in this design document [2]. Hope this helps, Fabian [1] https://docs.google.com/document/d/1G1OS1z3xEBOrYD4wSu-LuBCyPUWyFd9l3T9WyssQ63w/edit#heading=h.2suhu1fjxmp4 [2] https://cwiki.apache.org/confluence/display/FLINK/FLIP-8%3A+Rescalable+Non-Partitioned+State 2017-01-11 8:34 GMT+01:00 gallenvara <gaolunin...@gmail.com>: > Hi, everyone. Now, Flink can't do with auto-scaling and we can realize > this > by restart the savepoint with different parallelism. I wonder how flink > handle with state managerment. For example, for parallelism=3 to 4, how the > state of 3 deal with new parallelism? Can you explain the internal to me ? > Thanks a lot. > > > > > > -- > View this message in context: http://apache-flink-user- > mailing-list-archive.2336050.n4.nabble.com/manual-scaling- > with-savepoint-tp10974.html > Sent from the Apache Flink User Mailing List archive. mailing list archive > at Nabble.com. >
