Hi! Sorry for not responding earlier.
Thanks for taking an initiative there and drafting this. I would like to have a close look after the 0.9 release is out. I am in favor of making this a major item for 0.10, to get the state with all its characteristics defined in a principles fashion: repartitionable, synchronous vs. asynchronous checkpointing, full vs incremental checkpoints. Greetings, Stephan On Fri, May 29, 2015 at 2:13 PM, Paris Carbone <par...@kth.se> wrote: > Hello fellow squirrels! > > > We just made a PR [1] of a prototype targeting flexible state management > for streaming tasks with the prospect of further implementing on top > different strategies such as lazy state updates, incremental snapshots and > state partitioning. You can read more regarding the motivation behind this > design in the doc Gyula created [2]. > > > As described in the doc, "managed" operator state and updates are > explicitly specified by the OperatorState abstraction. Furthermore, > OperatorState can be partitioned by a key and retrieved by the getState > method. If the state is partitioned the getState method implicitly returns > the state of the respective key that is currently processed by the > operator. It will still be possible to add functionality in order to access > keys arbitrarily if needed but for now this looks hopefully clear enough. > > > Let's discuss here how we can use/modify this api to fit our general needs > and vision for state management. > > > Paris > > > [1] https://github.com/apache/flink/pull/747 > > [2] > https://docs.google.com/document/d/1nTn4Tpafsnt-TCT6L1vlHtGGgRevU90yRsUQEmkRMjk > > ? > >
