Re: [DISCUSS] Breaking Savepoint Compatibility from 1.1 to 1.2

2016-08-15 Thread Ufuk Celebi
I agree that it is acceptable to break it (again). At the moment, we already prohibit users from resuming 1.0 savepoints with 1.1 and so far (~1 week since release) no one complained. I think that at this point most users use it only with the same major version as they used when triggering the save

Re: [DISCUSS] Breaking Savepoint Compatibility from 1.1 to 1.2

2016-08-12 Thread Maximilian Michels
Hi Aljoscha, I'm not very deep into the state backend implementation. However, I think a breaking change is unavoidable with the new key groups. The only way that we achieve backwards-compatibility is to include a translator from the old state format to the new one. As you already mentioned, this

Re: [DISCUSS] Breaking Savepoint Compatibility from 1.1 to 1.2

2016-08-11 Thread Gyula Fóra
Hi, I think this is a very important change for the the future of the system that provides a much cleaner internal representation of the states. You are right that this can in theory break programs written in 1.1 when upgraded to 1.2 but I wonder if this will be actually a practical problem in th

[DISCUSS] Breaking Savepoint Compatibility from 1.1 to 1.2

2016-08-11 Thread Aljoscha Krettek
Hi, Stefan and I are currently working on preparing our state infrastructure for the introduction of key-grouped state. This is the parent issue for key-grouped state https://issues.apache.org/jira/browse/FLINK-3755 while this is the specific issue that we are currently working on https://issues.ap