Re: Flink state migration from 1.9 to 1.13

2022-04-15 Thread XU Qinghui
Thanks Martijn. Now it's more clear to me of the issue. I'll try to see if the state processor API could help to convert the 1.9 savepoints to 1.13 compatible. BR, Le jeu. 14 avr. 2022 à 11:47, Martijn Visser a écrit : > Hi Qinghui, > > If you're using SQL, please be aware that there are unfor

Re: Flink state migration from 1.9 to 1.13

2022-04-14 Thread Martijn Visser
Hi Qinghui, If you're using SQL, please be aware that there are unfortunately no application state compatibility guarantees. You can read more about this in the documentation [1]. This is why the community has been working on FLIP-190 to support this in future versions [2] Best regards, Martijn

Re: Flink state migration from 1.9 to 1.13

2022-04-14 Thread XU Qinghui
Hello Yu'an, Thanks for the reply. I'm using the SQL api, not using the `DataStream` API in the job. So there's no `keyby` call directly in our code, but we do have some `group by` and joins in the SQL. (We are using deprecated table planners both before and after migration) Do you know what could

Re: Flink state migration from 1.9 to 1.13

2022-04-13 Thread yu'an huang
Hi Qinghui, Did you used a difference keyby() for your KeyedCoProcesserOperator? For example, did you use a fied name (keyBy(“id”)) in 1.9 and while use a lambda (keyBy(e->e.getId()) in 1.13. This will make the key serializer incompatible. You may reference this link for how to use Apache Flink

Flink state migration from 1.9 to 1.13

2022-04-13 Thread XU Qinghui
Hello dear community, We are trying to upgrade our flink from 1.9 to 1.13, but it seems the same job running in 1.13 cannot restore the checkpoint / savepoint created by 1.9. The stacktrace looks like: java.lang.Exception: Exception while creating StreamOperatorStateContext. at org.apache.fli