Re: Transition Flink job from Java to Scala with state migration

2020-10-05 Thread Daksh Talwar
Thanks for the pointers Andrey, they helped us zero in on the issue. We found that it was a bug in our Scala code that caused Flink to use Kryo serializer to read Avro-based state. We've fixed it, and are able to evolve state-schema seamlessly now. Cheers, Daksh On Tue, Aug 25, 2020 at 9:53 PM A

Re: Transition Flink job from Java to Scala with state migration

2020-08-25 Thread Andrey Zagrebin
Hi Daksh, You need to find which type causes the problem: Long, MyCustomObject or maybe something else. You could share the logs with full exception stack trace. My guess is that your scala code uses another serializer for the failing type. See also docs to understand serialization in Flink [1] Wh

Transition Flink job from Java to Scala with state migration

2020-08-25 Thread Daksh Talwar
Hello, We run a Stream API based Flink application on 1.10.0, coded in Java. While moving this job to Scala (reasons unrelated to Flink/this application), we are getting the following error when trying to instantiate Scala job from a savepoint taken in the Java job: *org.apache.flink.util.StateMi