Gordon Thanks for the detailed response. I have verified your assumption and that is, unfortunately, the case.
I also looked into creating a custom Kryo serializer but I got stuck on serializing arrays of complex types. It seems like this isn't trivial at all with Kryo. As an alternative, I've been looking into using Avro only for the Flink buffers. Basically, as a first step, we'd still be sending JSON messages through Kafka but we would use a custom TypeSerializer that converts the case classes to bytes using Avro and vice versa. Unfortunately, documentation is really scarce. In a different topic, http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Serialization-performance-td12019.html, it says that Avro is a bit of an odd citizen and that the AvroSerializer provided by Flink uses Kryo. This confirms what I've found by going through the source code of Flink myself. I hope that you can provide me with some pointers. Is extending TypeSerializer[T] the best way forward if we only want to use Avro for state buffers and thus utilize Avro's schema migration facilities? Any pointers would be greatly appreciated! Kind regards Marc -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
