Re: Serialization and kryo

2015-08-17 Thread Robert Metzger
Hi Jay, this is how you can register a custom Kryo serializer, yes. Flink has this project (https://github.com/magro/kryo-serializers) as a dependency. It contains a lot of Kryo Serializers for common types. They also added support for for Guava's ImmutableMap, but the version we are using (0.27)

Re: Serialization and kryo

2015-08-16 Thread jay vyas
ah ok gotcha, sort of like this ? ExecutionEnvironment.getExecutionEnvironment().registerTypeWithKryoSerializer(Product.class, new com.esotericsoftware.kryo.Serializer() { PS thanks, i think this should work... will update On Sun, Aug 16, 2015 at 11:01 AM, Stephan Ewen wrote: > Hi! > > The iss

Re: Serialization and kryo

2015-08-16 Thread Stephan Ewen
Hi! The issue you refer to is: Flink automatically registers types it encounters at Kryo, if the types are not handled by Flink itself. You can turn that off in any program by "ExecutionEnvionment.getConfig().disableAutoTypeRegistration()". I am not sure that causes your problem, as this makes Kr

Re: Serialization and kryo

2015-08-16 Thread jay vyas
1. Just wanted to updated.. possibly I'm hitting ? FLINK-1417 Automatically register nested types at Kryo ... On Sun, Aug 16, 2015 at 10:38 AM, jay vyas wrote: > Hi flink. > > I just ran into the following serialization error in BigPet

Serialization and kryo

2015-08-16 Thread jay vyas
Hi flink. I just ran into the following serialization error in BigPetStore Flink. It appears to be that someone is trying to add elements to a map during serialization. I doubt this is a bug in flink because it would have surely been caught early on. Some possible explanations ... 1 - Probably