Re: Serialization questions

2018-07-18 Thread Andrey Zagrebin
Another way would be also to make `EntitonAtom` extend `org.apache.flink.types.Value` and implement `IOReadableWritable` using custom (Kryo) serialisation. > On 18 Jul 2018, at 18:27, Andrey Zagrebin wrote: > > Hi Flavio, > > According to the current implementation of `disableGenericTypes`, t

Re: Serialization questions

2018-07-18 Thread Andrey Zagrebin
Hi Flavio, According to the current implementation of `disableGenericTypes`, the exception you get should be valid because Kryo still has to be used for `EntitonAtom` which might be classified as generic (non-serialisable by Flink). You cannot specify exceptions for this check at the moment. I

Re: Serialization questions

2018-07-18 Thread Flavio Pompermaier
Hi Minglei, using the registerTypeWithKryoSerializer with the 3 classes works (without disableGenericTypes) but the problem is that I would like to avoid Kryo serialization if this is useful to speedup the job performance, and thus I'd like to be able to run all jobs with disableGenericTypes. Best

Re: Serialization questions

2018-07-18 Thread zhangminglei
Hi, Flavio > addDefaultKryoSerializer differs from registerTypeWithKryoSerializer because > addDefaultKryoSerializer use the passed serializer also for subclasses of the > configured class. Am I right? This is not very clear in the method's Javadoc… I think it is not exactly a problem with flin

Serialization questions

2018-07-17 Thread Flavio Pompermaier
Hi to all, I was trying to check whether our jobs are properly typed or not. I've started disabling generic types[1] in order to discover untyped transformations and so I added the proper returns() to operators. Unfortunately there are jobs where we serialize Thrift and DateTime objects, so I need