Hi Minglei, using the registerTypeWithKryoSerializer with the 3 classes works (without disableGenericTypes) but the problem is that I would like to avoid Kryo serialization if this is useful to speedup the job performance, and thus I'd like to be able to run all jobs with disableGenericTypes.
Best, Flavio On Wed, Jul 18, 2018 at 11:10 AM, zhangminglei <18717838...@163.com> wrote: > Hi, Flavio > > > - addDefaultKryoSerializer differs from registerTypeWithKryoSerializer > because addDefaultKryoSerializer use the passed serializer also for > subclasses of the configured class. Am I right? This is not very clear in > the method's Javadoc… > > > I think it is not exactly a problem with flink. Instead of a kryo problem. > For example, addDefaultKryoSerializer corresponding to the > addDefaultSerializer(int[].class, IntArraySerializer.class) in kryo, > whereas registerTypeWithKryoSerializer corresponding to the > register(int.class, new IntSerializer()) in kryo.With register, you > explicitly assign an id for that type plus serializer. The default > serializer just tells kryo which serializer to use when this type has to be > serialized, kryo will then implicitly register the serializer. And the > advantage of using register would be [1]. when setting > setRegistrationRequired(true), which is recommended (and will be the > default in 5.0), you'd have to register every occurring type explicitly. > > > - how to avoid that exception? > > You can try below and do not make disableGenericTypes and see what happens. > > env.registerTypeWithKryoSerializer(DateTime.class, > JodaDateTimeSerializer.class); > env.registerTypeWithKryoSerializer(EntitonAtom.class, > TBaseSerializer.class); > env.registerTypeWithKryoSerializer(EntitonQuad.class, > TBaseSerializer.class); > > > [1] https://github.com/EsotericSoftware/kryo/blob/ > master/README.md#registration > > Cheers > Minglei > > > > > > > 在 2018年7月17日，下午9:00，Flavio Pompermaier <pomperma...@okkam.it> 写道： > > Hi to all, > I was trying to check whether our jobs are properly typed or not. > I've started disabling generic types[1] in order to discover untyped > transformations and so I added the proper returns() to operators. > > Unfortunately there are jobs where we serialize Thrift and DateTime > objects, so I need to properly configure the serializers in the > ExecutionEnvironment: > > env.registerTypeWithKryoSerializer(DateTime.class, > JodaDateTimeSerializer.class); > env.getConfig().addDefaultKryoSerializer(EntitonAtom.class, > TBaseSerializer.class); > env.getConfig().addDefaultKryoSerializer(EntitonQuad.class, > TBaseSerializer.class); > > Those jobs don't work when I disable generic types and I get the following > exception: > > Exception in thread "main" java.lang.UnsupportedOperationException: > Generic types have been > disabled in the ExecutionConfig and type xxx.EntitonAtom is treated as a > generic type. > > I have a couple of questions: > > - addDefaultKryoSerializer differs from registerTypeWithKryoSerializer > because addDefaultKryoSerializer use the passed serializer also for > subclasses of the configured class. Am I right? This is not very clear in > the method's Javadoc... > - how to avoid that exception? > > Best, > Flavio > > [1] env.getConfig().disableGenericTypes(); > > >
