Hi Gerard,
you are correct, Kryo serializers are only used when no built-in Flink
serializer is available.
Actually, the tuple and case class serializers are one of the most
performant serializers in Flink (due to their fixed length, no null
support). If you really want to reduce the serialization overhead you
could look into the object reuse mode. We had this topic on the mailing
list recently, I will just copy it here:
If you want to improve the performance of a collect() between operators,
you could also enable object reuse. You can read more about this here
[1] (section "Issue 2: Object Reuse"), but make sure your implementation
is correct because an operator could modify the objects of follwing
operators.
I hope this helps.
Regards,
Timo
[1]
https://data-artisans.com/blog/curious-case-broken-benchmark-revisiting-apache-flink-vs-databricks-runtime
Am 17.08.18 um 17:29 schrieb gerardg:
Hello,
I can't seem to be able to override the CaseClassSerializer with my custom
serializer. I'm using env.getConfig.addDefaultKryoSerializer() to add the
custom serializer but I don't see it being used. I guess it is because it
only uses Kryo based serializers if it can't find a Flink serializer?
Is then worth it to replace the CaseClassSerializer with a custom
serializer? (when I profile the CaseClassSerializer.(de)serialize method
appears as the most used so I wanted to give it a try) If so, how can I do
it?
Thanks,
Gerard
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
