Hi,
I want to use t-digest by Ted Dunning (
https://github.com/tdunning/t-digest/blob/master/src/main/java/com/tdunning/math/stats/ArrayDigest.java)
on Flink.
Locally that works perfectly. But on the cluster I get the following error:
java.lang.Exception: Call to registerInputOutput() of invokable failed
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:504)
at java.lang.Thread.run(Thread.java:853)
Caused by: java.lang.RuntimeException: Initializing the output handlers
failed: Could not load deserializer from the configuration.
at
org.apache.flink.runtime.operators.RegularPactTask.registerInputOutput(RegularPactTask.java:256)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:501)
... 1 more
Caused by: java.lang.RuntimeException: Could not load deserializer from the
configuration.
at
org.apache.flink.api.java.typeutils.runtime.RuntimeSerializerFactory.readParametersFromConfig(RuntimeSerializerFactory.java:83)
at
org.apache.flink.runtime.operators.util.TaskConfig.getTypeSerializerFactory(TaskConfig.java:1085)
at
org.apache.flink.runtime.operators.util.TaskConfig.getOutputSerializer(TaskConfig.java:543)
at
org.apache.flink.runtime.operators.RegularPactTask.getOutputCollector(RegularPactTask.java:1267)
at
org.apache.flink.runtime.operators.RegularPactTask.initOutputs(RegularPactTask.java:1375)
at
org.apache.flink.runtime.operators.RegularPactTask.initOutputs(RegularPactTask.java:1076)
at
org.apache.flink.runtime.operators.RegularPactTask.registerInputOutput(RegularPactTask.java:254)
... 2 more
Caused by: java.io.InvalidClassException: java.lang.Integer; local class
incompatible: stream classdesc serialVersionUID = -8644826526760479189,
local class serialVersion
UID = 1360826667806852920
at
java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:698)
at
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1705)
at
java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1600)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1873)
at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1432)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:428)
at
org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:302)
at
org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:264)
at
org.apache.flink.api.java.typeutils.runtime.RuntimeSerializerFactory.readParametersFromConfig(RuntimeSerializerFactory.java:76)
... 8 more
Is this an issue of Kryo in Flink? Or do I have to implement a custom
serializer?
Thank you for your help,
Felix
