vinoyang created KYLIN-3963:
-------------------------------
Summary: Provide a custom serializer for PercentileCounter
Key: KYLIN-3963
URL: https://issues.apache.org/jira/browse/KYLIN-3963
Project: Kylin
Issue Type: Sub-task
Components: Flink Engine
Reporter: vinoyang
Assignee: vinoyang
{{PercentileCounter}} contains an instance field of Type {{TDigest which is not
serializable, so it is marked as {{}}transient}}. By default, it triggers a NPE
when use Flink cube engine:
{code:java}
Caused by: org.apache.flink.client.program.ProgramInvocationException: Job
failed. (JobID: 8077ff0e0ff01a26f62b1babe90a34b7) at
org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:268)
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:487)
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:475) at
org.apache.flink.client.program.ContextEnvironment.execute(ContextEnvironment.java:62)
at
org.apache.kylin.engine.flink.FlinkCubingByLayer.execute(FlinkCubingByLayer.java:196)
at
org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
... 18 more Caused by: org.apache.flink.runtime.client.JobExecutionException:
Job execution failed. at
org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146)
at
org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:265)
... 23 more Caused by: java.lang.NullPointerException at
org.apache.kylin.measure.percentile.PercentileCounter.writeRegisters(PercentileCounter.java:68)
at
org.apache.kylin.measure.percentile.PercentileSerializer.serialize(PercentileSerializer.java:70)
at
org.apache.kylin.measure.percentile.PercentileSerializer.serialize(PercentileSerializer.java:26)
at org.apache.kylin.measure.MeasureCodec.encode(MeasureCodec.java:76) at
org.apache.kylin.measure.BufferedMeasureCodec.encode(BufferedMeasureCodec.java:93)
at
org.apache.kylin.engine.flink.FlinkCubingByLayer$1.map(FlinkCubingByLayer.java:234)
at
org.apache.kylin.engine.flink.FlinkCubingByLayer$1.map(FlinkCubingByLayer.java:218)
at org.apache.flink.runtime.operators.MapDriver.run(MapDriver.java:103) at
org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503) at
org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368) at
org.apache.flink.runtime.taskmanager.Task.run(Task.java:704) at
java.lang.Thread.run(Thread.java:745)
{code}
So we should provide a customized serializer for {{PercentileCounter}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
