Hi Team,
Please ignore the previous mail.
I am trying to create a connector for carbondata. Currently i am working on
creating OutputFormat for carbondata. For that i need to create a
carboncontext which i initialise in configure method. Internally
CarbonContext uses sparkContext.
I am able to create carbonContext through my code but when it run statement
"executionEnvironment.execute()
it throws the below error:
Exception in thread "main" org.apache.flink.optimizer.CompilerException:
Error translating node 'Data Sink
"org.apache.flink.CarbonDataOutputFormat@76225435"
: NONE [[ GlobalProperties [partitioning=RANDOM_PARTITIONED] ]] [[
LocalProperties [ordering=null, grouped=null, unique=null] ]]': Could not
write the user code wrapper class org.apache.flink.api.common.
operators.util.UserCodeObjectWrapper : java.io.NotSerializableException:
org.apache.spark.sql.hive.client.ClientWrapper
at org.apache.flink.optimizer.plantranslate.JobGraphGenerator.preVisit(
JobGraphGenerator.java:381)
at org.apache.flink.optimizer.plantranslate.JobGraphGenerator.preVisit(
JobGraphGenerator.java:106)
at org.apache.flink.optimizer.plan.SingleInputPlanNode.
accept(SingleInputPlanNode.java:198)
at org.apache.flink.optimizer.plan.OptimizedPlan.accept(
OptimizedPlan.java:128)
at org.apache.flink.optimizer.plantranslate.JobGraphGenerator.
compileJobGraph(JobGraphGenerator.java:192)
at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:188)
at org.apache.flink.api.java.LocalEnvironment.execute(
LocalEnvironment.java:91)
at org.apache.flink.api.java.ExecutionEnvironment.execute(
ExecutionEnvironment.java:926)
at CarbonJDBCExample.main(CarbonJDBCExample.java:86)
Caused by:
org.apache.flink.runtime.operators.util.CorruptConfigurationException:
Could not write the user code wrapper class org.apache.flink.api.common.
operators.util.UserCodeObjectWrapper : java.io.NotSerializableException:
org.apache.spark.sql.hive.client.ClientWrapper
at org.apache.flink.runtime.operators.util.TaskConfig.
setStubWrapper(TaskConfig.java:281)
at org.apache.flink.optimizer.plantranslate.JobGraphGenerator.
createDataSinkVertex(JobGraphGenerator.java:903)
at org.apache.flink.optimizer.plantranslate.JobGraphGenerator.preVisit(
JobGraphGenerator.java:278)
... 8 more
Caused by: java.io.NotSerializableException: org.apache.spark.sql.hive.
client.ClientWrapper
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)
at java.io.ObjectOutputStream.defaultWriteFields(
ObjectOutputStream.java:1548)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
at java.io.ObjectOutputStream.writeOrdinaryObject(
ObjectOutputStream.java:1432)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.defaultWriteFields(
ObjectOutputStream.java:1548)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
at java.io.ObjectOutputStream.writeOrdinaryObject(
ObjectOutputStream.java:1432)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.defaultWriteFields(
ObjectOutputStream.java:1548)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
at java.io.ObjectOutputStream.writeOrdinaryObject(
ObjectOutputStream.java:1432)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
at org.apache.flink.util.InstantiationUtil.serializeObject(
InstantiationUtil.java:317)
at org.apache.flink.util.InstantiationUtil.writeObjectToConfig(
InstantiationUtil.java:254)
at org.apache.flink.runtime.operators.util.TaskConfig.
setStubWrapper(TaskConfig.java:279)
... 10 more
Help me with the same.
On Mon, Apr 17, 2017 at 12:39 PM, Sangeeta Gulia <sangeeta.gu...@knoldus.in>
wrote:
> Hi Team,
>
> I am trying to create a connector for carbondata. Currently i am working
> on creating OutputFormat for carbondata. For that i need to create a
> carboncontext which i initialise in configure method. Internally
> CarbonContext uses sparkContext.
>
> I am able to create carbonContext through my code but when it run statement
> "executionEnvironment.execute()
>
> Exception in thread "main" org.apache.flink.optimizer.CompilerException:
> Error translating node 'Data Sink "org.apache.flink.
> CarbonDataOutputFormat@76225435" : NONE [[ GlobalProperties
> [partitioning=RANDOM_PARTITIONED] ]] [[ LocalProperties [ordering=null,
> grouped=null, unique=null] ]]': Could not write the user code wrapper class
> org.apache.flink.api.common.operators.util.UserCodeObjectWrapper : java.io
> .NotSerializableException: org.apache.spark.sql.hive.client.ClientWrapper
> at org.apache.flink.optimizer.plantranslate.JobGraphGenerator.preVisit(
> JobGraphGenerator.java:381)
> at org.apache.flink.optimizer.plantranslate.JobGraphGenerator.preVisit(
> JobGraphGenerator.java:106)
> at org.apache.flink.optimizer.plan.SingleInputPlanNode.
> accept(SingleInputPlanNode.java:198)
> at org.apache.flink.optimizer.plan.OptimizedPlan.accept(
> OptimizedPlan.java:128)
> at org.apache.flink.optimizer.plantranslate.JobGraphGenerator.
> compileJobGraph(JobGraphGenerator.java:192)
> at org.apache.flink.client.LocalExecutor.executePlan(
> LocalExecutor.java:188)
> at org.apache.flink.api.java.LocalEnvironment.execute(
> LocalEnvironment.java:91)
> at org.apache.flink.api.java.ExecutionEnvironment.execute(
> ExecutionEnvironment.java:926)
> at CarbonJDBCExample.main(CarbonJDBCExample.java:86)
> Caused by:
> org.apache.flink.runtime.operators.util.CorruptConfigurationException:
> Could not write the user code wrapper class org.apache.flink.api.common.
> operators.util.UserCodeObjectWrapper : java.io.NotSerializableException:
> org.apache.spark.sql.hive.client.ClientWrapper
> at org.apache.flink.runtime.operators.util.TaskConfig.
> setStubWrapper(TaskConfig.java:281)
> at org.apache.flink.optimizer.plantranslate.JobGraphGenerator.
> createDataSinkVertex(JobGraphGenerator.java:903)
> at org.apache.flink.optimizer.plantranslate.JobGraphGenerator.preVisit(
> JobGraphGenerator.java:278)
> ... 8 more
> Caused by: java.io.NotSerializableException: org.apache.spark.sql.hive.
> client.ClientWrapper
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)
> at java.io.ObjectOutputStream.defaultWriteFields(
> ObjectOutputStream.java:1548)
> at java.io.ObjectOutputStream.writeSerialData(
> ObjectOutputStream.java:1509)
> at java.io.ObjectOutputStream.writeOrdinaryObject(
> ObjectOutputStream.java:1432)
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
> at java.io.ObjectOutputStream.defaultWriteFields(
> ObjectOutputStream.java:1548)
> at java.io.ObjectOutputStream.writeSerialData(
> ObjectOutputStream.java:1509)
> at java.io.ObjectOutputStream.writeOrdinaryObject(
> ObjectOutputStream.java:1432)
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
> at java.io.ObjectOutputStream.defaultWriteFields(
> ObjectOutputStream.java:1548)
> at java.io.ObjectOutputStream.writeSerialData(
> ObjectOutputStream.java:1509)
> at java.io.ObjectOutputStream.writeOrdinaryObject(
> ObjectOutputStream.java:1432)
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
> at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
> at org.apache.flink.util.InstantiationUtil.serializeObject(
> InstantiationUtil.java:317)
> at org.apache.flink.util.InstantiationUtil.writeObjectToConfig(
> InstantiationUtil.java:254)
> at org.apache.flink.runtime.operators.util.TaskConfig.
> setStubWrapper(TaskConfig.java:279)
> ... 10 more
>
>
>
> --
> Warm Regards,
>
> Sangeeta Gulia
> Software Consultant
> Knoldus Software LLP
>
--
Warm Regards,
Sangeeta Gulia
Software Consultant
Knoldus Software LLP
