I forgot to mention that I'm using Flink 1.6.2 compiled for cloudera CDH 5.11.2:
/opt/shared/devel/apache-maven-3.3.9/bin/mvn clean install -Dhadoop.version=2.6.0-cdh5.11.2 -Dhbase.version=1.2.0-cdh5.11.2 -Dhadoop.core.version=2.6.0-mr1-cdh5.11.2 -DskipTests -Pvendor-repos On Wed, Nov 7, 2018 at 11:48 AM Flavio Pompermaier <[email protected]> wrote: > Hi to all, > we tried to upgrade our jobs to Flink 1.6.2 but now we get the following > error (we saw a similar issue with spark that was caused by different java > version on the cluster servers so we checked them and they are all to the > same version - oracle-8-191): > > Caused by: org.apache.flink.runtime.client.JobExecutionException: Cannot > initialize task 'DataSink (Parquet write: > hdfs:/rivela/1/1/0_staging/parquet)': Deserializing the OutputFormat > (org.apache.flink.api.java.hadoop.mapreduce.HadoopOutputFormat@54a4c7c8) > failed: unread block data > at > org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder.buildGraph(ExecutionGraphBuilder.java:220) > at > org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder.buildGraph(ExecutionGraphBuilder.java:100) > at > org.apache.flink.runtime.jobmaster.JobMaster.createExecutionGraph(JobMaster.java:1151) > at > org.apache.flink.runtime.jobmaster.JobMaster.createAndRestoreExecutionGraph(JobMaster.java:1131) > at > org.apache.flink.runtime.jobmaster.JobMaster.<init>(JobMaster.java:294) > at > org.apache.flink.runtime.jobmaster.JobManagerRunner.<init>(JobManagerRunner.java:157) > ... 10 more > Caused by: java.lang.Exception: Deserializing the OutputFormat > (org.apache.flink.api.java.hadoop.mapreduce.HadoopOutputFormat@54a4c7c8) > failed: unread block data > at > org.apache.flink.runtime.jobgraph.OutputFormatVertex.initializeOnMaster(OutputFormatVertex.java:63) > at > org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder.buildGraph(ExecutionGraphBuilder.java:216) > ... 15 more > Caused by: java.lang.IllegalStateException: unread block data > at > java.io.ObjectInputStream$BlockDataInputStream.setBlockDataMode(ObjectInputStream.java:2783) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1605) > at > java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287) > at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211) > at > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573) > at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431) > at > org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:502) > at > org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:489) > at > org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:477) > at > org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:438) > at > org.apache.flink.runtime.operators.util.TaskConfig.getStubWrapper(TaskConfig.java:288) > at > org.apache.flink.runtime.jobgraph.OutputFormatVertex.initializeOnMaster(OutputFormatVertex.java:60) > ... 16 more > > > Has anyone faced this problem before? How can we try to solve it? > Best,Flavio >
