YarnTaskExecutorRunner should contains MapReduce classes

2021-07-22 Thread chenkaibit
Hi: I followed instructions described in [https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/table/hive] and tested hive streaming sink, met this exception Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.JobConf [http://apache-flink.147419.n

flink job will restart over and over again if a taskmanager's disk damages

2020-10-21 Thread chenkaibit
Hi everyone: I met this Exception when a hard disk was damaged: https://issues.apache.org/jira/secure/attachment/13009035/13009035_flink_disk_error.png I checked the code and found that flink will create a temp file when Record length > 5 MB: // SpillingAdaptiveSpanningRecordDeserializer.java