Hi Stephan,
sorry for the late answer, didn't have access to cluster.
Here is log and stacktrace.
Hope this helps,
Maxim.
---------------------------------------------------------------------------------------------------------------------------------
2019-09-16 18:00:31,804 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
--------------------------------------------------------------------------------
2019-09-16 18:00:31,806 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Starting
YarnSessionClusterEntrypoint (Version: 1.9.0, Rev:9c32ed9, Date:19.08.2019
@ 16:16:55 UTC)
2019-09-16 18:00:31,806 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - OS
current user: run
2019-09-16 18:00:32,285 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Current
Hadoop/Kerberos user: run
2019-09-16 18:00:32,285 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JVM: Java
HotSpot(TM) 64-Bit Server VM - Oracle Corporation - 1.8/25.152-b16
2019-09-16 18:00:32,285 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Maximum
heap size: 161 MiBytes
2019-09-16 18:00:32,285 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
JAVA_HOME: /pb/apps/java/jdk1.8.0_152/
2019-09-16 18:00:32,286 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Hadoop
version: 2.7.0-mapr-1808
2019-09-16 18:00:32,286 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JVM
Options:
2019-09-16 18:00:32,286 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
-Xms168m
2019-09-16 18:00:32,286 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
-Xmx168m
2019-09-16 18:00:32,286 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
-Dlog.file=/opt/mapr/hadoop/hadoop-2.7.0/logs/userlogs/application_1568125115996_13508/container_e64_1568125115996_13508_01_000001/jobmanager.log
2019-09-16 18:00:32,286 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
-Dlog4j.configuration=file:log4j.properties
2019-09-16 18:00:32,286 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Program
Arguments: (none)
2019-09-16 18:00:32,287 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
Classpath:
log4j.properties:flink.jar:flink-conf.yaml::/opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/hadoop-common-2.7.0-mapr-1808-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/hadoop-common-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/hadoop-nfs-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jackson-databind-2.9.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/activation-1.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jsr305-3.0.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jersey-server-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/junit-4.11.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/log4j-1.2.17.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/asm-3.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprdb-6.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/avro-1.7.6.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mockito-all-1.8.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jettison-1.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mapr-spark-yarn-shuffle.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-cli-1.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/netty-3.6.2.Final.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-codec-1.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jetty-6.1.26.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-collections-3.2.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/xz-1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-compress-1.4.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jetty-util-6.1.26.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-configuration-1.6.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/json-smart-1.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-digester-1.8.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprfs-6.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-httpclient-3.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprdb-6.1.0-mapr-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-io-2.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/azure-data-lake-store-sdk-2.1.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-lang-2.6.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-logging-1.1.3.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/hadoop-auth-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-math3-3.1.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprdb-cdc-6.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-net-3.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jsch-0.1.54.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/curator-client-2.7.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/json-1.8.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/curator-framework-2.7.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/slf4j-api-1.7.12.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/curator-recipes-2.7.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mapr-streams-6.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/gson-2.2.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/antlr4-runtime-4.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jets3t-0.9.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/aws-java-sdk-bundle-1.11.199.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/slf4j-log4j12-1.7.12.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mapr-hbase-6.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprdb-shell-6.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/zookeeper-3.4.11-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/hamcrest-core-1.3.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/azure-storage-2.0.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/htrace-core-3.1.0-incubating.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/guava-14.0.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/httpclient-4.5.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ojai-3.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/httpcore-4.4.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mapr-hbase-6.1.0-mapr-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mysql-connector-java-5.1.25.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jsp-api-2.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprfs-6.1.0-mapr-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mapr-java-utils-6.1.0-mapr-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ojai-scala-3.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mapr-tools-6.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jersey-core-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jackson-core-2.9.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jersey-json-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/paranamer-2.3.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mapr-tools-6.1.0-mapr-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/servlet-api-2.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jackson-annotations-2.9.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/snappy-java-1.0.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/stax-api-1.0-2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/xmlenc-0.52.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/hadoop-annotations-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/hadoop-aws-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/hadoop-azure-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/hadoop-azure-datalake-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprfs-diagnostic-tools-6.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/central-logging-6.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprdb-cdc-6.1.0-mapr-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprdb-mapreduce-6.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprdb-mapreduce-6.1.0-mapr-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprdb-shell-6.1.0-mapr-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mapr-streams-6.1.0-mapr-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ojai-mapreduce-3.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/hadoop-hdfs-2.7.0-mapr-1808-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/hadoop-hdfs-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/hadoop-hdfs-nfs-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/asm-3.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/commons-io-2.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/guava-14.0.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/htrace-core-3.1.0-incubating.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/netty-all-4.0.23.Final.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-client-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-api-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-common-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-registry-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-common-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/activation-1.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/asm-3.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-cli-1.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-codec-1.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-collections-3.2.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-io-2.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-lang-2.6.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/eval-0.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/guava-14.0.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/guice-3.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/javax.inject-1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jersey-client-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jersey-core-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jersey-json-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jersey-server-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jettison-1.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jetty-6.1.26.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/log4j-1.2.17.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/servlet-api-2.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/xz-1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/eval-0.5-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/fst-2.24.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/zookeeper-3.4.11-mapr-1808-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/zookeeper-3.4.11-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/audience-annotations-0.5.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/javassist-3.18.1-GA.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/objenesis-2.1.jar
2019-09-16 18:00:32,287 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
--------------------------------------------------------------------------------
2019-09-16 18:00:32,288 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Registered
UNIX signal handlers for [TERM, HUP, INT]
2019-09-16 18:00:32,290 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - YARN
daemon is running as: run Yarn client user obtainer: run
2019-09-16 18:00:32,292 INFO
org.apache.flink.configuration.GlobalConfiguration - Loading
configuration property: fs.default-scheme, maprfs:///user/run/flink/
2019-09-16 18:00:32,292 INFO
org.apache.flink.configuration.GlobalConfiguration - Loading
configuration property: yarn.tags, flinksessioncluster
2019-09-16 18:00:32,293 INFO
org.apache.flink.configuration.GlobalConfiguration - Loading
configuration property: high-availability.cluster-id,
application_1568125115996_13508
2019-09-16 18:00:32,293 INFO
org.apache.flink.configuration.GlobalConfiguration - Loading
configuration property: FLINK_PLUGINS_DIR, /home/run/flink-1.9.0/plugins
2019-09-16 18:00:32,293 INFO
org.apache.flink.configuration.GlobalConfiguration - Loading
configuration property: state.savepoints.dir,
maprfs:///user/run/flink/savepoints/
2019-09-16 18:00:32,293 INFO
org.apache.flink.configuration.GlobalConfiguration - Loading
configuration property: high-availability.zookeeper.path.root, /flink
2019-09-16 18:00:32,293 INFO
org.apache.flink.configuration.GlobalConfiguration - Loading
configuration property: high-availability.storageDir,
maprfs:///user/run/flink/recovery
2019-09-16 18:00:32,293 INFO
org.apache.flink.configuration.GlobalConfiguration - Loading
configuration property: high-availability.zookeeper.quorum,
server11.internal.com:5181
2019-09-16 18:00:32,293 INFO
org.apache.flink.configuration.GlobalConfiguration - Loading
configuration property: internal.cluster.execution-mode, NORMAL
2019-09-16 18:00:32,293 INFO
org.apache.flink.configuration.GlobalConfiguration - Loading
configuration property: high-availability, zookeeper
2019-09-16 18:00:32,293 INFO
org.apache.flink.configuration.GlobalConfiguration - Loading
configuration property: yarn.application-attempts, 10
2019-09-16 18:00:32,293 INFO
org.apache.flink.configuration.GlobalConfiguration - Loading
configuration property: taskmanager.numberOfTaskSlots, 1
2019-09-16 18:00:32,293 INFO
org.apache.flink.configuration.GlobalConfiguration - Loading
configuration property: state.backend, filesystem
2019-09-16 18:00:32,293 INFO
org.apache.flink.configuration.GlobalConfiguration - Loading
configuration property: taskmanager.heap.size, 768m
2019-09-16 18:00:32,293 INFO
org.apache.flink.configuration.GlobalConfiguration - Loading
configuration property: state.checkpoints.dir,
maprfs:///user/run/flink/checkpoints/
2019-09-16 18:00:32,294 INFO
org.apache.flink.configuration.GlobalConfiguration - Loading
configuration property: jobmanager.heap.size, 768m
2019-09-16 18:00:32,311 WARN org.apache.flink.configuration.Configuration
- Config uses deprecated configuration key 'web.port'
instead of proper key 'rest.bind-port'
2019-09-16 18:00:32,314 INFO
org.apache.flink.runtime.clusterframework.BootstrapTools - Setting
directories for temporary files to:
/tmp/hadoop-mapr/nm-local-dir/usercache/run/appcache/application_1568125115996_13508
2019-09-16 18:00:32,318 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Starting
YarnSessionClusterEntrypoint.
2019-09-16 18:00:32,318 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Install
default filesystem.
2019-09-16 18:00:32,318 WARN org.apache.flink.core.plugin.PluginConfig
- Environment variable [FLINK_PLUGINS_DIR] is set to
[/home/run/flink-1.9.0/plugins] but the directory doesn't exist
2019-09-16 18:00:32,359 WARN
org.apache.flink.runtime.security.modules.HadoopModule - Hadoop
security is enabled but current login user does not have Kerberos
credentials
2019-09-16 18:00:32,359 INFO
org.apache.flink.runtime.security.modules.HadoopModule - Hadoop
user set to run (auth:CUSTOM)
2019-09-16 18:00:32,362 INFO
org.apache.flink.runtime.entrypoint.ClusterEntrypoint -
Initializing cluster services.
2019-09-16 18:00:32,475 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils - Trying to
start actor system at server11.internal.com:0
2019-09-16 18:00:32,802 INFO akka.event.slf4j.Slf4jLogger
- Slf4jLogger started
2019-09-16 18:00:32,820 INFO akka.remote.Remoting
- Starting remoting
2019-09-16 18:00:32,911 INFO akka.remote.Remoting
- Remoting started; listening on addresses :[akka.tcp://
fl...@server11.internal.com:45399]
2019-09-16 18:00:33,006 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils - Actor
system started at akka.tcp://fl...@server11.internal.com:45399
2019-09-16 18:00:33,018 INFO
org.apache.flink.runtime.fs.maprfs.MapRFsFactory - Trying to
load and instantiate MapR File System
2019-09-16 18:00:33,062 INFO
org.apache.flink.runtime.blob.FileSystemBlobStore - Creating
highly available BLOB storage directory at
maprfs:///user/run/flink/recovery/application_1568125115996_13508/blob
2019-09-16 18:00:33,091 INFO org.apache.flink.runtime.util.ZooKeeperUtils
- Enforcing default ACL for ZK connections
2019-09-16 18:00:33,092 INFO org.apache.flink.runtime.util.ZooKeeperUtils
- Using '/flink/application_1568125115996_13508' as
Zookeeper namespace.
2019-09-16 18:00:33,132 INFO
org.apache.flink.shaded.curator.org.apache.curator.framework.imps.CuratorFrameworkImpl
- Starting
2019-09-16 18:00:33,138 INFO
org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper - Client
environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f,
built on 03/23/2017 10:13 GMT
2019-09-16 18:00:33,138 INFO
org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper - Client
environment:host.name=server11.internal.com
2019-09-16 18:00:33,138 INFO
org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper - Client
environment:java.version=1.8.0_152
2019-09-16 18:00:33,138 INFO
org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper - Client
environment:java.vendor=Oracle Corporation
2019-09-16 18:00:33,138 INFO
org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper - Client
environment:java.home=/pb/apps/java/jdk1.8.0_152/jre
2019-09-16 18:00:33,138 INFO
org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper - Client
environment:java.class.path=log4j.properties:flink.jar:flink-conf.yaml::/opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/hadoop-common-2.7.0-mapr-1808-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/hadoop-common-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/hadoop-nfs-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jackson-databind-2.9.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/activation-1.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jsr305-3.0.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jersey-server-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/junit-4.11.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/log4j-1.2.17.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/asm-3.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprdb-6.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/avro-1.7.6.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mockito-all-1.8.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jettison-1.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mapr-spark-yarn-shuffle.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-cli-1.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/netty-3.6.2.Final.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-codec-1.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jetty-6.1.26.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-collections-3.2.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/xz-1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-compress-1.4.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jetty-util-6.1.26.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-configuration-1.6.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/json-smart-1.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-digester-1.8.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprfs-6.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-httpclient-3.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprdb-6.1.0-mapr-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-io-2.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/azure-data-lake-store-sdk-2.1.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-lang-2.6.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-logging-1.1.3.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/hadoop-auth-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-math3-3.1.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprdb-cdc-6.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/commons-net-3.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jsch-0.1.54.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/curator-client-2.7.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/json-1.8.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/curator-framework-2.7.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/slf4j-api-1.7.12.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/curator-recipes-2.7.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mapr-streams-6.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/gson-2.2.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/antlr4-runtime-4.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jets3t-0.9.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/aws-java-sdk-bundle-1.11.199.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/slf4j-log4j12-1.7.12.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mapr-hbase-6.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprdb-shell-6.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/zookeeper-3.4.11-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/hamcrest-core-1.3.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/azure-storage-2.0.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/htrace-core-3.1.0-incubating.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/guava-14.0.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/httpclient-4.5.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ojai-3.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/httpcore-4.4.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mapr-hbase-6.1.0-mapr-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mysql-connector-java-5.1.25.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jsp-api-2.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprfs-6.1.0-mapr-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mapr-java-utils-6.1.0-mapr-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ojai-scala-3.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mapr-tools-6.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jersey-core-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jackson-core-2.9.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jersey-json-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/paranamer-2.3.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mapr-tools-6.1.0-mapr-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/servlet-api-2.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/jackson-annotations-2.9.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/snappy-java-1.0.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/stax-api-1.0-2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/xmlenc-0.52.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/hadoop-annotations-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/hadoop-aws-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/hadoop-azure-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/hadoop-azure-datalake-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprfs-diagnostic-tools-6.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/central-logging-6.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprdb-cdc-6.1.0-mapr-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprdb-mapreduce-6.1.0-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprdb-mapreduce-6.1.0-mapr-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/maprdb-shell-6.1.0-mapr-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/mapr-streams-6.1.0-mapr-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/common/lib/ojai-mapreduce-3.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/hadoop-hdfs-2.7.0-mapr-1808-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/hadoop-hdfs-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/hadoop-hdfs-nfs-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/asm-3.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/commons-io-2.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/guava-14.0.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/htrace-core-3.1.0-incubating.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/netty-all-4.0.23.Final.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-client-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-api-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-common-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-registry-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-common-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.7.0-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/activation-1.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/asm-3.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-cli-1.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-codec-1.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-collections-3.2.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-io-2.4.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-lang-2.6.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/eval-0.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/guava-14.0.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/guice-3.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/javax.inject-1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jersey-client-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jersey-core-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jersey-json-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jersey-server-1.9.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jettison-1.1.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jetty-6.1.26.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/log4j-1.2.17.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/servlet-api-2.5.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/xz-1.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/eval-0.5-mapr.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/fst-2.24.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/zookeeper-3.4.11-mapr-1808-tests.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/zookeeper-3.4.11-mapr-1808.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/audience-annotations-0.5.0.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/javassist-3.18.1-GA.jar:/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/yarn/lib/objenesis-2.1.jar
2019-09-16 18:00:33,138 INFO
org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper - Client
environment:java.library.path=/opt/mapr/hadoop/hadoop-2.7.0/lib/native:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2019-09-16 18:00:33,138 INFO
org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper - Client
environment:java.io.tmpdir=/tmp
2019-09-16 18:00:33,138 INFO
org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper - Client
environment:java.compiler=<NA>
2019-09-16 18:00:33,138 INFO
org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper - Client
environment:os.name=Linux
2019-09-16 18:00:33,138 INFO
org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper - Client
environment:os.arch=amd64
2019-09-16 18:00:33,138 INFO
org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper - Client
environment:os.version=3.10.0-957.21.3.el7.x86_64
2019-09-16 18:00:33,138 INFO
org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper - Client
environment:user.name=run
2019-09-16 18:00:33,139 INFO
org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper - Client
environment:user.home=/home/run
2019-09-16 18:00:33,139 INFO
org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper - Client
environment:user.dir=/tmp/hadoop-mapr/nm-local-dir/usercache/run/appcache/application_1568125115996_13508/container_e64_1568125115996_13508_01_000001
2019-09-16 18:00:33,139 INFO
org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper -
Initiating client connection, connectString=server11.internal.com:5181
sessionTimeout=60000
watcher=org.apache.flink.shaded.curator.org.apache.curator.ConnectionState@6b530eb9
2019-09-16 18:00:33,151 INFO
org.apache.flink.shaded.zookeeper.org.apache.zookeeper.Login - Client
successfully logged in.
2019-09-16 18:00:33,155 INFO
org.apache.flink.shaded.zookeeper.org.apache.zookeeper.client.ZooKeeperSaslClient
- Client will use GSSAPI as SASL mechanism.
2019-09-16 18:00:33,155 INFO org.apache.flink.runtime.blob.BlobServer
- Created BLOB server storage directory
/tmp/hadoop-mapr/nm-local-dir/usercache/run/appcache/application_1568125115996_13508/blobStore-59e8dea5-cabf-441e-8fbd-598396a22f27
2019-09-16 18:00:33,158 INFO org.apache.flink.runtime.blob.BlobServer
- Started BLOB server at 0.0.0.0:33509 - max concurrent
requests: 50 - max backlog: 1000
2019-09-16 18:00:33,160 INFO
org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn -
Opening socket connection to server
server11.internal.com/10.200.133.203:5181. Will attempt to
SASL-authenticate using Login Context section 'Client'
2019-09-16 18:00:33,162 INFO
org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn -
Socket connection established to server11.internal.com/10.200.133.203:5181,
initiating session
2019-09-16 18:00:33,167 INFO
org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn -
Session establishment complete on server
server11.internal.com/10.200.133.203:5181, sessionid = 0x100af52c3c16c26,
negotiated timeout = 40000
2019-09-16 18:00:33,169 INFO
org.apache.flink.runtime.metrics.MetricRegistryImpl - No metrics
reporter configured, no metrics will be exposed/reported.
2019-09-16 18:00:33,170 ERROR
org.apache.flink.shaded.zookeeper.org.apache.zookeeper.client.ZooKeeperSaslClient
- An error: (java.security.PrivilegedActionException:
javax.security.sasl.SaslException: GSS initiate failed [Caused by
GSSException: No valid credentials provided (Mechanism level: Failed to
find any Kerberos tgt)]) occurred when evaluating Zookeeper Quorum Member's
received SASL token. Zookeeper Client will go to AUTH_FAILED state.
2019-09-16 18:00:33,170 ERROR
org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn - SASL
authentication with Zookeeper Quorum member failed:
javax.security.sasl.SaslException: An error:
(java.security.PrivilegedActionException:
javax.security.sasl.SaslException: GSS initiate failed [Caused by
GSSException: No valid credentials provided (Mechanism level: Failed to
find any Kerberos tgt)]) occurred when evaluating Zookeeper Quorum Member's
received SASL token. Zookeeper Client will go to AUTH_FAILED state.
2019-09-16 18:00:33,170 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils - Trying to
start actor system at server11.internal.com:0
2019-09-16 18:00:33,172 INFO
org.apache.flink.shaded.curator.org.apache.curator.framework.state.ConnectionStateManager
- State change: CONNECTED
2019-09-16 18:00:33,172 ERROR
org.apache.flink.shaded.curator.org.apache.curator.ConnectionState -
Authentication failed
2019-09-16 18:00:33,184 INFO akka.event.slf4j.Slf4jLogger
- Slf4jLogger started
2019-09-16 18:00:33,186 INFO akka.remote.Remoting
- Starting remoting
2019-09-16 18:00:33,199 INFO akka.remote.Remoting
- Remoting started; listening on addresses :[akka.tcp://
flink-metr...@server11.internal.com:36539]
2019-09-16 18:00:33,227 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils - Actor
system started at akka.tcp://flink-metr...@server11.internal.com:36539
2019-09-16 18:00:33,232 INFO
org.apache.flink.runtime.rpc.akka.AkkaRpcService - Starting
RPC endpoint for org.apache.flink.runtime.metrics.dump.MetricQueryService
at akka://flink-metrics/user/MetricQueryService .
2019-09-16 18:00:33,240 INFO
org.apache.flink.runtime.dispatcher.FileArchivedExecutionGraphStore -
Initializing FileArchivedExecutionGraphStore: Storage directory
/tmp/hadoop-mapr/nm-local-dir/usercache/run/appcache/application_1568125115996_13508/executionGraphStore-0af12230-9fff-4481-ba1c-13841eb6a679,
expiration time 3600000, maximum cache size 52428800 bytes.
2019-09-16 18:00:33,277 WARN org.apache.flink.configuration.Configuration
- Config uses deprecated configuration key 'web.port'
instead of proper key 'rest.bind-port'
2019-09-16 18:00:33,279 INFO
org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Upload
directory
/tmp/flink-web-ecf8f921-2fa6-4e2a-94c4-302862c79f7b/flink-web-upload does
not exist.
2019-09-16 18:00:33,279 INFO
org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Created
directory
/tmp/flink-web-ecf8f921-2fa6-4e2a-94c4-302862c79f7b/flink-web-upload for
file uploads.
2019-09-16 18:00:33,280 INFO
org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Starting
rest endpoint.
2019-09-16 18:00:33,445 INFO
org.apache.flink.runtime.webmonitor.WebMonitorUtils - Determined
location of main cluster component log file:
/opt/mapr/hadoop/hadoop-2.7.0/logs/userlogs/application_1568125115996_13508/container_e64_1568125115996_13508_01_000001/jobmanager.log
2019-09-16 18:00:33,445 INFO
org.apache.flink.runtime.webmonitor.WebMonitorUtils - Determined
location of main cluster component stdout file:
/opt/mapr/hadoop/hadoop-2.7.0/logs/userlogs/application_1568125115996_13508/container_e64_1568125115996_13508_01_000001/jobmanager.out
2019-09-16 18:00:33,600 INFO
org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint - Rest
endpoint listening at server11.internal.com:40850
2019-09-16 18:00:33,601 INFO
org.apache.flink.runtime.leaderelection.ZooKeeperLeaderElectionService -
Starting ZooKeeperLeaderElectionService
ZooKeeperLeaderElectionService{leaderPath='/leader/rest_server_lock'}.
2019-09-16 18:00:48,614 ERROR
org.apache.flink.shaded.curator.org.apache.curator.ConnectionState -
Connection timed out for connection string (server11.internal.com:5181) and
timeout (15000) / elapsed (15440)
org.apache.flink.shaded.curator.org.apache.curator.CuratorConnectionLossException:
KeeperErrorCode = ConnectionLoss
at
org.apache.flink.shaded.curator.org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:225)
at
org.apache.flink.shaded.curator.org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:94)
at
org.apache.flink.shaded.curator.org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:117)
at
org.apache.flink.shaded.curator.org.apache.curator.framework.imps.NamespaceImpl$1.call(NamespaceImpl.java:90)
at
org.apache.flink.shaded.curator.org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:109)
at
org.apache.flink.shaded.curator.org.apache.curator.framework.imps.NamespaceImpl.fixForNamespace(NamespaceImpl.java:83)
at
org.apache.flink.shaded.curator.org.apache.curator.framework.imps.CuratorFrameworkImpl.fixForNamespace(CuratorFrameworkImpl.java:594)
at
org.apache.flink.shaded.curator.org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:158)
at
org.apache.flink.shaded.curator.org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:32)
at
org.apache.flink.shaded.curator.org.apache.curator.framework.recipes.cache.NodeCache.reset(NodeCache.java:242)
at
org.apache.flink.shaded.curator.org.apache.curator.framework.recipes.cache.NodeCache.start(NodeCache.java:175)
at
org.apache.flink.shaded.curator.org.apache.curator.framework.recipes.cache.NodeCache.start(NodeCache.java:154)
at
org.apache.flink.runtime.leaderelection.ZooKeeperLeaderElectionService.start(ZooKeeperLeaderElectionService.java:134)
at
org.apache.flink.runtime.webmonitor.WebMonitorEndpoint.startInternal(WebMonitorEndpoint.java:666)
at
org.apache.flink.runtime.rest.RestServerEndpoint.start(RestServerEndpoint.java:241)
at
org.apache.flink.runtime.entrypoint.component.AbstractDispatcherResourceManagerComponentFactory.create(AbstractDispatcherResourceManagerComponentFactory.java:161)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:210)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1669)
at
org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:163)
at
org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:501)
at
org.apache.flink.yarn.entrypoint.YarnSessionClusterEntrypoint.main(YarnSessionClusterEntrypoint.java:93)
2019-09-16 18:00:48,619 ERROR
org.apache.flink.shaded.curator.org.apache.curator.ConnectionState -
Connection timed out for connection string (server11.internal.com:5181) and
timeout (15000) / elapsed (15447)
org.apache.flink.shaded.curator.org.apache.curator.CuratorConnectionLossException:
KeeperErrorCode = ConnectionLoss
at
org.apache.flink.shaded.curator.org.apache.curator.ConnectionState.checkTimeouts(ConnectionState.java:225)
at
org.apache.flink.shaded.curator.org.apache.curator.ConnectionState.getZooKeeper(ConnectionState.java:94)
at
org.apache.flink.shaded.curator.org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:117)
at
org.apache.flink.shaded.curator.org.apache.curator.framework.imps.NamespaceImpl$1.call(NamespaceImpl.java:90)
at
org.apache.flink.shaded.curator.org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:109)
at
org.apache.flink.shaded.curator.org.apache.curator.framework.imps.NamespaceImpl.fixForNamespace(NamespaceImpl.java:83)
at
org.apache.flink.shaded.curator.org.apache.curator.framework.imps.CuratorFrameworkImpl.fixForNamespace(CuratorFrameworkImpl.java:599)
at
org.apache.flink.shaded.curator.org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:465)
at
org.apache.flink.shaded.curator.org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:44)
at
org.apache.flink.shaded.curator.org.apache.curator.framework.recipes.leader.LeaderLatch.reset(LeaderLatch.java:505)
at
org.apache.flink.shaded.curator.org.apache.curator.framework.recipes.leader.LeaderLatch.internalStart(LeaderLatch.java:515)
at
org.apache.flink.shaded.curator.org.apache.curator.framework.recipes.leader.LeaderLatch.access$100(LeaderLatch.java:64)
at
org.apache.flink.shaded.curator.org.apache.curator.framework.recipes.leader.LeaderLatch$3.run(LeaderLatch.java:169)
at
org.apache.flink.shaded.curator.org.apache.curator.framework.recipes.AfterConnectionEstablished$1.run(AfterConnectionEstablished.java:56)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
On Fri, Aug 30, 2019 at 2:41 PM Stephan Ewen <se...@apache.org> wrote:
> Could you share the stack trace where the failure occurs, so we can see
> why the Flink ZK is used during MapR FS access?
>
> /CC Till and Tison - just FYI
>
> On Fri, Aug 30, 2019 at 9:40 AM Maxim Parkachov <lazy.gop...@gmail.com>
> wrote:
>
>> Hi Stephan,
>>
>> With previous versions, I tried around 1.7, I always had to compile MapR
>> hadoop to get it working.
>> With 1.9 I took hadoop-less Flink, which worked with MapR FS until I
>> switched on HA.
>> So it is hard to say if this is regression or not.
>>
>> The error happens when Flink tries to initialize BLOB storage on MapR FS.
>> Without HA it takes
>> zookeeper from classpath (MapR org.apache.zookeeper) and with HA it takes
>> shaded one.
>>
>> After fixing couple of issue with pom, I was able to compile Flink with
>> MapR zookeeper and now
>> when I start with HA mode it uses shaded zookeeper (which is now MapR) to
>> initialize BLOB and
>> org.apache.zookeeper (which is as well MapR) for HA recovery.
>>
>> It works, but, I was expecting it to work without compiling MapR
>> dependencies.
>>
>> Hope this helps,
>> Maxim.
>>
>> On Thu, Aug 29, 2019 at 7:00 PM Stephan Ewen <se...@apache.org> wrote:
>>
>>> Hi Maxim!
>>>
>>> The change of the MapR dependency should not have an impact on that.
>>> Do you know if the same thing worked in prior Flink versions? Is that a
>>> regression in 1.9?
>>>
>>> The exception that you report, is that from Flink's HA services trying
>>> to connect to ZK, or from the MapR FS client trying to connect to ZK?
>>>
>>> Best,
>>> Stephan
>>>
>>>
>>> On Tue, Aug 27, 2019 at 11:03 AM Maxim Parkachov <lazy.gop...@gmail.com>
>>> wrote:
>>>
>>>> Hi everyone,
>>>>
>>>> I'm testing release 1.9 on MapR secure cluster. I took flink binaries
>>>> from download page and trying to start Yarn session cluster. All MapR
>>>> specific libraries and configs are added according to documentation.
>>>>
>>>> When I start yarn-session without high availability, it uses zookeeper
>>>> from MapR distribution (org.apache.zookeeper) and correctly connects to
>>>> cluster and access to maprfs works as expected.
>>>>
>>>> But if I add zookeeper as high-avalability option, instead of MapR
>>>> zookeeper it tries to use shaded zookeeper and this one could not connect
>>>> with mapr credentials:
>>>>
>>>> 2019-08-27 10:42:45,240 ERROR
>>>> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.client.ZooKeeperSaslClient
>>>> - An error: (java.security.PrivilegedActionException:
>>>> javax.security.sasl.SaslException: GSS initiate failed [Caused by
>>>> GSSException: No valid credentials provided (Mechanism level: Failed to
>>>> find any Kerberos tgt)]) occurred when evaluating Zookeeper Quorum
>>>> Member's received SASL token. Zookeeper Client will go to AUTH_FAILED
>>>> state.
>>>> 2019-08-27 10:42:45,240 ERROR
>>>> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn - SASL
>>>> authentication with Zookeeper Quorum member failed:
>>>> javax.security.sasl.SaslException: An error:
>>>> (java.security.PrivilegedActionException:
>>>> javax.security.sasl.SaslException: GSS initiate failed [Caused by
>>>> GSSException: No valid credentials provided (Mechanism level: Failed to
>>>> find any Kerberos tgt)]) occurred when evaluating Zookeeper Quorum
>>>> Member's received SASL token. Zookeeper Client will go to AUTH_FAILED
>>>> state.
>>>> I tried to use separate zookeeper cluster for HA, but maprfs still doesn't
>>>> work.
>>>>
>>>> Is this related to removal of MapR specific settings in Release 1.9 ?
>>>> Should I still compile custom version of Flink with MapR dependencies ?
>>>> (trying to do now, but getting some errors during compilation).
>>>>
>>>> Can I somehow force flink to use MapR zookeeper even with HA mode ?
>>>>
>>>> Thanks in advance,
>>>> Maxim.
>>>>
>>>>
