Hi All, I'm the getting the following WARNING while running Spark jobs in standalone mode: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Please note that I have configured the native path and the other ENV variables as follows: export JAVA_HOME=/usr/lib/jvm/java-8-oracle export HADOOP_HOME=/usr/local/hadoop export HADOOP_COMMON_LIB_NATIVE_DIR=/usr/local/hadoop/lib/native export LD_LIBRARY_PATH=/usr/local/hadoop/lib/native/:$LD_LIBRARY_PATH export JAVA_LIBRARY_PATH=/usr/local/hadoop/lib/native export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/native" Although my Spark job executes successfully and writes the results to a file at the end. However, I am not getting any logs to track the progress. Could someone help me to solve this problem? Regards, _________________________________ *Md. Rezaul Karim*, BSc, MSc PhD Researcher, INSIGHT Centre for Data Analytics National University of Ireland, Galway IDA Business Park, Dangan, Galway, Ireland Web: http://www.reza-analytics.eu/index.html <http://139.59.184.114/index.html>
