I have been trying to record event logging in a standalone application submitted to spark on AWS EC2. However, the application keeps failing when trying to write to the event logs. I tried various different logging directories by setting spark.eventLog.dir but it does not work. I tried the following directories (I made sure all the directories were created and had the right permissions):
hdfs:///spark_logs /root/spark_logs hdfs://<hdfs namenode hostname>:8020/spark_logs Nothing seems to work. Can you give me some advise why it is not working? Zhen -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/spark-eventLog-enabled-not-working-on-spark-on-AWS-EC2-tp7605.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
