I had the same issue a couple days ago. It's a bug in 1.3.0 that is fixed in 1.3.1 and up.
https://issues.apache.org/jira/browse/SPARK-6036 The ordering that the event logs are moved from in-progress to complete is coded to be after the Master tries to build the history page for the logs. The only reason it even works on occasion in 1.3.0 is because the Master part is run asynchronously and the event log status change is synchronous, so the Master part on some occasions could be executed afterwards as a race condition. On Wed, Jun 3, 2015 at 2:17 AM, ayan guha <guha.a...@gmail.com> wrote: > Have you done sc.stop() ? :) > On 3 Jun 2015 14:05, "amghost" <zhengweita...@outlook.com> wrote: > >> I run spark application in spark standalone cluster with client deploy >> mode. >> I want to check out the logs of my finished application, but I always >> get a >> page telling me "Application history not found - Application xxx is still >> in >> process". >> I am pretty sure that the application has indeed completed because I can >> see >> it in the Completed Applications list show by Spark WebUI, and I have also >> found the log file with suffix ".inprocess"in the directory set by >> "spark.eventLog.dir" in my spark-default.conf >> >> Oh, BTW, I am using spark 1.3.0 >> >> So, is there anything I missed? >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/Application-is-always-in-process-when-I-check-out-logs-of-completed-application-tp23123.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >> For additional commands, e-mail: user-h...@spark.apache.org >> >>
