Re: Concurrency issue in SQLExecution.withNewExecutionId

2015-09-11 Thread Olivier Toupin
@Andrew_Or-2 I am using Scala futures. -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Concurrency-issue-in-SQLExecution-withNewExecutionId-tp14035p14068.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com. -

Concurrency issue in SQLExecution.withNewExecutionId

2015-09-10 Thread Olivier Toupin
Look at this code: https://github.com/apache/spark/blob/branch-1.5/sql/core/src/main/scala/org/apache/spark/sql/execution/SQLExecution.scala#L42 and https://github.com/apache/spark/blob/branch-1.5/sql/core/src/main/scala/org/apache/spark/sql/execution/SQLExecution.scala#L87 This exception is t

Spark UI history job duration is wrong

2015-01-05 Thread Olivier Toupin
Hello, I'm using Spark 1.2.0 and when running an application, if I go into the UI and then in the job tab ("/jobs/") the jobs duration are relevant and the posted durations looks ok. However when I open the history ("history/app-/jobs/") for that job, the duration are wrong showing milliseconds i