Intermittent timeout failure org/apache/spark/sql/hive/thriftserver/CliSuite.scala

[Tim Preece]

I was just debugging an intermittent timeout failure in the testsuite CliSuite.scala

 

I traced it down to a timing window in the Scala library class sys.process.ProcessImpl.scala. Sometimes the input pipe to a process becomes 'None' before the process has had a chance to read any input at all. This results in a subsequent timeout failure in CliSuite.

 

There is a JIRA on the Scala side https://issues.scala-lang.org/browse/SI-8768 which should resolve the problem ( although that JIRA was raised for slightly different symptoms ) and this is fixed in scala 2.12.0

 

I also wonder if some fixes ( e.g https://issues.apache.org/jira/browse/SPARK-7973 ) may be a result of this Scala issue.

 

I am new to the Spark community. Is there a preferred way to track the fact the Spark testcase CliSuite has a dependency on the above Scala issue ?

 

Tim Preece

--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org