Hi, I was just fixing a problem with too short a timeout on one of the unit tests I added (https://issues.apache.org/jira/browse/SPARK-7919), and I was wondering if this is a common problem w/ a lot of our flaky tests.
Its really hard to know what to set the timeouts to -- you set the timeout so it runs comfortably on your dev machine, but then discover its far too short for the build machines. so up it some more, and then it turns out there are some even slower build machines, and you need to make it longer still ... Scalatest actually has a feature to handle this. You can wrap all timeouts in scaled(...), and then you can pass in a flag with how much the timeouts should get scaled by. http://doc.scalatest.org/2.0/index.html#org.scalatest.concurrent.ScaledTimeSpans eg., I was looking at DriverSuite -- its been turned off because it was too flaky, though the 60 second timeout already seems super long. ( https://issues.apache.org/jira/browse/SPARK-7415) does anybody know if timeouts are a common source of flaky tests? Is it worth trying to change existing timeouts in tests to use scaled() so keep tests passing on slow build machines? Does it make sense for all future tests to always use scaled() on timeouts? thanks, Imran
