Issue with supporting this imo is the fact that scala-test uses the same vm for all the tests (surefire plugin supports fork, but scala-test ignores it iirc). So different tests would initialize different spark context, and can potentially step on each others toes.
Regards, Mridul On Fri, Aug 8, 2014 at 9:31 PM, Nicholas Chammas <nicholas.cham...@gmail.com> wrote: > Howdy, > > Do we think it's both feasible and worthwhile to invest in getting our unit > tests to finish in under 5 minutes (or something similarly brief) when run > by Jenkins? > > Unit tests currently seem to take anywhere from 30 min to 2 hours. As > people add more tests, I imagine this time will only grow. I think it would > be better for both contributors and reviewers if they didn't have to wait > so long for test results; PR reviews would be shorter, if nothing else. > > I don't know how how this is normally done, but maybe it wouldn't be too > much work to get a test cycle to feel lighter. > > Most unit tests are independent and can be run concurrently, right? Would > it make sense to build a given patch on many servers at once and send > disjoint sets of unit tests to each? > > I'd be interested in working on something like that if possible (and > sensible). > > Nick --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
