Re: Using FunSuite to test Spark throws NullPointerException

2014-10-06 Thread Mario Pastorelli
The problem was mine: I was using FunSuite in the wrong way. The test method of FunSuite registers a "test" to be triggered when the tests are running. My localTest is instead creating and stopping the SparkContext during the test registration and as result my SparkContext is stopped when the t

Re: Using FunSuite to test Spark throws NullPointerException

2014-10-06 Thread Matei Zaharia
Weird, it seems like this is trying to use the SparkContext before it's initialized, or something like that. Have you tried unrolling this into a single method? I wonder if you just have multiple versions of these libraries on your classpath or something. Matei On Oct 4, 2014, at 1:40 PM, Mari

Using FunSuite to test Spark throws NullPointerException

2014-10-04 Thread Mario Pastorelli
I would like to use FunSuite to test my Spark jobs by extending FunSuite with a new function, called |localTest|, that runs a test with a default SparkContext: |class SparkFunSuite extends FunSuite { def localTest(name: