Standalone Spark program

[Akshat Aranya]

Hi,

I am building a Spark-based service which requires initialization of a
SparkContext in a main():

def main(args: Array[String]) {
    val conf = new SparkConf(false)
      .setMaster("spark://foo.example.com:7077")
      .setAppName("foobar")

    val sc = new SparkContext(conf)
    val rdd = sc.parallelize(0 until 255)
    val res =  rdd.mapPartitions(it => it).take(1)
    println(s"res=$res")
    sc.stop()
}

This code works fine via REPL, but not as a standalone program; it causes a
ClassNotFoundException.  This has me confused about how code is shipped out
to executors.  When using via REPL, does the mapPartitions closure, it=>it,
get sent out when the REPL statement is executed?  When this code is run as
a standalone program (not via spark-submit), is the compiled code expected
to be present at the the executor?

Thanks,
Akshat