I am looking to run a SparkContext in a web application that is outside of my Spark cluster. I understand that I can use the "client" deployment mode and use the spark-submit script that hsipts with Spark but I'm really interested in running this inside of a SpringWeb application that can be started inside of a web container like Jetty.
I saw a post [1] that briefly mentions SparkSubmit.main() could be called directly without the script and I think may be what I'm looking for- though in my case, I really want to submit a spark context to run the driver in a JVM which is already running (the web application running in the Jetty container). Can the SparkSubmit be used in this way? If it can, that will be immensely powerful in giving us the ability to wire in Spark stages as a query layer to our web applications. [1] http://apache-spark-user-list.1001560.n3.nabble.com/Using-Spark-as-web-app-backend-td8163.html
