Re: Spark Streaming - java.lang.ClassNotFoundException Scala anonymous function

2017-03-01 Thread Dominik Safaric
The jars I am submitting are the following: bin/spark-submit --class topology.SimpleProcessingTopology --master spark://10.0.0.8:7077 --jars /tmp/spark_streaming-1.0-SNAPSHOT.jar /tmp//tmp/spark_streaming-1.0-SNAPSHOT.jar /tmp/streaming.properties I’ve even tried using the spark.executor.extraC

Re: Spark Streaming - java.lang.ClassNotFoundException Scala anonymous function

2017-03-01 Thread Sean Owen
What is the --jars you are submitting? You may have conflicting copies of Spark classes that interfere. On Wed, Mar 1, 2017, 14:20 Dominik Safaric wrote: > I've been trying to submit a Spark Streaming application using > spark-submit to a cluster of mine consisting of a master and two worker > n

Spark Streaming - java.lang.ClassNotFoundException Scala anonymous function

2017-03-01 Thread Dominik Safaric
I've been trying to submit a Spark Streaming application using spark-submit to a cluster of mine consisting of a master and two worker nodes. The application has been written in Scala, and build using Maven. Importantly, the Maven build is configured to produce a fat JAR containing all dependenc