Which release of Spark are you using ? Can you pastebin stack trace of executor(s) so that we can have some more clue ?
Thanks On Mon, Jan 11, 2016 at 1:10 PM, Andrew Wooster <[email protected]> wrote: > I have a very simple program that runs fine on my Linux server that runs > Spark master and worker in standalone mode. > > public class SimpleSpark { > public int sum () { > SparkConf conf = new SparkConf() > .setAppName("Magellan") > .setMaster("spark:// > ec2-nnn-nnn-nnn-nnn.compute-1.amazonaws.com:11407") > .setJars(new String[] > {"target/magellan-spark-1.0-SNAPSHOT.jar"}); > JavaSparkContext sc = new JavaSparkContext(conf); > > List<Integer> data = Arrays.asList(1, 2, 3, 4, 5); > JavaRDD<Integer> distData = sc.parallelize(data); > int total = distData.reduce(new SumFunc()); > return total; > } > > public static class SumFunc implements Function2<Integer, Integer, > Integer> { > public Integer call(Integer a, Integer b) { > return a + b; > } > }; > > However, when I run the same driver from a Windows machine it outputs the > following message and never completes: > 16/01/11 20:51:11 WARN TaskSchedulerImpl: Initial job has not accepted > any resources; check your cluster UI to ensure that workers are registered > and have sufficient resources > > I have checked the cluster UI and the job is marked as RUNNING (so it does > not appears to be waiting on a worker). I do not see anything out of the > ordinary in the master and worker logs. > > How do I debug a problem like this? > -Andrew >
