Hi, Did you run the word count example in Spark local mode or other mode, in local mode you have to set Local[n], where n >=2. For other mode, make sure available cores larger than 1. Because the receiver inside Spark Streaming wraps as a long-running task, which will at least occupy one core.
Besides using lsof -p <pid> or netstat to make sure Spark executor backend is connected to the nc process. Also grep the executor's log to see if there's log like "Connecting to <host> <port>" and "Connected to <host> <port>" which shows that receiver is correctly connected to nc process. Thanks Jerry 2015-03-27 8:45 GMT+08:00 Mohit Anchlia <mohitanch...@gmail.com>: > What's the best way to troubleshoot inside spark to see why Spark is not > connecting to nc on port 9999? I don't see any errors either. > > On Thu, Mar 26, 2015 at 2:38 PM, Mohit Anchlia <mohitanch...@gmail.com> > wrote: > >> I am trying to run the word count example but for some reason it's not >> working as expected. I start "nc" server on port 9999 and then submit the >> spark job to the cluster. Spark job gets successfully submitting but I >> never see any connection from spark getting established. I also tried to >> type words on the console where "nc" is listening and waiting on the >> prompt, however I don't see any output. I also don't see any errors. >> >> Here is the conf: >> >> SparkConf conf = *new* SparkConf().setMaster(masterUrl).setAppName( >> "NetworkWordCount"); >> >> JavaStreamingContext *jssc* = *new* JavaStreamingContext(conf, Durations. >> *seconds*(1)); >> >> JavaReceiverInputDStream<String> lines = jssc.socketTextStream( >> "localhost", 9999); >> > >
