Re: Spark streaming app that processes Kafka DStreams produces no output and no error

2017-01-19 Thread shyla deshpande
There was a issue connecting to Kafka, once that was fixed the spark app works. Hope this helps someone. Thanks On Mon, Jan 16, 2017 at 7:58 AM, shyla deshpande wrote: > Hello, > I checked the log file on the worker node and don't see any error there. > This is the first time I am asked to run

Re: Spark streaming app that processes Kafka DStreams produces no output and no error

2017-01-16 Thread shyla deshpande
Hello, I checked the log file on the worker node and don't see any error there. This is the first time I am asked to run on such a small cluster. I feel its the resources issue, but it will be great help is somebody can confirm this or share your experience. Thanks On Sat, Jan 14, 2017 at 4:01 PM

Re: Spark streaming app that processes Kafka DStreams produces no output and no error

2017-01-14 Thread shyla deshpande
Hello, I want to add that, I don't even see the streaming tab in the application UI on port 4040 when I run it on the cluster. The cluster on EC2 has 1 master node and 1 worker node. The cores used on the worker node is 2 of 2 and memory used is 6GB of 6.3GB. Can I run a spark streaming job with