Fw: How Spark Choose Worker Nodes for respective HDFS block

Hi All Is there any communication between Spark MASTER node and Hadoop NameNode while distributing work to WORKER nodes, like we have in MapReduce. Please suggest TIA --  Anish Sneh "Experience is the best teacher." http://in.linkedin.com/in/anishsneh

How Spark Choose Worker Nodes for respective HDFS block

Hi All I am new to Spark, workin on 3 node test cluster. I am trying to explore Spark scope in analytics, my Spark codes interacts with HDFS mostly. I have a confusion that how Spark choose on which node it will distribute its work. Since we assume that it can be an alternative to Hadoop MapRe

Spark Performace Comparison Spark on YARN vs Spark Standalone

Hi All I need to analyse performance of Spark YARN vs Spark Standalone Please suggest if we have some pre-published comparison statistics available. TIA -- Anish Sneh http://in.linkedin.com/in/anishsneh

Re: Spark Streaming and Shark | Streaming Taking All CPUs

Thanks Mayur, it is working :) -- Anish Sneh http://in.linkedin.com/in/anishsneh

Spark Streaming and Shark | Streaming Taking All CPUs

Hi All I am new to Spark, I was trying to use Spark Streaming and Shark at the same time. I was recieiving messages from Kafka and pushing them to HDFS after minor processing. It was workin fine, but it was taking all the CPUs and at the same time on other terminal i tried to access shark but