Fw: How Spark Choose Worker Nodes for respective HDFS block

Hi All Is there any communication between Spark MASTER node and Hadoop NameNode while distributing work to WORKER nodes, like we have in MapReduce. Please suggest TIA --  Anish Sneh "Experience is the best teacher." http://in.linkedin.com/in/anishsneh

Re: Big-Endian (IBM Power7) Spark Serialization issue

Hi All Yesterday,we rewrite the class Serializable's UID as static .Then find that the Executor could resolve the class .But it also thow the Exception imcompitable UID.We also debug the program in remote mode found jvm use bigendian in master and slave. Once we try ./run-example HDFSWordCount on

openstack swift integration with Spark

If you are interested in openstack/swift integration with Spark, please drop me a line. We are looking into improving the integration. Thanks.