Re: Necessity for rdd replication.

2014-12-04 Thread Sameer Farooqui
st the RDD, but again.. generally speaking, you shouldn't need to replicate it. On Wed, Dec 3, 2014 at 11:54 PM, rapelly kartheek wrote: > Hi, > > I was just thinking about necessity for rdd replication. One category > could be something like large number of threads requiring same

Necessity for rdd replication.

2014-12-03 Thread rapelly kartheek
Hi, I was just thinking about necessity for rdd replication. One category could be something like large number of threads requiring same rdd. Even though, a single rdd can be shared by multiple threads belonging to "same application" , I believe we can extract better parallelism if

Rdd replication

2014-11-09 Thread rapelly kartheek
Hi, I am trying to understand rdd replication code. In the process, I frequently execute one spark application whenever I make a change to the code to see effect. My problem is, after a set of repeated executions of the same application, I find that my cluster behaves unusually. Ideally, when