Re: Need some tutorials and examples about customized partitioner

2014-02-27 Thread Tao Xiao
Also thanks Matei 2014-02-26 15:19 GMT+08:00 Matei Zaharia : > Take a look at the "advanced Spark features" talk here too: > http://ampcamp.berkeley.edu/amp-camp-one-berkeley-2012/. > > Matei > > On Feb 25, 2014, at 6:22 PM, Tao Xiao wrote: > > Thank you Mayur, I think that will help me a lot >

Re: Need some tutorials and examples about customized partitioner

2014-02-25 Thread Matei Zaharia
Take a look at the “advanced Spark features” talk here too: http://ampcamp.berkeley.edu/amp-camp-one-berkeley-2012/. Matei On Feb 25, 2014, at 6:22 PM, Tao Xiao wrote: > Thank you Mayur, I think that will help me a lot > > > Best, > Tao > > > 2014-02-26 8:56 GMT+08:00 Mayur Rustagi : > Ty

Re: Need some tutorials and examples about customized partitioner

2014-02-25 Thread Tao Xiao
Thank you Mayur, I think that will help me a lot Best, Tao 2014-02-26 8:56 GMT+08:00 Mayur Rustagi : > Type of Shuffling is best explained by Matei in Spark Internals . > http://www.youtube.com/watch?v=49Hr5xZyTEA#t=2203 > Why dont you look at that & then if you have follow up questions ask he

Re: Need some tutorials and examples about customized partitioner

2014-02-25 Thread Mayur Rustagi
Type of Shuffling is best explained by Matei in Spark Internals . http://www.youtube.com/watch?v=49Hr5xZyTEA#t=2203 Why dont you look at that & then if you have follow up questions ask here, also would be good to watch this whole talk as it talks about Spark job flows in a lot more detail. SCALA i

Need some tutorials and examples about customized partitioner

2014-02-25 Thread Tao Xiao
I am a newbie to Spark and I need to know how RDD partitioning can be controlled in the process of shuffling. I have googled for examples but haven't found much concrete examples, in contrast with the fact that there are many good tutorials about Hadoop's shuffling and partitioner. Can anybody sho