Re: pull request template

2016-03-11 Thread Marcelo Vanzin
Hey all, Just wanted to ask: how do people like this new template? While I think it's great to have instructions for people to write proper commit messages, I think the current template has a few downsides. - I tend to write verbose commit messages already when I'm preparing a PR. Now when I ope

Re: Contributing to managed memory, Tungsten..

2016-03-11 Thread Amit Chavan
Hi Jan, Welcome to the group. I have used mapdb on some personal project and really enjoyed working with it. I am also willing to contribute to the spark community and would like to pair with you on this. I can help out with any tasks that you can give me. Thanks, Amit On Fri, Mar 11, 2016 at 2:

Re: Running ALS on comparitively large RDD

2016-03-11 Thread Deepak Gopalakrishnan
Executor memory : 45g X 4 executors , 1 Driver with 45g memory Data Source is from S3 and I've logs that tells me the Rating objects are loaded fine. On Fri, Mar 11, 2016 at 2:13 PM, Nick Pentreath wrote: > Hmmm, something else is going on there. What data source are you reading > from? How much

Re: Running ALS on comparitively large RDD

2016-03-11 Thread Nick Pentreath
Hmmm, something else is going on there. What data source are you reading from? How much driver and executor memory have you provided to Spark? On Fri, 11 Mar 2016 at 09:21 Deepak Gopalakrishnan wrote: > 1. I'm using about 1 million users against few thousand products. I > basically have around