gt;
>
> From: Evo Eftimov [mailto:evo.efti...@isecc.com]
> Sent: Wednesday, July 8, 2015 1:26 PM
> To: 'Aniruddh Sharma'; 'user@spark.apache.org'
> Subject: RE: Out of Memory Errors on less number of cores in proportion to
> Partitions in Data
>
>
>
&g
: 'Aniruddh Sharma'; 'user@spark.apache.org'
Subject: RE: Out of Memory Errors on less number of cores in proportion to
Partitions in Data
Are you sure you have actually increased the RAM (how exactly did you do that
and does it show in Spark UI)
Also use the SPARK UI an
them will be in a suspended mode
waiting for free core (Thread contexts also occupy additional RAM )
From: Aniruddh Sharma [mailto:asharma...@gmail.com]
Sent: Wednesday, July 8, 2015 12:52 PM
To: Evo Eftimov
Subject: Re: Out of Memory Errors on less number of cores in proportion to
Partitions
This is most likely due to the internal implementation of ALS in MLib. Probably
for each parallel unit of execution (partition in Spark terms) the
implementation allocates and uses a RAM buffer where it keeps interim results
during the ALS iterations
If we assume that the size of that intern
