Nice explanation... Thanks!
On Thu, Jun 5, 2014 at 5:50 PM, Sandy Ryza wrote:
> Hi Xu,
>
> As crazy as it might sound, this all makes sense.
>
> There are a few different quantities at play here:
> * the heap size of the executor (controlled by --executor-memory)
> * the amount of memory spark
Hi Xu,
As crazy as it might sound, this all makes sense.
There are a few different quantities at play here:
* the heap size of the executor (controlled by --executor-memory)
* the amount of memory spark requests from yarn (the heap size plus
384 mb to account for fixed memory costs outside if the
