Re: API to add/remove containers inside an application

2014-09-04 Thread Praveen Seluka
Mailed our list - will send it to Spark Dev On Fri, Sep 5, 2014 at 11:28 AM, Rajat Gupta wrote: > +1 on this. First step to more automated autoscaling of spark application > master... > > > On Fri, Sep 5, 2014 at 12:56 AM, Praveen Seluka > wrote: > >> +user >> >> >> >> On Thu, Sep 4, 2014 at 1

Re: API to add/remove containers inside an application

2014-09-04 Thread Praveen Seluka
+user On Thu, Sep 4, 2014 at 10:53 PM, Praveen Seluka wrote: > Spark on Yarn has static allocation of resources. > https://issues.apache.org/jira/browse/SPARK-3174 - This JIRA by Sandy is > about adding and removing executors dynamically based on load. Even before > doing this, can we expose an