I am +1. I like the proposal and it feels like it will provide what users need for the use cases in the SPIP. Having done similar things on other projects in the past I know this is just the beginning to get something that is truly user and admin friendly, but this is a great start.
Thanks, Bobby On Wed, Sep 4, 2019 at 9:24 AM Thomas graves <tgra...@apache.org> wrote: > Hey everyone, > > I'd like to call for a vote on SPARK-27495 SPIP: Support Stage level > resource configuration and scheduling > > This is for supporting stage level resource configuration and > scheduling. The basic idea is to allow the user to specify executor > and task resource requirements for each stage to allow the user to > control the resources required at a finer grain. One good example here > is doing some ETL to preprocess your data in one stage and then feed > that data into an ML algorithm (like tensorflow) that would run as a > separate stage. The ETL could need totally different resource > requirements for the executors/tasks than the ML stage does. > > The text for the SPIP is in the jira description: > > https://issues.apache.org/jira/browse/SPARK-27495 > > I split the API and Design parts into a google doc that is linked to > from the jira. > > This vote is open until next Fri (Sept 13th). > > [ ] +1: Accept the proposal as an official SPIP > [ ] +0 > [ ] -1: I don't think this is a good idea because ... > > I'll start with my +1 > > Thanks, > Tom > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > >
