Re: Dynamic allocator requests -1 executors

2015-06-13 Thread Andrew Or
Hi Patrick, The fix you need is SPARK-6954: https://github.com/apache/spark/pull/5704. If possible, you may cherry-pick the following commit into your Spark deployment and it should resolve the issue: https://github.com/apache/spark/commit/98ac39d2f5828fbdad8c9a4e563ad1169e3b9948 Note that this

Re: Dynamic allocator requests -1 executors

2015-06-13 Thread Patrick Woody
Hey Sandy, I'll test it out on 1.4. Do you have a bug number or PR that I could reference as well? Thanks! -Pat Sent from my iPhone > On Jun 13, 2015, at 11:38 AM, Sandy Ryza wrote: > > Hi Patrick, > > I'm noticing that you're using Spark 1.3.1. We fixed a bug in dynamic > allocation in 1

Re: Dynamic allocator requests -1 executors

2015-06-13 Thread Sandy Ryza
Hi Patrick, I'm noticing that you're using Spark 1.3.1. We fixed a bug in dynamic allocation in 1.4 that permitted requesting negative numbers of executors. Any chance you'd be able to try with the newer version and see if the problem persists? -Sandy On Fri, Jun 12, 2015 at 7:42 PM, Patrick Wo