Hi Yash, One of the main advantages is that, if you turn dynamic allocation on, and executors are discarded, your application is still able to get at the shuffle data that they wrote out.
-Sandy On Thu, Jun 25, 2015 at 11:08 PM, yash datta <sau...@gmail.com> wrote: > Hi devs, > > Can someone point out if there are any distinct advantages of using > external shuffle service over yarn (runs on node manager as an auxiliary > service > > https://issues.apache.org/jira/browse/SPARK-3797) instead of the default > execution in the executor containers ? > > Please also mention if you have seen any differences having used both ways > ? > > Thanks and Best Regards > Yash > > -- > When events unfold with calm and ease > When the winds that blow are merely breeze > Learn from nature, from birds and bees > Live your life in love, and let joy not cease. >
