Sorry about that. Will keep the list in my replies. So, just to clarify I am not using an older version of sparks shuffle service. This is a brand new cluster with just Spark 2.2.0 installed alongside hadoop 2.7.3. Could there be anything else I am missing, or I can try differently ?
Thanks ! On Wed, Jul 19, 2017 at 12:03 PM, Marcelo Vanzin <van...@cloudera.com> wrote: > Please include the list on your replies, so others can benefit from > the discussion too. > > On Wed, Jul 19, 2017 at 11:43 AM, Udit Mehrotra > <udit.mehrotr...@gmail.com> wrote: > > Hi Marcelo, > > > > Thanks a lot for confirming that. Can you explain what you mean by > upgrading > > the version of shuffle service ? Wont it automatically use the > corresponding > > class from spark 2.2.0 to start the external shuffle service ? > > That depends on how you deploy your shuffle service. Normally YARN > will have no idea that your application is using a new Spark - it will > still have the old version of the service jar in its classpath. > > > -- > Marcelo >
