Thanks for the responses, I am running this on Amazon EMR which runs the Yarn cluster manager.
On Sat, Feb 25, 2017 at 4:45 PM, liangyhg...@gmail.com < liangyhg...@gmail.com> wrote: > Hi, > I think you are using the local model of Spark. There > are mainly four models, which are local, standalone, yarn > and Mesos. Also, "blocks" is relative to hdfs, "partitions" > is relative to spark. > > liangyihuai > > ---Original--- > *From:* "Jacek Laskowski "<ja...@japila.pl> > *Date:* 2017/2/25 02:45:20 > *To:* "prithish"<prith...@gmail.com>; > *Cc:* "user"<user@spark.apache.org>; > *Subject:* Re: RDD blocks on Spark Driver > > Hi, > > Guess you're use local mode which has only one executor called driver. Is > my guessing correct? > > Jacek > > On 23 Feb 2017 2:03 a.m., <prith...@gmail.com> wrote: > >> Hello, >> >> Had a question. When I look at the executors tab in Spark UI, I notice >> that some RDD blocks are assigned to the driver as well. Can someone please >> tell me why? >> >> Thanks for the help. >> >
