Prithish, It would be helpful for you to share the spark-submit command you are running.
~ Jonathan On Sun, Feb 26, 2017 at 8:29 AM Prithish <prith...@gmail.com> wrote: > Thanks for the responses, I am running this on Amazon EMR which runs the > Yarn cluster manager. > > On Sat, Feb 25, 2017 at 4:45 PM, liangyhg...@gmail.com < > liangyhg...@gmail.com> wrote: > > Hi, > > I think you are using the local model of Spark. There are mainly four > models, which are local, standalone, yarn and Mesos. Also, "blocks" is > relative to hdfs, "partitions" is relative to spark. > > liangyihuai > > ---Original--- > *From:* "Jacek Laskowski "<ja...@japila.pl> > *Date:* 2017/2/25 02:45:20 > *To:* "prithish"<prith...@gmail.com>; > *Cc:* "user"<user@spark.apache.org>; > *Subject:* Re: RDD blocks on Spark Driver > > Hi, > > Guess you're use local mode which has only one executor called driver. Is > my guessing correct? > > Jacek > > On 23 Feb 2017 2:03 a.m., <prith...@gmail.com> wrote: > > Hello, > > Had a question. When I look at the executors tab in Spark UI, I notice > that some RDD blocks are assigned to the driver as well. Can someone please > tell me why? > > Thanks for the help. > > >
