Which Spark / hadoop release are you running ? Thanks
On Thu, Feb 25, 2016 at 4:28 AM, Jan Štěrba <i...@jansterba.com> wrote: > Hello, > > I have quite a weird behaviour that I can't quite wrap my head around. > I am running Spark on a Hadoop YARN cluster. I have Spark configured > in such a way that it utilizes all free vcores in the cluster (setting > max vcores per executor and number of executors to use all vcores in > cluster). > > Once oozie launcher job and spark AM claim their job, there should be > free resources for 8 spark executor nodes but in spark UI I only see 7 > active nodes (there should be two spark nodes per one hadoop host). I > have checked what containers are running on each hadoop nodes and > discovered that one node is in deed running more spark containers than > is reported in spark UI. > > This behaviour is very strange to me and I have no idea what to make > of it or how to debug it. > > Any thoughts? > > Thanks. > > -- > Jan Sterba > https://twitter.com/honzasterba | http://flickr.com/honzasterba | > http://500px.com/honzasterba > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
