custom spark app name in yarn-cluster mode

[Tomer Benyamini]

Hi,

I'm trying to set a custom spark app name when running a java spark app in
yarn-cluster mode.

 SparkConf sparkConf = new SparkConf();

 sparkConf.setMaster(System.getProperty("spark.master"));

 sparkConf.setAppName("myCustomName");

 sparkConf.set("spark.logConf", "true");

 JavaSparkContext sc = new JavaSparkContext(sparkConf);


Apparently this only works when running in yarn-client mode; in
yarn-cluster mode the app name is the class name, when viewing the app in
the cluster manager UI. Any idea?


Thanks,

Tomer