I tried anather test code: def main(args: Array[String]) { if (args.length
!= 1) { Util.printLog("ERROR", "Args error - arg1: BASE_DIR")
exit(101) } val currentFile = args(0).toString val DB = "test_spark"
val tableName = "src"
val sparkConf = new SparkConf().setAppName(s"HiveFromSpark") val sc =
new SparkContext(sparkConf) val hiveContext = new HiveContext(sc)
// Before exit Util.printLog("INFO", "Exit") exit(100)}
There were two `exit` in this code. If the args was wrong, the spark-submit
will get the return code 101, but, if the args is correct, spark-submit cannot
get the second return code 100. What's the difference between these two
`exit`? I was so confused.
From: [email protected]
To: [email protected]
Subject: RE: Issue on [SPARK-3877][YARN]: Return code of the spark-submit in
yarn-cluster mode
Date: Fri, 5 Dec 2014 17:11:39 +0800
I tried in spark client mode, spark-submit can get the correct return code from
spark job. But in yarn-cluster mode, It failed.
From: [email protected]
To: [email protected]
Subject: Issue on [SPARK-3877][YARN]: Return code of the spark-submit in
yarn-cluster mode
Date: Fri, 5 Dec 2014 16:55:37 +0800
Hi, all:
According to https://github.com/apache/spark/pull/2732, When a spark job fails
or exits nonzero in yarn-cluster mode, the spark-submit will get the
corresponding return code of the spark job. But I tried in spark-1.1.1 yarn
cluster, spark-submit return zero anyway.
Here is my spark code:
try { val dropTable = s"drop table $DB.$tableName"
hiveContext.hql(dropTable) val createTbl = do some thing...
hiveContext.hql(createTbl) } catch { case ex: Exception => {
Util.printLog("ERROR", s"create db error.") exit(-1) } }
Maybe I did something wrong. Is there any hint? Thanks.
