Hi guys: I'm trying to running 24 APPs simultaneously in one spark
cluster.
However, everytime my cluster can only running 17 APPs in the same
time. Other APPs disappeared, no logs, no failures. Any ideas will be
appreciated.
Here is my code:object GeneCompare5{
def main(args: Array[String]) {
val conf = new
SparkConf().setAppName("GeneCompareSubmitter2").setMaster("spark://sparkmaster:7077").set("spark.executor.memory",
"1g")
val sc = new SparkContext(conf) val resultRDD =
sc.parallelize(Array("1","2","3","4","5","6","7","8","9","10","11","12","13","14","15","16","17","18","19","20","21","22","X","Y"))
//theoretically func1 should be called 24 times.
resultRDD.map{i =>
GeneCompare5.func1(i)
}.collect() sc.stop()
}
def func1(i: String){ //to submit another spark APP "CompareGenePiece i"
to the same spark cluster
......
}}
--------------------------------
Thanks&Best regards!
San.Luo
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
