Hi,
I have written custom InputFormat and RecordReader for Spark, I need to use
user variables from spark client program.
I added them in SparkConf
val sparkConf = new
SparkConf().setAppName(args(0)).set("developer","MyName")
*and in InputFormat class*
protected boolean isSplitable(JobContext context, Path filename) {
System.out.println("#########################################
Developer "
+ context.getConfiguration().get("developer") );
return false;
}
but its return me *null* , is there any way I can pass user variables to my
custom code?
Thanks !!
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/How-to-send-user-variables-from-Spark-client-to-custom-InputFormat-or-RecordReader-tp21755.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
