Hi I have Java String array which contains 45 string which is basically
Schema
String[] fieldNames = {"col1","col2",...};
Currently I am storing above array of String in a driver static field. My
job is running slow so trying to refactor code
I am using String array in creating DataFrame
DataFrame df = sourceFrame.toDF(fieldNames);
I want to do the above using broadcast variable to that we dont ship huge
string array to executor I believe we can do something like the following to
create broadcast
String[] brArray = sc.broadcast(fieldNames);
DataFrame df = sourceFrame.toDF(???);//how do I use above broadcast can I
use it as is by passing brArray
Please guide thanks much.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/How-to-create-broadcast-variable-from-Java-String-array-tp24666.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
