Could anybody help? I wonder if I asked a stupid question or I didn't make the question clear?
2014-07-31 21:47 GMT+08:00 Fengyun RAO <raofeng...@gmail.com>: > As shown here: > 2 - Why Is My Spark Job so Slow and Only Using a Single Thread? > <http://engineering.sharethrough.com/blog/2013/09/13/top-3-troubleshooting-tips-to-keep-you-sparking/> > > > 123456789101112131415 > > object JSONParser { def parse(raw: String): String = ...}object > MyFirstSparkJob { def main(args: Array[String]) { val sc = new > SparkContext() val lines = sc.textFileStream("beacons.txt") > lines.map(line => JSONParser.parse(line)) lines.foreach(line => > println(line)) ssc.start() }} > > It says " parser instance is now a singleton created in the scope of our > driver program" which I thought was in the scope of executor. Am I wrong, > or why? > > What if the parser is not serializable, and I want to share it among > tasks in the same worker node? > > > > >
