Re: No parallelism in map transformation

2014-07-15 Thread Roch Denis
r. However that doesn't work, the example context creation needs to be change from this: "sc = SparkContext("*local*", "Simple App")" to this "sc = SparkContext("*local[*]"*, "Simple App")" -- View this message in context: http://a

No parallelism in map transformation

2014-07-15 Thread Roch Denis
ext("local", "Simple App") logData = sc.textFile(logFile) countNb = logData.map(parse_line).count() print "count:", countNb, "partition nb:", logData._jrdd.splits().size() Thanks for the help in advance! -- View this message in context