will spark aggregate and treeaggregate case a shuflle action?

2016-11-10 Thread codlife
Hi Users: I'm doubt about whether spark aggregate and treeaggregate case a shuffle action? if not how spark do the combine option in spark internal.Any tips is appreciated, thank you! -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/will-spark-aggregate-and

Why the json file used by sparkSession.read.json must be a valid json object per line

2016-10-15 Thread codlife
Hi: I'm doubt about the design of spark.read.json, why the json file is not a standard json file, who can tell me the internal reason. Any advice is appreciated. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Why-the-json-file-used-by-sparkSession-read-