Splitting dataframe using Spark 1.4 for nested json input

2015-07-04 Thread Mike Tracy
Hello, I am having issues with splitting contents of a dataframe column using Spark 1.4. The dataframe was created by reading a nested complex json file. I used df.explode but keep getting error message. scala> val df = sqlContext.read.json("/Users/xx/target/statsfile.json") scala> df.show() +-

Error with splitting contents of a dataframe column using Spark 1.4 for nested complex json file

2015-07-01 Thread Mike Tracy
Hello, I am having issues with splitting contents of a dataframe column using Spark 1.4. The dataframe was created by reading a nested complex json file. I used df.explode but keep getting error message. The json file format looks like [ { "neid":{ }, "mi":{