RE: [SparkR] creating dataframe from json file

2015-07-15 Thread Sun, Rui
: Re: [SparkR] creating dataframe from json file Thanks. t <- getField(df$hashtags, "text") does return a Column. But when I tried to call t <- getField(df$hashtags, "text"), it would give an error: Error: All select() inputs must resolve to integer column positions. The follo

Re: [SparkR] creating dataframe from json file

2015-07-15 Thread jianshu Weng
Thanks. t <- getField(df$hashtags, "text") does return a Column. But when I tried to call t <- getField(df$hashtags, "text"), it would give an error: Error: All select() inputs must resolve to integer column positions. The following do not: * getField(df$hashtags, "text") In fact, the "text" fi

RE: [SparkR] creating dataframe from json file

2015-07-15 Thread Sun, Rui
suppose df <- jsonFile(sqlContext, "") You can extract hashtags.text as a Column object using the following command: t <- getField(df$hashtags, "text") and then you can perform operations on the column. You can extract hashtags.text as a DataFrame using the following command: t <- select(d