Document listing spark sql aggregate functions

2016-10-03 Thread Ashish Tadose
Hi Team, Is there a documentation page which lists all the aggregation functions supported in Spark sql query language. Same as listed DataFrame aggregate functions as below https://spark.apache.org/docs/1.6.2/api/scala/index.html#org.apache.spark.sql.functions$ I was looking for spark sql query

Returning DataFrame as Scala method return type

2016-09-08 Thread Ashish Tadose
Hi Team, I have Spark job with large number of dataframe operations. This job reads various lookup data from external table as MySql and also run lot of dataframe operations on large data on hdfs in parquet. Job works fine in cluster however jobdriver code looks clumsy because of large number of