Re: best fit - Dataframe and spark sql use cases

2016-05-10 Thread Mathieu Longtin
Spark SQL is translated to DataFrame operations by the SQL engine. Use whichever is more comfortable for the task. Unless I'm doing something very straight forward, I go with SQL, since any improvement to the SQL engine will improve the resulting DataFrame operations. Hard-coded DataFrame operation

best fit - Dataframe and spark sql use cases

2016-05-09 Thread Divya Gehlot
Hi, I would like to know the uses cases where data frames is best fit and use cases where Spark SQL is best fit based on the one's experience . Thanks, Divya