Re: Dataframes Question

2015-04-19 Thread Ted Yu
That's right. On Sun, Apr 19, 2015 at 8:59 AM, Arun Patel wrote: > Thanks Ted. > > So, whatever the operations I am performing now are DataFrames and not > SchemaRDD? Is that right? > > Regards, > Venkat > > On Sun, Apr 19, 2015 at 9:13 AM, Ted Yu wrote: > >> bq. SchemaRDD is not existing in 1

Re: Dataframes Question

2015-04-19 Thread Arun Patel
Thanks Ted. So, whatever the operations I am performing now are DataFrames and not SchemaRDD? Is that right? Regards, Venkat On Sun, Apr 19, 2015 at 9:13 AM, Ted Yu wrote: > bq. SchemaRDD is not existing in 1.3? > > That's right. > > See this thread for more background: > > http://search-hado

Re: Dataframes Question

2015-04-19 Thread Ted Yu
bq. SchemaRDD is not existing in 1.3? That's right. See this thread for more background: http://search-hadoop.com/m/JW1q5zQ1Xw/spark+DataFrame+schemardd&subj=renaming+SchemaRDD+gt+DataFrame On Sat, Apr 18, 2015 at 5:43 PM, Abhishek R. Singh < abhis...@tetrationanalytics.com> wrote: > I am no

Re: Dataframes Question

2015-04-18 Thread Abhishek R. Singh
I am no expert myself, but from what I understand DataFrame is grandfathering SchemaRDD. This was done for API stability as spark sql matured out of alpha as part of 1.3.0 release. It is forward looking and brings (dataframe like) syntax that was not available with the older schema RDD. On Ap

Dataframes Question

2015-04-18 Thread Arun Patel
Experts, I have few basic questions on DataFrames vs Spark SQL. My confusion is more with DataFrames. 1) What is the difference between Spark SQL and DataFrames? Are they same? 2) Documentation says SchemaRDD is renamed as DataFrame. This means SchemaRDD is not existing in 1.3? 3) As per doc