Re: Data Frame types

2015-03-06 Thread Michael Armbrust
No, the UDT API is not a public API as we have not stabilized the implementation. For this reason its only accessible to projects inside of Spark. On Fri, Mar 6, 2015 at 8:25 AM, Jaonary Rabarisoa wrote: > Hi Cesar, > > Yes, you can define an UDT with the new DataFrame, the same way that > Sche

Re: Data Frame types

2015-03-06 Thread Jaonary Rabarisoa
Hi Cesar, Yes, you can define an UDT with the new DataFrame, the same way that SchemaRDD did. Jaonary On Fri, Mar 6, 2015 at 4:22 PM, Cesar Flores wrote: > > The SchemaRDD supports the storage of user defined classes. However, in > order to do that, the user class needs to extends the UserDefi

Data Frame types

2015-03-06 Thread Cesar Flores
The SchemaRDD supports the storage of user defined classes. However, in order to do that, the user class needs to extends the UserDefinedType interface (see for example VectorUDT in org.apache.spark.mllib.linalg). My question is: Do the new Data Frame Structure (to be released in spark 1.3) will b