Re: Unable to save dataframe with UDT created with sqlContext.createDataFrame

2015-04-03 Thread Jaonary Rabarisoa
Good! Thank you. On Thu, Apr 2, 2015 at 9:05 AM, Xiangrui Meng wrote: > I reproduced the bug on master and submitted a patch for it: > https://github.com/apache/spark/pull/5329. It may get into Spark > 1.3.1. Thanks for reporting the bug! -Xiangrui > > On Wed, Apr 1, 2015 at 12:57 AM, Jaonary Ra

Re: Unable to save dataframe with UDT created with sqlContext.createDataFrame

2015-04-02 Thread Xiangrui Meng
I reproduced the bug on master and submitted a patch for it: https://github.com/apache/spark/pull/5329. It may get into Spark 1.3.1. Thanks for reporting the bug! -Xiangrui On Wed, Apr 1, 2015 at 12:57 AM, Jaonary Rabarisoa wrote: > Hmm, I got the same error with the master. Here is another test

Re: Unable to save dataframe with UDT created with sqlContext.createDataFrame

2015-04-01 Thread Jaonary Rabarisoa
Hmm, I got the same error with the master. Here is another test example that fails. Here, I explicitly create a Row RDD which corresponds to the use case I am in : *object TestDataFrame { def main(args: Array[String]): Unit = { val conf = new SparkConf().setAppName("TestDataFrame").setMas

Re: Unable to save dataframe with UDT created with sqlContext.createDataFrame

2015-03-31 Thread Xiangrui Meng
I cannot reproduce this error on master, but I'm not aware of any recent bug fixes that are related. Could you build and try the current master? -Xiangrui On Tue, Mar 31, 2015 at 4:10 AM, Jaonary Rabarisoa wrote: > Hi all, > > DataFrame with an user defined type (here mllib.Vector) created with >