Re: schema for schema

Thanks! On Thu, Sep 18, 2014 at 1:14 PM, Davies Liu wrote: > Thanks for reporting this, it will be fixed by > https://github.com/apache/spark/pull/2448 > > On Thu, Sep 18, 2014 at 12:32 PM, Michael Armbrust > wrote: > > This looks like a bug, we are investigating. > > > > On Thu, Sep 18, 2014 a

Re: schema for schema

Thanks for reporting this, it will be fixed by https://github.com/apache/spark/pull/2448 On Thu, Sep 18, 2014 at 12:32 PM, Michael Armbrust wrote: > This looks like a bug, we are investigating. > > On Thu, Sep 18, 2014 at 8:49 AM, Eric Friedman > wrote: >> >> I have a SchemaRDD which I've gotten

Re: schema for schema

This looks like a bug, we are investigating. On Thu, Sep 18, 2014 at 8:49 AM, Eric Friedman wrote: > I have a SchemaRDD which I've gotten from a parquetFile. > > Did some transforms on it and now want to save it back out as parquet > again. > > Getting a SchemaRDD proves challenging because some

schema for schema

I have a SchemaRDD which I've gotten from a parquetFile. Did some transforms on it and now want to save it back out as parquet again. Getting a SchemaRDD proves challenging because some of my fields can be null/None and SQLContext.inferSchema abjects those. So, I decided to use the schema on the