Re: Spark schema evolution

2016-03-22 Thread Michael Armbrust
Which version of Spark? This sounds like a bug (that might be fixed). On Tue, Mar 22, 2016 at 6:34 AM, gtinside wrote: > Hi , > > I have a table sourced from* 2 parquet files* with few extra columns in one > of the parquet file. Simple * queries works fine but queries with predicate > on extra

Re: Spark schema evolution

2016-03-22 Thread Chris Miller
With Avro you solve this by using a default value for the new field... maybe Parquet is the same? -- Chris Miller On Tue, Mar 22, 2016 at 9:34 PM, gtinside wrote: > Hi , > > I have a table sourced from* 2 parquet files* with few extra columns in one > of the parquet file. Simple * queries work