Re: Spark inserting into parquet files with different schema

2015-08-10 Thread Simeon Simeonov
gt; Cc: user mailto:user@spark.apache.org>> Subject: Re: Spark inserting into parquet files with different schema What is the error you are getting? It would also be awesome if you could try with Spark 1.5 when the first preview comes out (hopefully early next week). On Mon, Aug 10, 2015 a

Re: Spark inserting into parquet files with different schema

2015-08-10 Thread Michael Armbrust
Date: Monday, August 10, 2015 at 2:36 PM > To: Simeon Simeonov > Cc: user > Subject: Re: Spark inserting into parquet files with different schema > > Older versions of Spark (i.e. when it was still called SchemaRDD instead > of DataFrame) did not support merging different parqu

Re: Spark inserting into parquet files with different schema

2015-08-10 Thread Simeon Simeonov
, Sim From: Michael Armbrust mailto:mich...@databricks.com>> Date: Monday, August 10, 2015 at 2:36 PM To: Simeon Simeonov mailto:s...@swoop.com>> Cc: user mailto:user@spark.apache.org>> Subject: Re: Spark inserting into parquet files with different schema Older versions of Spark

Re: Spark inserting into parquet files with different schema

2015-08-10 Thread Michael Armbrust
Older versions of Spark (i.e. when it was still called SchemaRDD instead of DataFrame) did not support merging different parquet schema. However, Spark 1.4+ should. On Sat, Aug 8, 2015 at 8:58 PM, sim wrote: > Adam, did you find a solution for this? > > > > -- > View this message in context: >

Re: Spark inserting into parquet files with different schema

2015-08-08 Thread sim
Adam, did you find a solution for this? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-inserting-into-parquet-files-with-different-schema-tp20706p24181.html Sent from the Apache Spark User List mailing list archive at Nabble.com.