Re: SparkSQL - Partitioned Parquet

2014-07-07 Thread Michael Armbrust
The only partitioning that is currently supported is through Hive partitioned tables. Supporting this for parquet as well is on our radar, but probably won't happen for 1.1. On Sun, Jul 6, 2014 at 10:00 PM, Raffael Marty wrote: > Does SparkSQL support partitioned parquet tables? How do I save

SparkSQL - Partitioned Parquet

2014-07-06 Thread Raffael Marty
Does SparkSQL support partitioned parquet tables? How do I save to a partitioned parquet file from within Python? table.saveAsParquetFile("table.parquet”) This call doesn’t seem to support a partition argument. Or does my schemaRDD have to be setup a specific way?