RE: [SPARK-26160] Make assertNotBucketed call in DataFrameWriter::save optional

2018-12-13 Thread JOAQUIN GUANTER GONZALBEZ
ica.com>> CC: Spark dev list mailto:dev@spark.apache.org>> Asunto: Re: [SPARK-26160] Make assertNotBucketed call in DataFrameWriter::save optional The bucket feature is designed to only work with data sources with table support, and currently the table support is not public yet, w

Re: [SPARK-26160] Make assertNotBucketed call in DataFrameWriter::save optional

2018-12-10 Thread Ryan Blue
ted to the > DataSourceV2 community sync? > > > > Thanks! > > Ximo. > > > > *De:* Wenchen Fan > *Enviado el:* miƩrcoles, 5 de diciembre de 2018 15:51 > *Para:* JOAQUIN GUANTER GONZALBEZ > > *CC:* Spark dev list > *Asunto:* Re: [SPARK-26160] Make asse

RE: [SPARK-26160] Make assertNotBucketed call in DataFrameWriter::save optional

2018-12-10 Thread JOAQUIN GUANTER GONZALBEZ
: miƩrcoles, 5 de diciembre de 2018 15:51 Para: JOAQUIN GUANTER GONZALBEZ CC: Spark dev list Asunto: Re: [SPARK-26160] Make assertNotBucketed call in DataFrameWriter::save optional The bucket feature is designed to only work with data sources with table support, and currently the table support is

Re: [SPARK-26160] Make assertNotBucketed call in DataFrameWriter::save optional

2018-12-05 Thread Wenchen Fan
The bucket feature is designed to only work with data sources with table support, and currently the table support is not public yet, which means no external data sources can access bucketing information right now. The bucket feature only works with Spark native file source tables. We are working o

[SPARK-26160] Make assertNotBucketed call in DataFrameWriter::save optional

2018-11-26 Thread JOAQUIN GUANTER GONZALBEZ
Hello, I have a proposal for a small improvement in the Datasource API and I'd like to know if it sounds like a change the Spark project would accept. Currently, the `.save` method in DataFrameWriter will fail if the dataframe is bucketed and/or sorted. This makes sense, since there is no way o