Re: Ad-hoc partition bucketing

2023-07-06 Thread Joseph Allemandou
Thank you Manu and Russell for your answers, Would there be any document / ticket / commit where I could find some information or example on how the partition transforms are implemented, and the various code places they involve touching ? Many thanks again :) Joseph On Wed, Jul 5, 2023 at 1:43 P

Re: [DISCUSS] Apache Iceberg Release 1.3.1

2023-07-06 Thread Jean-Baptiste Onofré
Hi, It sounds good to me to have 1.3.1. Thanks ! Regards JB On Fri, Jul 7, 2023 at 12:53 AM Szehon Ho wrote: > > Hi > > I wanted to start a discussion for whether its the right time for 1.3.1, a > patch release of 1.3.0. It was started based on the issue found by Xiangyang > (@ConeyLiu) : >

Re: [DISCUSS] Apache Iceberg Release 1.3.1

2023-07-06 Thread Ajantha Bhat
+1 for the 1.3.1 release as it is a functional bug for Hive catalog. While we are at it, I would like to have https://github.com/apache/iceberg/pull/7893 as part of 1.3.1 which can unblock the Trino side bug fix for Nessie. Note: We also need https://github.com/apache/iceberg/pull/7641 for a smoo

[DISCUSS] Apache Iceberg Release 1.3.1

2023-07-06 Thread Szehon Ho
Hi I wanted to start a discussion for whether its the right time for 1.3.1, a patch release of 1.3.0. It was started based on the issue found by Xiangyang (@ConeyLiu) : https://github.com/apache/iceberg/pull/7931#pullrequestreview-1507935277. Do people have any other bug fixes that should be inc

Rust support support

2023-07-06 Thread Brian Olsen
Hey Iceberg Nation, >From Reddit: https://www.reddit.com/r/dataengineering/comments/14rcaj9/iceberg_won_the_table_format_war_but_not_in_the/jqva6if/ What I want most right now is a native library for Iceberg format, that can be used to read/write Iceberg table from multiple programming languages,

Re: allowing configs to be specified in SQLConf for Spark reads/writes

2023-07-06 Thread Liwei Li
Also +1 for OPTIONS hints. It is useful to allow some options to be specified in SQLConf. On Thu, Jul 6, 2023 at 1:05 AM Ryan Blue wrote: > Cheng, that's true of certain options that are targeted at administrators. > But the DataFrameReader or DataFrameWriter options are job-specific, which > i

The roadmap is outdated

2023-07-06 Thread Liwei Li
Hi, The roadmap has not been updated for a long time, do we have any plans to update it? Thanks, Liwei