Hi JB, thanks for raising this. - With the Gradle version update, we will be able to upgrade to Parquet > 1.14.0
We might want to defer this until Parquet 1.14.1 gets released. There is an issue found with Jackson <https://github.com/apache/parquet-java/pull/1349> that prohibits Spark from upgrading to 1.14.0. It might be that this is also the case with other query engines. Gang already started a DISCUSS thread <https://lists.apache.org/thread/h589zkwnpo592gc9n17v0y7qvd9jv1z4> on running the minor release. - Depending of the timing, I will include new Avro releases I would love that! With the release it allows us to use the BlockingDirectBinaryEncoder <https://github.com/apache/avro/pull/2732>, which will allow efficient skipping of lists and maps <https://github.com/apache/iceberg/pull/8625> in the file. This will speed up operations like expire snapshots quite a bit because we can jump easily over all the statistics. It would be great if there were some Flink reviewers to get some eyes on this PR to run Flink without Hadoop <https://github.com/apache/iceberg/pull/7369>. We have to change some APIs and with the introduction of Flink 1.19 in Iceberg 1.16.0 this is a great moment. Kind regards, Fokko Op ma 10 jun 2024 om 00:24 schreef Jean-Baptiste Onofré <j...@nanthrax.net>: > Hi folks, > > As discussed during the last community meeting, we are heading to the > Apache Iceberg 1.6.0 release. > > The Iceberg 1.6.0 milestone is present for both GitHub Issues and PRs. > > I'm targeting major updates for this release: > - the Kafka commit coordinator (PR is in review) > - Revapi "fix" to be able to upgrade to latest Gradle version (I have > one PR ready and I have another exploring a new option) > - With the Gradle version update, we will be able to upgrade to Parquet > 1.14.0 > - Depending of the timing, I will include new Avro releases > > Please, if you have anything you want to include in the 1.6.0 release, > please let me know and create an issue on GitHub with the "Iceberg > 1.6.0" milestone. > > Thanks ! > > Regards > JB >
