JB, we do have tests for converting filters as well as for checking actual pushdown. Looks like we initially missed decimals but I've added them in PR 8682.
The more tests we have the better. If you have a bit of time, it would be nice to go back and check what else we missed. I'd start by looking at TestSparkV2Filter and TestFilterPushdown classes. On 2023/09/30 04:56:07 Jean-Baptiste Onofré wrote: > Thanks Anton, > > Do we have unit tests about filter pushdown ? Maybe worth to add > something around that right ? > > Anyway, I'm adding filter test cases in my "samples" repo. > > Thanks ! > Regards > JB > > On Fri, Sep 29, 2023 at 6:30 PM Anton Okolnychyi <aokolnyc...@apache.org> > wrote: > > > > Ugh, it looks like the filter pushdown issue is a regression. I tested > > 1.3.1 and it worked. I guess it is because we migrated to V2 filters and > > their behavior is different. I need to take a closer look. > > > > On 2023/09/29 11:36:09 Eduard Tudenhoefner wrote: > > > +1 (non-binding) > > > > > > * validated checksum and signature > > > * checked license docs & ran RAT checks > > > * ran build and tests with JDK17 > > > * ran some tests with Spark 3.4 + 3.5 and the new iceberg-aws-bundle.jar > > > * ran some internal tests > > > > > > I found two test issues with Flink (#8680 > > > <https://github.com/apache/iceberg/issues/8680> and #8679 > > > <https://github.com/apache/iceberg/issues/8679>) while running tests > > > locally, but that shouldn't block the release. > > > > > > Also thanks to Anton for doing the release and everyone else who > > > contributed! > > > > > > On Fri, Sep 29, 2023 at 11:44 AM Fokko Driesprong <fo...@apache.org> > > > wrote: > > > > > > > +1 (binding) > > > > > > > > Thanks Anton for running the release and everyone who contributed! > > > > Checks > > > > I did: > > > > > > > > - Updated the docker-spark-iceberg repo > > > > <https://github.com/tabular-io/docker-spark-iceberg/pull/93>, and > > > > everything runs fine (still with Spark 3.4 since there were some > > > > problems > > > > with Jupyters' Scala 2.13 kernel). This includes new new aws-bundle 🥳 > > > > - Tested against Trino <https://github.com/trinodb/trino/pull/19188>, > > > > and found three differences, but expected: > > > > - More defensive cleaning up of files on a failed commit, to make > > > > table recovery easier when needed. > > > > - A new property that's set on the table, indicating zstd > > > > compression. > > > > - Changes in the exceptions when binding a transform to a column > > > > type that is not allowed. > > > > > > > > Kind regards, Fokko > > > > > > > > > > > > Op vr 29 sep 2023 om 07:35 schreef Jean-Baptiste Onofré > > > > <j...@nanthrax.net>: > > > > > > > >> +1 (non binding) > > > >> > > > >> I checked: > > > >> - signatures and hash are ok > > > >> - asf headers are present > > > >> - no binary in the source distribution > > > >> - build is ok > > > >> > > > >> NB: I’m working on a set of use cases with different data sets but it’s > > > >> not yet complete. I should have it for next release and be able to > > > >> compare > > > >> queries time and behavior between releases. > > > >> > > > >> Thanks ! > > > >> Regards > > > >> JB > > > >> > > > >> Le jeu. 28 sept. 2023 à 04:02, Anton Okolnychyi > > > >> <aokolnyc...@apple.com.invalid> a écrit : > > > >> > > > >>> Hi Everyone, > > > >>> > > > >>> I propose that we release the following RC as the official Apache > > > >>> Iceberg 1.4.0 release. > > > >>> > > > >>> The commit ID is 8f37faa6a21e863551b17992370edc0f8706465d > > > >>> * This corresponds to the tag: apache-iceberg-1.4.0-rc1 > > > >>> * https://github.com/apache/iceberg/commits/apache-iceberg-1.4.0-rc1 > > > >>> * > > > >>> https://github.com/apache/iceberg/tree/8f37faa6a21e863551b17992370edc0f8706465d > > > >>> > > > >>> The release tarball, signature, and checksums are here: > > > >>> * > > > >>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.4.0-rc1 > > > >>> > > > >>> You can find the KEYS file here: > > > >>> * https://dist.apache.org/repos/dist/dev/iceberg/KEYS > > > >>> > > > >>> Convenience binary artifacts are staged on Nexus. The Maven repository > > > >>> URL is: > > > >>> * > > > >>> https://repository.apache.org/content/repositories/orgapacheiceberg-1145/ > > > >>> > > > >>> Please download, verify, and test. > > > >>> > > > >>> Please vote in the next 72 hours. (Weekends excluded) > > > >>> > > > >>> [ ] +1 Release this as Apache Iceberg 1.4.0 > > > >>> [ ] +0 > > > >>> [ ] -1 Do not release this because... > > > >>> > > > >>> Only PMC members have binding votes, but other community members are > > > >>> encouraged to cast non-binding votes. This vote will pass if there > > > >>> are 3 > > > >>> binding +1 votes and more binding +1 votes than -1 votes. > > > >>> > > > >>> - Anton > > > >> > > > >> > > > >
