I managed to repro this: Currently I'm in Central Europe and the current date is 16th Nov, but if I set the system timezone to "Pacific/Auckland" then LocalDate.now() gives me 17th Nov. This won't match with the current UTC time created by ExpressionUtil.ExpressionSanitizer.today that is UTC.
I have to examine the use of 'ExpressionUtil.ExpressionSanitizer.today' and 'now' a bit further but it seems that these are compared with predicate literals that in turn I hink are in UTC. So the code using UTC seems fine, just the test has to also use UTC for getting the current day/time. On Wed, Nov 16, 2022 at 11:53 AM Gabor Kaszab <[email protected]> wrote: > I believe the issue is that ExpressionUtil takes the current time as UTC > <https://github.com/apache/iceberg/blob/c7365f0302048d5437a75ee30a5215d8d32aaef3/api/src/main/java/org/apache/iceberg/expressions/ExpressionUtil.java#L130> > while these tests use LocalDate.now() > <https://github.com/apache/iceberg/blob/c7365f0302048d5437a75ee30a5215d8d32aaef3/api/src/test/java/org/apache/iceberg/expressions/TestExpressionUtil.java#L541> > that uses the "default" timezone > <https://www.geeksforgeeks.org/localdate-now-method-in-java-with-examples/> > that apparently is not UTC in your case. > I think this is a test issue. > > Gabor > > On Wed, Nov 16, 2022 at 9:37 AM Gabor Kaszab <[email protected]> > wrote: > >> Thanks for the verification Steven! >> I also ran the iceberg-api tests multiple times but I haven't run into >> this issue. Is it possible that this is due to some local configs in your >> environment? It seems suspicious that all these 3 failing tests use >> LocalDate.now() and these are the only tests that use this. >> >> Cheers, >> Gabor >> >> >> On Wed, Nov 16, 2022 at 7:33 AM Steven Wu <[email protected]> wrote: >> >>> -1 (non-binding) >>> >>> >>> >>> 1. Downloaded the source tarball. >>> 1. Verified signature and checksum. >>> 2. Untar the source archive. >>> 3. ./gradlew build. There are unit test failures (details in the >>> end). >>> 2. Flink SQL testing (both 1.15 and 1.16) >>> 1. Downloaded Iceberg-flink-runtime jars (1.15 and 1.16) from >>> Nexus staging repository. >>> 2. Batch write worked >>> 3. FLIP-27 source batch read worked >>> 4. FLIP-27 source streaming read worked >>> 5. There is a regression in Flink 1.16.0 (not related to Iceberg >>> release). Filed ticket. FLINK-30035. >>> >>> >>> Thanks, >>> >>> Steven >>> >>> >>> > Task :iceberg-api:test >>> >>> >>> org.apache.iceberg.expressions.TestExpressionUtil > >>> testSanitizeDateNextWeek FAILED >>> >>> java.lang.AssertionError: Literals should match >>> expected:<["(date-7-days-from-now)"]> but was:<["(date-6-days-from-now)"]> >>> >>> at org.junit.Assert.fail(Assert.java:89) >>> >>> at org.junit.Assert.failNotEquals(Assert.java:835) >>> >>> at org.junit.Assert.assertEquals(Assert.java:120) >>> >>> at >>> org.apache.iceberg.expressions.TestExpressionUtil.assertEquals(TestExpressionUtil.java:825) >>> >>> at >>> org.apache.iceberg.expressions.TestExpressionUtil.assertEquals(TestExpressionUtil.java:819) >>> >>> at >>> org.apache.iceberg.expressions.TestExpressionUtil.testSanitizeDateNextWeek(TestExpressionUtil.java:583) >>> >>> >>> org.apache.iceberg.expressions.TestExpressionUtil > >>> testSanitizeDateLastWeek FAILED >>> >>> java.lang.AssertionError: Literals should match >>> expected:<["(date-7-days-ago)"]> but was:<["(date-8-days-ago)"]> >>> >>> at org.junit.Assert.fail(Assert.java:89) >>> >>> at org.junit.Assert.failNotEquals(Assert.java:835) >>> >>> at org.junit.Assert.assertEquals(Assert.java:120) >>> >>> at >>> org.apache.iceberg.expressions.TestExpressionUtil.assertEquals(TestExpressionUtil.java:825) >>> >>> at >>> org.apache.iceberg.expressions.TestExpressionUtil.assertEquals(TestExpressionUtil.java:819) >>> >>> at >>> org.apache.iceberg.expressions.TestExpressionUtil.testSanitizeDateLastWeek(TestExpressionUtil.java:563) >>> >>> >>> org.apache.iceberg.expressions.TestExpressionUtil > >>> testSanitizeDateToday FAILED >>> >>> java.lang.AssertionError: Literals should match >>> expected:<["(date-today)"]> but was:<["(date-1-days-ago)"]> >>> >>> at org.junit.Assert.fail(Assert.java:89) >>> >>> at org.junit.Assert.failNotEquals(Assert.java:835) >>> >>> at org.junit.Assert.assertEquals(Assert.java:120) >>> >>> at >>> org.apache.iceberg.expressions.TestExpressionUtil.assertEquals(TestExpressionUtil.java:825) >>> >>> at >>> org.apache.iceberg.expressions.TestExpressionUtil.assertEquals(TestExpressionUtil.java:819) >>> >>> at >>> org.apache.iceberg.expressions.TestExpressionUtil.testSanitizeDateToday(TestExpressionUtil.java:543) >>> >>> On Tue, Nov 15, 2022 at 8:42 PM Jean-Baptiste Onofré <[email protected]> >>> wrote: >>> >>>> +1 (non binding) >>>> >>>> Quickly check build and "Apache legal related", not deep dive tests as >>>> I'm still learning Iceberg ;) >>>> >>>> Regards >>>> JB >>>> >>>> On Tue, Nov 15, 2022 at 12:28 PM Gabor Kaszab <[email protected]> >>>> wrote: >>>> > >>>> > Hi Everyone, >>>> > >>>> > I propose that we release the following RC as the official Apache >>>> Iceberg 1.1.0 release. >>>> > >>>> > The commit ID is 1d10c53906847893b45c6acb0137dcb55a15353d >>>> > * This corresponds to the tag: apache-iceberg-1.1.0-rc1 >>>> > * https://github.com/apache/iceberg/commits/apache-iceberg-1.1.0-rc1 >>>> > * >>>> https://github.com/apache/iceberg/tree/1d10c53906847893b45c6acb0137dcb55a15353d >>>> > >>>> > The release tarball, signature, and checksums are here: >>>> > * >>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.1.0-rc1 >>>> > >>>> > You can find the KEYS file here: >>>> > * https://dist.apache.org/repos/dist/dev/iceberg/KEYS >>>> > >>>> > Convenience binary artifacts are staged on Nexus. The Maven >>>> repository URL is: >>>> > * >>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1109/ >>>> > >>>> > Please download, verify, and test. >>>> > >>>> > Please vote in the next 72 hours. >>>> > >>>> > [ ] +1 Release this as Apache Iceberg 1.1.0 >>>> > [ ] +0 >>>> > [ ] -1 Do not release this because... >>>> > >>>> > PS.: Thanks Fokko for helping me out with the steps requiring >>>> committer privileges! >>>> >>>
