This is going to fix it: https://github.com/apache/iceberg/pull/6200 The question is do we want to create another release candidate that contains this fix. I think we should. Adding the above patch to the 1.1.0 Milestone <https://github.com/apache/iceberg/milestone/24>.
Gabor On Wed, Nov 16, 2022 at 2:04 PM Gabor Kaszab <gaborkas...@apache.org> wrote: > I managed to repro this: Currently I'm in Central Europe and the current > date is 16th Nov, but if I set the system timezone to "Pacific/Auckland" > then LocalDate.now() gives me 17th Nov. This won't match with the current > UTC time created by ExpressionUtil.ExpressionSanitizer.today that is UTC. > > I have to examine the use of 'ExpressionUtil.ExpressionSanitizer.today' > and 'now' a bit further but it seems that these are compared with predicate > literals that in turn I hink are in UTC. So the code using UTC seems fine, > just the test has to also use UTC for getting the current day/time. > > On Wed, Nov 16, 2022 at 11:53 AM Gabor Kaszab <gaborkas...@apache.org> > wrote: > >> I believe the issue is that ExpressionUtil takes the current time as UTC >> <https://github.com/apache/iceberg/blob/c7365f0302048d5437a75ee30a5215d8d32aaef3/api/src/main/java/org/apache/iceberg/expressions/ExpressionUtil.java#L130> >> while these tests use LocalDate.now() >> <https://github.com/apache/iceberg/blob/c7365f0302048d5437a75ee30a5215d8d32aaef3/api/src/test/java/org/apache/iceberg/expressions/TestExpressionUtil.java#L541> >> that uses the "default" timezone >> <https://www.geeksforgeeks.org/localdate-now-method-in-java-with-examples/> >> that apparently is not UTC in your case. >> I think this is a test issue. >> >> Gabor >> >> On Wed, Nov 16, 2022 at 9:37 AM Gabor Kaszab <gaborkas...@apache.org> >> wrote: >> >>> Thanks for the verification Steven! >>> I also ran the iceberg-api tests multiple times but I haven't run into >>> this issue. Is it possible that this is due to some local configs in your >>> environment? It seems suspicious that all these 3 failing tests use >>> LocalDate.now() and these are the only tests that use this. >>> >>> Cheers, >>> Gabor >>> >>> >>> On Wed, Nov 16, 2022 at 7:33 AM Steven Wu <stevenz...@gmail.com> wrote: >>> >>>> -1 (non-binding) >>>> >>>> >>>> >>>> 1. Downloaded the source tarball. >>>> 1. Verified signature and checksum. >>>> 2. Untar the source archive. >>>> 3. ./gradlew build. There are unit test failures (details in the >>>> end). >>>> 2. Flink SQL testing (both 1.15 and 1.16) >>>> 1. Downloaded Iceberg-flink-runtime jars (1.15 and 1.16) from >>>> Nexus staging repository. >>>> 2. Batch write worked >>>> 3. FLIP-27 source batch read worked >>>> 4. FLIP-27 source streaming read worked >>>> 5. There is a regression in Flink 1.16.0 (not related to Iceberg >>>> release). Filed ticket. FLINK-30035. >>>> >>>> >>>> Thanks, >>>> >>>> Steven >>>> >>>> >>>> > Task :iceberg-api:test >>>> >>>> >>>> org.apache.iceberg.expressions.TestExpressionUtil > >>>> testSanitizeDateNextWeek FAILED >>>> >>>> java.lang.AssertionError: Literals should match >>>> expected:<["(date-7-days-from-now)"]> but was:<["(date-6-days-from-now)"]> >>>> >>>> at org.junit.Assert.fail(Assert.java:89) >>>> >>>> at org.junit.Assert.failNotEquals(Assert.java:835) >>>> >>>> at org.junit.Assert.assertEquals(Assert.java:120) >>>> >>>> at >>>> org.apache.iceberg.expressions.TestExpressionUtil.assertEquals(TestExpressionUtil.java:825) >>>> >>>> at >>>> org.apache.iceberg.expressions.TestExpressionUtil.assertEquals(TestExpressionUtil.java:819) >>>> >>>> at >>>> org.apache.iceberg.expressions.TestExpressionUtil.testSanitizeDateNextWeek(TestExpressionUtil.java:583) >>>> >>>> >>>> org.apache.iceberg.expressions.TestExpressionUtil > >>>> testSanitizeDateLastWeek FAILED >>>> >>>> java.lang.AssertionError: Literals should match >>>> expected:<["(date-7-days-ago)"]> but was:<["(date-8-days-ago)"]> >>>> >>>> at org.junit.Assert.fail(Assert.java:89) >>>> >>>> at org.junit.Assert.failNotEquals(Assert.java:835) >>>> >>>> at org.junit.Assert.assertEquals(Assert.java:120) >>>> >>>> at >>>> org.apache.iceberg.expressions.TestExpressionUtil.assertEquals(TestExpressionUtil.java:825) >>>> >>>> at >>>> org.apache.iceberg.expressions.TestExpressionUtil.assertEquals(TestExpressionUtil.java:819) >>>> >>>> at >>>> org.apache.iceberg.expressions.TestExpressionUtil.testSanitizeDateLastWeek(TestExpressionUtil.java:563) >>>> >>>> >>>> org.apache.iceberg.expressions.TestExpressionUtil > >>>> testSanitizeDateToday FAILED >>>> >>>> java.lang.AssertionError: Literals should match >>>> expected:<["(date-today)"]> but was:<["(date-1-days-ago)"]> >>>> >>>> at org.junit.Assert.fail(Assert.java:89) >>>> >>>> at org.junit.Assert.failNotEquals(Assert.java:835) >>>> >>>> at org.junit.Assert.assertEquals(Assert.java:120) >>>> >>>> at >>>> org.apache.iceberg.expressions.TestExpressionUtil.assertEquals(TestExpressionUtil.java:825) >>>> >>>> at >>>> org.apache.iceberg.expressions.TestExpressionUtil.assertEquals(TestExpressionUtil.java:819) >>>> >>>> at >>>> org.apache.iceberg.expressions.TestExpressionUtil.testSanitizeDateToday(TestExpressionUtil.java:543) >>>> >>>> On Tue, Nov 15, 2022 at 8:42 PM Jean-Baptiste Onofré <j...@nanthrax.net> >>>> wrote: >>>> >>>>> +1 (non binding) >>>>> >>>>> Quickly check build and "Apache legal related", not deep dive tests as >>>>> I'm still learning Iceberg ;) >>>>> >>>>> Regards >>>>> JB >>>>> >>>>> On Tue, Nov 15, 2022 at 12:28 PM Gabor Kaszab <gaborkas...@apache.org> >>>>> wrote: >>>>> > >>>>> > Hi Everyone, >>>>> > >>>>> > I propose that we release the following RC as the official Apache >>>>> Iceberg 1.1.0 release. >>>>> > >>>>> > The commit ID is 1d10c53906847893b45c6acb0137dcb55a15353d >>>>> > * This corresponds to the tag: apache-iceberg-1.1.0-rc1 >>>>> > * https://github.com/apache/iceberg/commits/apache-iceberg-1.1.0-rc1 >>>>> > * >>>>> https://github.com/apache/iceberg/tree/1d10c53906847893b45c6acb0137dcb55a15353d >>>>> > >>>>> > The release tarball, signature, and checksums are here: >>>>> > * >>>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.1.0-rc1 >>>>> > >>>>> > You can find the KEYS file here: >>>>> > * https://dist.apache.org/repos/dist/dev/iceberg/KEYS >>>>> > >>>>> > Convenience binary artifacts are staged on Nexus. The Maven >>>>> repository URL is: >>>>> > * >>>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1109/ >>>>> > >>>>> > Please download, verify, and test. >>>>> > >>>>> > Please vote in the next 72 hours. >>>>> > >>>>> > [ ] +1 Release this as Apache Iceberg 1.1.0 >>>>> > [ ] +0 >>>>> > [ ] -1 Do not release this because... >>>>> > >>>>> > PS.: Thanks Fokko for helping me out with the steps requiring >>>>> committer privileges! >>>>> >>>>
