I just merged the fix. Thanks for catching this! On Wed, Nov 16, 2022 at 5:29 AM Gabor Kaszab <gaborkas...@apache.org> wrote:
> This is going to fix it: https://github.com/apache/iceberg/pull/6200 > The question is do we want to create another release candidate that > contains this fix. I think we should. Adding the above patch to the 1.1.0 > Milestone <https://github.com/apache/iceberg/milestone/24>. > > Gabor > > On Wed, Nov 16, 2022 at 2:04 PM Gabor Kaszab <gaborkas...@apache.org> > wrote: > >> I managed to repro this: Currently I'm in Central Europe and the current >> date is 16th Nov, but if I set the system timezone to "Pacific/Auckland" >> then LocalDate.now() gives me 17th Nov. This won't match with the current >> UTC time created by ExpressionUtil.ExpressionSanitizer.today that is UTC. >> >> I have to examine the use of 'ExpressionUtil.ExpressionSanitizer.today' >> and 'now' a bit further but it seems that these are compared with predicate >> literals that in turn I hink are in UTC. So the code using UTC seems fine, >> just the test has to also use UTC for getting the current day/time. >> >> On Wed, Nov 16, 2022 at 11:53 AM Gabor Kaszab <gaborkas...@apache.org> >> wrote: >> >>> I believe the issue is that ExpressionUtil takes the current time as UTC >>> <https://github.com/apache/iceberg/blob/c7365f0302048d5437a75ee30a5215d8d32aaef3/api/src/main/java/org/apache/iceberg/expressions/ExpressionUtil.java#L130> >>> while these tests use LocalDate.now() >>> <https://github.com/apache/iceberg/blob/c7365f0302048d5437a75ee30a5215d8d32aaef3/api/src/test/java/org/apache/iceberg/expressions/TestExpressionUtil.java#L541> >>> that uses the "default" timezone >>> <https://www.geeksforgeeks.org/localdate-now-method-in-java-with-examples/> >>> that apparently is not UTC in your case. >>> I think this is a test issue. >>> >>> Gabor >>> >>> On Wed, Nov 16, 2022 at 9:37 AM Gabor Kaszab <gaborkas...@apache.org> >>> wrote: >>> >>>> Thanks for the verification Steven! >>>> I also ran the iceberg-api tests multiple times but I haven't run into >>>> this issue. Is it possible that this is due to some local configs in your >>>> environment? It seems suspicious that all these 3 failing tests use >>>> LocalDate.now() and these are the only tests that use this. >>>> >>>> Cheers, >>>> Gabor >>>> >>>> >>>> On Wed, Nov 16, 2022 at 7:33 AM Steven Wu <stevenz...@gmail.com> wrote: >>>> >>>>> -1 (non-binding) >>>>> >>>>> >>>>> >>>>> 1. Downloaded the source tarball. >>>>> 1. Verified signature and checksum. >>>>> 2. Untar the source archive. >>>>> 3. ./gradlew build. There are unit test failures (details in >>>>> the end). >>>>> 2. Flink SQL testing (both 1.15 and 1.16) >>>>> 1. Downloaded Iceberg-flink-runtime jars (1.15 and 1.16) from >>>>> Nexus staging repository. >>>>> 2. Batch write worked >>>>> 3. FLIP-27 source batch read worked >>>>> 4. FLIP-27 source streaming read worked >>>>> 5. There is a regression in Flink 1.16.0 (not related to >>>>> Iceberg release). Filed ticket. FLINK-30035. >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Steven >>>>> >>>>> >>>>> > Task :iceberg-api:test >>>>> >>>>> >>>>> org.apache.iceberg.expressions.TestExpressionUtil > >>>>> testSanitizeDateNextWeek FAILED >>>>> >>>>> java.lang.AssertionError: Literals should match >>>>> expected:<["(date-7-days-from-now)"]> but was:<["(date-6-days-from-now)"]> >>>>> >>>>> at org.junit.Assert.fail(Assert.java:89) >>>>> >>>>> at org.junit.Assert.failNotEquals(Assert.java:835) >>>>> >>>>> at org.junit.Assert.assertEquals(Assert.java:120) >>>>> >>>>> at >>>>> org.apache.iceberg.expressions.TestExpressionUtil.assertEquals(TestExpressionUtil.java:825) >>>>> >>>>> at >>>>> org.apache.iceberg.expressions.TestExpressionUtil.assertEquals(TestExpressionUtil.java:819) >>>>> >>>>> at >>>>> org.apache.iceberg.expressions.TestExpressionUtil.testSanitizeDateNextWeek(TestExpressionUtil.java:583) >>>>> >>>>> >>>>> org.apache.iceberg.expressions.TestExpressionUtil > >>>>> testSanitizeDateLastWeek FAILED >>>>> >>>>> java.lang.AssertionError: Literals should match >>>>> expected:<["(date-7-days-ago)"]> but was:<["(date-8-days-ago)"]> >>>>> >>>>> at org.junit.Assert.fail(Assert.java:89) >>>>> >>>>> at org.junit.Assert.failNotEquals(Assert.java:835) >>>>> >>>>> at org.junit.Assert.assertEquals(Assert.java:120) >>>>> >>>>> at >>>>> org.apache.iceberg.expressions.TestExpressionUtil.assertEquals(TestExpressionUtil.java:825) >>>>> >>>>> at >>>>> org.apache.iceberg.expressions.TestExpressionUtil.assertEquals(TestExpressionUtil.java:819) >>>>> >>>>> at >>>>> org.apache.iceberg.expressions.TestExpressionUtil.testSanitizeDateLastWeek(TestExpressionUtil.java:563) >>>>> >>>>> >>>>> org.apache.iceberg.expressions.TestExpressionUtil > >>>>> testSanitizeDateToday FAILED >>>>> >>>>> java.lang.AssertionError: Literals should match >>>>> expected:<["(date-today)"]> but was:<["(date-1-days-ago)"]> >>>>> >>>>> at org.junit.Assert.fail(Assert.java:89) >>>>> >>>>> at org.junit.Assert.failNotEquals(Assert.java:835) >>>>> >>>>> at org.junit.Assert.assertEquals(Assert.java:120) >>>>> >>>>> at >>>>> org.apache.iceberg.expressions.TestExpressionUtil.assertEquals(TestExpressionUtil.java:825) >>>>> >>>>> at >>>>> org.apache.iceberg.expressions.TestExpressionUtil.assertEquals(TestExpressionUtil.java:819) >>>>> >>>>> at >>>>> org.apache.iceberg.expressions.TestExpressionUtil.testSanitizeDateToday(TestExpressionUtil.java:543) >>>>> >>>>> On Tue, Nov 15, 2022 at 8:42 PM Jean-Baptiste Onofré <j...@nanthrax.net> >>>>> wrote: >>>>> >>>>>> +1 (non binding) >>>>>> >>>>>> Quickly check build and "Apache legal related", not deep dive tests as >>>>>> I'm still learning Iceberg ;) >>>>>> >>>>>> Regards >>>>>> JB >>>>>> >>>>>> On Tue, Nov 15, 2022 at 12:28 PM Gabor Kaszab <gaborkas...@apache.org> >>>>>> wrote: >>>>>> > >>>>>> > Hi Everyone, >>>>>> > >>>>>> > I propose that we release the following RC as the official Apache >>>>>> Iceberg 1.1.0 release. >>>>>> > >>>>>> > The commit ID is 1d10c53906847893b45c6acb0137dcb55a15353d >>>>>> > * This corresponds to the tag: apache-iceberg-1.1.0-rc1 >>>>>> > * >>>>>> https://github.com/apache/iceberg/commits/apache-iceberg-1.1.0-rc1 >>>>>> > * >>>>>> https://github.com/apache/iceberg/tree/1d10c53906847893b45c6acb0137dcb55a15353d >>>>>> > >>>>>> > The release tarball, signature, and checksums are here: >>>>>> > * >>>>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.1.0-rc1 >>>>>> > >>>>>> > You can find the KEYS file here: >>>>>> > * https://dist.apache.org/repos/dist/dev/iceberg/KEYS >>>>>> > >>>>>> > Convenience binary artifacts are staged on Nexus. The Maven >>>>>> repository URL is: >>>>>> > * >>>>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1109/ >>>>>> > >>>>>> > Please download, verify, and test. >>>>>> > >>>>>> > Please vote in the next 72 hours. >>>>>> > >>>>>> > [ ] +1 Release this as Apache Iceberg 1.1.0 >>>>>> > [ ] +0 >>>>>> > [ ] -1 Do not release this because... >>>>>> > >>>>>> > PS.: Thanks Fokko for helping me out with the steps requiring >>>>>> committer privileges! >>>>>> >>>>> -- Ryan Blue Tabular
