About the part where we plan to drop the Hive module. This seems like a thing that has already been discussed and decided, so I might have missed that conversation.
I don’t remember saying that we plan to drop the Hive module, but given that Iceberg support has moved into Hive I think that we’ve been assuming that we would no longer maintain the Hive module. Would it make sense to involve Hive devs into this decision, though, so that we can make sure there is no difficulty on Hive side due to this? Mayba a vote (if we hadn’t had it already)? Yes, we should discuss deprecating the Hive support in Iceberg. Both whether we intend to deprecate and remove it and how/when we can do that. For example, we’ve removed support for older Spark versions in between major release versions and I think it would be fine to do so here as well. People are still free to use the modules from older Iceberg releases without problems and we use format versioning to guarantee compatibility with the older artifacts. I don’t think there’s a compelling reason to keep re-releasing those modules when they are not changing and are unmaintained. Does anyone disagree with the idea that we can stop releasing those modules before a 2.0 release? Ryan On Thu, Jul 11, 2024 at 1:09 AM Gabor Kaszab <gaborkas...@apache.org> wrote: > Hey, > > About the part where we plan to drop the Hive module. This seems like a > thing that has already been discussed and decided, so I might have missed > that conversation. Would it make sense to involve Hive devs into this > decision, though, so that we can make sure there is no difficulty on Hive > side due to this? Mayba a vote (if we hadn't had it already)? > > Cheers, > Gabor > > > On Wed, Jul 10, 2024 at 5:34 PM Robert Stupp <sn...@snazy.de> wrote: > >> +1 on deprecating Java 8 in the next "catchable" Iceberg release. >> >> +1 on removing Java 8 in Iceberg 2.0 >> >> +1 on _building_ for Java 11 in Iceberg 2.0 >> >> +1 on _building_ with Java 17 in Iceberg 2.0 >> >> >> I'd recommend to _build_ for Java 11 wherever possible (`javac --release >> 11`), because many users are concerned about bumping 2 Java LTS versions (8 >> ---> 17). >> >> >> From our experience, it's fine to bump the JDK in CI to Java 17, that's >> the Java version that works for Spark 3.3..3.5 (with some JPMS options). >> I'm less concerned about Spark 4 requiring Java 17 (`--release 17` can be >> configured for Spark 4, leaving everything at a lower version). >> >> I'm quite concerned about Avro 1.12 requiring Java 17 - maybe we can >> stick w/ Avro 1.11.x. >> >> >> On 10.07.24 15:13, Piotr Findeisen wrote: >> >> Hi, >> >> Thanks for additional feedback. It sounds like we're OK enabling builds >> and testing with JDK 21 with the caveats that formatter is off. >> The non-negotiable condition is that CI still checks format. >> I think this is exactly what this PR >> <https://github.com/apache/iceberg/pull/10474> is doing, which means we >> maybe can merge it. One small step forward. >> >> BTW there is no risk of `./gradlew spotlessApply` producing different >> results when being executed under JDK 21. >> The spotless plugin needs to be fully disabled, and then this command >> will fail saying the target is not found. >> It would be even better to provide custom spotlessApply command stub that >> would produce an actionable error message. >> I will add something like this to the above PR. >> >> re dropping Java 8 >> we need to figure out exact deprecation path and I don't know yet the >> customs to drive this fully. >> But happy to help as much as I can nonetheless. >> >> >> Best >> Piotr >> >> >> >> >> >> >> On Wed, 10 Jul 2024 at 11:19, Eduard Tudenhöfner < >> etudenhoef...@apache.org> wrote: >> >>> >>>> It has a caveat (we can't run formatter on 21 and 8, and we need to >>>>> choose one). >>>> >>>> >>>> Would it format differently? I would go for 21 since that's the path >>>> forward, but I'm also fine with JB's suggestion 👍 >>>> >>>>> >>>>> >>> Yeah it would format differently, because the underlying >>> *google-java-format* picks a different formatting depending on the JDK >>> version and that's why we had to pin the *google-java-format* version >>> to the one that still supports JDK8. >>> >>> I think it's ok to disable the formatter when running with JDK 21 for >>> people that want to use that JDK but we need to make sure that CI checks >>> still run JDK < 21 in order to make sure code is properly formatted. >>> >>> The one risk I see is that it will be difficult to understand why >>> running the build with JDK 21 passes locally but fails on CI (where an >>> older JDK is used). >>> Maybe it's sufficient to mention in the docs that *./gradlew >>> spotlessApply* needs to be executed locally with an older JDK. >>> >>> Eduard >>> >> -- >> Robert Stupp >> @snazy >> >> -- Ryan Blue Databricks
