Thanks for this lively discussion, it is great to see so many great people involved!
We have unanimous agreement that we add support for JDK 21. Partial support (without spotless) will be added after 1.6.0 release is out (just not to mess up with the release). Full support (with spotless) will be added as soon as JDK 8 is dropped. For 21 we have clarity and we don't have preconditions, this is great. And this is a non-destructive operation too. We do not have agreement for dropping Hive module. This will be discussed separately on a new thread. We also seem to have unanimous agreement for dropping JDK 8. As to the timeline, it was proposed to do this in 2.0 release, so let's roll with this, unless there are new objections Since dropping support for something can be seen as a destructive operation, does it require a formal vote? Or do we treat +1 and -1 on this thread as votes already cast? Best, Piotr On Sat, 20 Jul 2024 at 00:12, Jack Ye <yezhao...@gmail.com> wrote: > +1 for dropping JDK8 support and adding JDK21. > > > What does dropping Java 8 support mean to companies that are still using > Java 8 for Iceberg in production? > > From the AWS side, AWS Corretto JDK8 end of life is July 2026, see: > https://aws.amazon.com/corretto/faqs/#support_calendar. I would suggest > at least migrate before that time. > > -Jack > > > > On Fri, Jul 19, 2024 at 3:02 PM John Zhuge <jzh...@apache.org> wrote: > >> +1 adding java 21 support >> +1 removing java 8 support >> >> On Fri, Jul 19, 2024 at 1:33 PM Daniel Weeks <dwe...@apache.org> wrote: >> >>> I'm also in favor of removing Java 8 support. Hive docs state Hive 3 >>> requires java 8 <https://github.com/apache/hive?tab=readme-ov-file#java> and >>> in prior cases there were potential correctness issues when running with >>> newer Java versions (these may have been addressed). >>> >>> As long as we're not updating the target version, I think we should be >>> ok as they can still run in Java 8 if that remains a requirement. >>> >>> +1 to removing Java 8 support >>> +1 to adding Java 21 support. >>> >>> -Dan >>> >>> >>> >>> On Fri, Jul 19, 2024 at 1:04 PM Ryan Blue <b...@databricks.com.invalid> >>> wrote: >>> >>>> I agree that if we can separate the discussion about how to support >>>> Hive, then we should do that. >>>> >>>> +1 to removing Java 8 support >>>> +1 to adding Java 21 support. >>>> >>>> On Fri, Jul 19, 2024 at 12:58 PM huaxin gao <huaxin.ga...@gmail.com> >>>> wrote: >>>> >>>>> +1 in favor of adding java 21 support >>>>> +1 in favor of removing java 8 support >>>>> >>>>> I am currently working on Spark 4.0 / Iceberg integration >>>>> <https://github.com/apache/iceberg/pull/10622>. Spark 4.0 runs on >>>>> Java 17/21. >>>>> >>>>> On Fri, Jul 19, 2024 at 4:58 AM Piotr Findeisen < >>>>> piotr.findei...@gmail.com> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> We recently started to test Hive3 with Java 11 and 17 >>>>>> <https://github.com/apache/iceberg/pull/10482> and the tests pass. >>>>>> So dropping Java 8 doesn't technically require removing the Hive 3 >>>>>> related modules, unless users cannot do anything useful with them >>>>>> (because >>>>>> e.g. they can only run Hive runtime with Java 8 for some reason). >>>>>> Peter, can you please confirm this is not the case? >>>>>> Then it seems we could proceed with JDK 8 drop and discuss what to do >>>>>> with Hive modules *separately*. >>>>>> >>>>>> re original question of adding JDK 21 support -- we seem to have >>>>>> strong consensus to add it. >>>>>> Eduard plans to merge the PR once 1.6.0 is out. So I think we no >>>>>> longer need to debate this topic, unless there are any new objections to >>>>>> be >>>>>> raised. >>>>>> >>>>>> >>>>>> Best >>>>>> Piotr >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Fri, 19 Jul 2024 at 13:49, Péter Váry <peter.vary.apa...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Back to the main topic: >>>>>>> >>>>>>> Removing java8 support still allows us to use the old HMS libraries, >>>>>>> so iceberg-hive-metastore module is not affected in this sense. We still >>>>>>> need to run the tests, to ensure that there is no changes in the >>>>>>> behaviour, >>>>>>> but it is unlikely. >>>>>>> >>>>>>> +1 in favor of adding java 21 support >>>>>>> +1 in favor of removing java 8 support - especially if it is not >>>>>>> bound to removing Hive support as well >>>>>>> >>>>>>> Thanks, >>>>>>> Peter >>>>>>> >>>>>>> Cheng Pan <pan3...@gmail.com> ezt írta (időpont: 2024. júl. 19., P, >>>>>>> 4:56): >>>>>>> >>>>>>>> A basic question, is iceberg-hive-metastore considered part of the >>>>>>>> "Hive module"? >>>>>>>> >>>>>>>> I suppose that HMS 2.x is still widely used. AFAIK, the current >>>>>>>> iceberg-hive-metastore is compatible with HMS 2.1+, based on Iceberg >>>>>>>> and Spark CI, I also suppose it works well with Java 8 to 21. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Cheng Pan >>>>>>>> >>>>>>>> On Fri, Jul 19, 2024 at 4:44 AM Ryan Blue >>>>>>>> <b...@databricks.com.invalid> wrote: >>>>>>>> > >>>>>>>> > Thanks for the context, Denys and Peter. Sounds like there's a >>>>>>>> good question here about where the Hive integration should live and the >>>>>>>> most recent decision was to maintain that support in Hive. I definitely >>>>>>>> hear the point about Hive 3 users depending on the Iceberg modules. I'm >>>>>>>> also glad to hear that some of the issues are expected to be fixed >>>>>>>> with the >>>>>>>> release of Hive 4.0.x. >>>>>>>> > >>>>>>>> > I think that we have two separate questions for how to move >>>>>>>> forward with Hive support, depending on the Hive version. There is a >>>>>>>> question about what we do with the current Hive modules and what to do >>>>>>>> with >>>>>>>> the Hive 4 support that has been developed externally. >>>>>>>> > >>>>>>>> > For Hive 2.x and 3.x, we have code in the Iceberg repo that is >>>>>>>> not being developed. Hive 2 is fairly easy since it is EOL. While Hive >>>>>>>> 3 is >>>>>>>> still used, I don't think it makes sense to keep releasing versions of >>>>>>>> it >>>>>>>> if it requires Java 8, which has not been publicly maintained for 5 >>>>>>>> years. >>>>>>>> We need to upgrade and that is at odds with keeping support for Hive >>>>>>>> 3. As >>>>>>>> Fokko and I both pointed out, people can still use older releases. >>>>>>>> > >>>>>>>> > For the question of how to maintain support for Hive 4, I think >>>>>>>> it's worth having a separate discussion (probably not on the thread >>>>>>>> about >>>>>>>> JDK versions) about where to maintain it. I think that it is best to >>>>>>>> maintain integration in engines and not in the Iceberg project; there >>>>>>>> are >>>>>>>> few implementations here and I think that it is a hard problem for >>>>>>>> Iceberg >>>>>>>> to maintain support for multiple versions (as you can see with support >>>>>>>> for >>>>>>>> so many different Flink, Hive, and Spark versions). >>>>>>>> > >>>>>>>> > Ryan >>>>>>>> > >>>>>>>> > On Thu, Jul 18, 2024 at 7:25 AM Denys Kuzmenko < >>>>>>>> dkuzme...@apache.org> wrote: >>>>>>>> >> >>>>>>>> >> In the following 1-2 months we plan to release HIVE-4.0.1 which >>>>>>>> includes bug fixes and then focus on HIVE-4.1.0 release with jdk17. >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > -- >>>>>>>> > Ryan Blue >>>>>>>> > Databricks >>>>>>>> >>>>>>> >>>> >>>> -- >>>> Ryan Blue >>>> Databricks >>>> >>> >> >> -- >> John Zhuge >> >
