Thanks for summarizing this, Piotr. I believe having a separate thread on dropping Java 8 is the right thing to do. We want to be as transparent about these changes as possible.
Kind regards, Fokko Driesprong Op ma 22 jul 2024 om 14:37 schreef Piotr Findeisen < piotr.findei...@gmail.com>: > Thanks for this lively discussion, it is great to see so many great people > involved! > > We have unanimous agreement that we add support for JDK 21. > Partial support (without spotless) will be added after 1.6.0 release is > out (just not to mess up with the release). > Full support (with spotless) will be added as soon as JDK 8 is dropped. > For 21 we have clarity and we don't have preconditions, this is great. > And this is a non-destructive operation too. > > We do not have agreement for dropping Hive module. This will be discussed > separately on a new thread. > > We also seem to have unanimous agreement for dropping JDK 8. > As to the timeline, it was proposed to do this in 2.0 release, so let's > roll with this, unless there are new objections > Since dropping support for something can be seen as a destructive > operation, does it require a formal vote? > Or do we treat +1 and -1 on this thread as votes already cast? > > Best, > Piotr > > > > > > > On Sat, 20 Jul 2024 at 00:12, Jack Ye <yezhao...@gmail.com> wrote: > >> +1 for dropping JDK8 support and adding JDK21. >> >> > What does dropping Java 8 support mean to companies that are still >> using Java 8 for Iceberg in production? >> >> From the AWS side, AWS Corretto JDK8 end of life is July 2026, see: >> https://aws.amazon.com/corretto/faqs/#support_calendar. I would suggest >> at least migrate before that time. >> >> -Jack >> >> >> >> On Fri, Jul 19, 2024 at 3:02 PM John Zhuge <jzh...@apache.org> wrote: >> >>> +1 adding java 21 support >>> +1 removing java 8 support >>> >>> On Fri, Jul 19, 2024 at 1:33 PM Daniel Weeks <dwe...@apache.org> wrote: >>> >>>> I'm also in favor of removing Java 8 support. Hive docs state Hive 3 >>>> requires java 8 >>>> <https://github.com/apache/hive?tab=readme-ov-file#java> and in prior >>>> cases there were potential correctness issues when running with newer Java >>>> versions (these may have been addressed). >>>> >>>> As long as we're not updating the target version, I think we should be >>>> ok as they can still run in Java 8 if that remains a requirement. >>>> >>>> +1 to removing Java 8 support >>>> +1 to adding Java 21 support. >>>> >>>> -Dan >>>> >>>> >>>> >>>> On Fri, Jul 19, 2024 at 1:04 PM Ryan Blue <b...@databricks.com.invalid> >>>> wrote: >>>> >>>>> I agree that if we can separate the discussion about how to support >>>>> Hive, then we should do that. >>>>> >>>>> +1 to removing Java 8 support >>>>> +1 to adding Java 21 support. >>>>> >>>>> On Fri, Jul 19, 2024 at 12:58 PM huaxin gao <huaxin.ga...@gmail.com> >>>>> wrote: >>>>> >>>>>> +1 in favor of adding java 21 support >>>>>> +1 in favor of removing java 8 support >>>>>> >>>>>> I am currently working on Spark 4.0 / Iceberg integration >>>>>> <https://github.com/apache/iceberg/pull/10622>. Spark 4.0 runs on >>>>>> Java 17/21. >>>>>> >>>>>> On Fri, Jul 19, 2024 at 4:58 AM Piotr Findeisen < >>>>>> piotr.findei...@gmail.com> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> We recently started to test Hive3 with Java 11 and 17 >>>>>>> <https://github.com/apache/iceberg/pull/10482> and the tests pass. >>>>>>> So dropping Java 8 doesn't technically require removing the Hive 3 >>>>>>> related modules, unless users cannot do anything useful with them >>>>>>> (because >>>>>>> e.g. they can only run Hive runtime with Java 8 for some reason). >>>>>>> Peter, can you please confirm this is not the case? >>>>>>> Then it seems we could proceed with JDK 8 drop and discuss what to >>>>>>> do with Hive modules *separately*. >>>>>>> >>>>>>> re original question of adding JDK 21 support -- we seem to have >>>>>>> strong consensus to add it. >>>>>>> Eduard plans to merge the PR once 1.6.0 is out. So I think we no >>>>>>> longer need to debate this topic, unless there are any new objections >>>>>>> to be >>>>>>> raised. >>>>>>> >>>>>>> >>>>>>> Best >>>>>>> Piotr >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Fri, 19 Jul 2024 at 13:49, Péter Váry < >>>>>>> peter.vary.apa...@gmail.com> wrote: >>>>>>> >>>>>>>> Back to the main topic: >>>>>>>> >>>>>>>> Removing java8 support still allows us to use the old HMS >>>>>>>> libraries, so iceberg-hive-metastore module is not affected in this >>>>>>>> sense. >>>>>>>> We still need to run the tests, to ensure that there is no changes in >>>>>>>> the >>>>>>>> behaviour, but it is unlikely. >>>>>>>> >>>>>>>> +1 in favor of adding java 21 support >>>>>>>> +1 in favor of removing java 8 support - especially if it is not >>>>>>>> bound to removing Hive support as well >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Peter >>>>>>>> >>>>>>>> Cheng Pan <pan3...@gmail.com> ezt írta (időpont: 2024. júl. 19., >>>>>>>> P, 4:56): >>>>>>>> >>>>>>>>> A basic question, is iceberg-hive-metastore considered part of the >>>>>>>>> "Hive module"? >>>>>>>>> >>>>>>>>> I suppose that HMS 2.x is still widely used. AFAIK, the current >>>>>>>>> iceberg-hive-metastore is compatible with HMS 2.1+, based on >>>>>>>>> Iceberg >>>>>>>>> and Spark CI, I also suppose it works well with Java 8 to 21. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Cheng Pan >>>>>>>>> >>>>>>>>> On Fri, Jul 19, 2024 at 4:44 AM Ryan Blue >>>>>>>>> <b...@databricks.com.invalid> wrote: >>>>>>>>> > >>>>>>>>> > Thanks for the context, Denys and Peter. Sounds like there's a >>>>>>>>> good question here about where the Hive integration should live and >>>>>>>>> the >>>>>>>>> most recent decision was to maintain that support in Hive. I >>>>>>>>> definitely >>>>>>>>> hear the point about Hive 3 users depending on the Iceberg modules. >>>>>>>>> I'm >>>>>>>>> also glad to hear that some of the issues are expected to be fixed >>>>>>>>> with the >>>>>>>>> release of Hive 4.0.x. >>>>>>>>> > >>>>>>>>> > I think that we have two separate questions for how to move >>>>>>>>> forward with Hive support, depending on the Hive version. There is a >>>>>>>>> question about what we do with the current Hive modules and what to >>>>>>>>> do with >>>>>>>>> the Hive 4 support that has been developed externally. >>>>>>>>> > >>>>>>>>> > For Hive 2.x and 3.x, we have code in the Iceberg repo that is >>>>>>>>> not being developed. Hive 2 is fairly easy since it is EOL. While >>>>>>>>> Hive 3 is >>>>>>>>> still used, I don't think it makes sense to keep releasing versions >>>>>>>>> of it >>>>>>>>> if it requires Java 8, which has not been publicly maintained for 5 >>>>>>>>> years. >>>>>>>>> We need to upgrade and that is at odds with keeping support for Hive >>>>>>>>> 3. As >>>>>>>>> Fokko and I both pointed out, people can still use older releases. >>>>>>>>> > >>>>>>>>> > For the question of how to maintain support for Hive 4, I think >>>>>>>>> it's worth having a separate discussion (probably not on the thread >>>>>>>>> about >>>>>>>>> JDK versions) about where to maintain it. I think that it is best to >>>>>>>>> maintain integration in engines and not in the Iceberg project; there >>>>>>>>> are >>>>>>>>> few implementations here and I think that it is a hard problem for >>>>>>>>> Iceberg >>>>>>>>> to maintain support for multiple versions (as you can see with >>>>>>>>> support for >>>>>>>>> so many different Flink, Hive, and Spark versions). >>>>>>>>> > >>>>>>>>> > Ryan >>>>>>>>> > >>>>>>>>> > On Thu, Jul 18, 2024 at 7:25 AM Denys Kuzmenko < >>>>>>>>> dkuzme...@apache.org> wrote: >>>>>>>>> >> >>>>>>>>> >> In the following 1-2 months we plan to release HIVE-4.0.1 which >>>>>>>>> includes bug fixes and then focus on HIVE-4.1.0 release with jdk17. >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > -- >>>>>>>>> > Ryan Blue >>>>>>>>> > Databricks >>>>>>>>> >>>>>>>> >>>>> >>>>> -- >>>>> Ryan Blue >>>>> Databricks >>>>> >>>> >>> >>> -- >>> John Zhuge >>> >>
