Thanks Fokko. I like the idea. Started a new "Dropping JDK 8 support" thread to ensure transparency.
Best Piotr On Mon, 22 Jul 2024 at 15:24, Fokko Driesprong <fo...@apache.org> wrote: > Thanks for summarizing this, Piotr. > > I believe having a separate thread on dropping Java 8 is the right thing > to do. We want to be as transparent about these changes as possible. > > Kind regards, > Fokko Driesprong > > Op ma 22 jul 2024 om 14:37 schreef Piotr Findeisen < > piotr.findei...@gmail.com>: > >> Thanks for this lively discussion, it is great to see so many great >> people involved! >> >> We have unanimous agreement that we add support for JDK 21. >> Partial support (without spotless) will be added after 1.6.0 release is >> out (just not to mess up with the release). >> Full support (with spotless) will be added as soon as JDK 8 is dropped. >> For 21 we have clarity and we don't have preconditions, this is great. >> And this is a non-destructive operation too. >> >> We do not have agreement for dropping Hive module. This will be discussed >> separately on a new thread. >> >> We also seem to have unanimous agreement for dropping JDK 8. >> As to the timeline, it was proposed to do this in 2.0 release, so let's >> roll with this, unless there are new objections >> Since dropping support for something can be seen as a destructive >> operation, does it require a formal vote? >> Or do we treat +1 and -1 on this thread as votes already cast? >> >> Best, >> Piotr >> >> >> >> >> >> >> On Sat, 20 Jul 2024 at 00:12, Jack Ye <yezhao...@gmail.com> wrote: >> >>> +1 for dropping JDK8 support and adding JDK21. >>> >>> > What does dropping Java 8 support mean to companies that are still >>> using Java 8 for Iceberg in production? >>> >>> From the AWS side, AWS Corretto JDK8 end of life is July 2026, see: >>> https://aws.amazon.com/corretto/faqs/#support_calendar. I would suggest >>> at least migrate before that time. >>> >>> -Jack >>> >>> >>> >>> On Fri, Jul 19, 2024 at 3:02 PM John Zhuge <jzh...@apache.org> wrote: >>> >>>> +1 adding java 21 support >>>> +1 removing java 8 support >>>> >>>> On Fri, Jul 19, 2024 at 1:33 PM Daniel Weeks <dwe...@apache.org> wrote: >>>> >>>>> I'm also in favor of removing Java 8 support. Hive docs state Hive 3 >>>>> requires java 8 >>>>> <https://github.com/apache/hive?tab=readme-ov-file#java> and in prior >>>>> cases there were potential correctness issues when running with newer Java >>>>> versions (these may have been addressed). >>>>> >>>>> As long as we're not updating the target version, I think we should be >>>>> ok as they can still run in Java 8 if that remains a requirement. >>>>> >>>>> +1 to removing Java 8 support >>>>> +1 to adding Java 21 support. >>>>> >>>>> -Dan >>>>> >>>>> >>>>> >>>>> On Fri, Jul 19, 2024 at 1:04 PM Ryan Blue <b...@databricks.com.invalid> >>>>> wrote: >>>>> >>>>>> I agree that if we can separate the discussion about how to support >>>>>> Hive, then we should do that. >>>>>> >>>>>> +1 to removing Java 8 support >>>>>> +1 to adding Java 21 support. >>>>>> >>>>>> On Fri, Jul 19, 2024 at 12:58 PM huaxin gao <huaxin.ga...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> +1 in favor of adding java 21 support >>>>>>> +1 in favor of removing java 8 support >>>>>>> >>>>>>> I am currently working on Spark 4.0 / Iceberg integration >>>>>>> <https://github.com/apache/iceberg/pull/10622>. Spark 4.0 runs on >>>>>>> Java 17/21. >>>>>>> >>>>>>> On Fri, Jul 19, 2024 at 4:58 AM Piotr Findeisen < >>>>>>> piotr.findei...@gmail.com> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> We recently started to test Hive3 with Java 11 and 17 >>>>>>>> <https://github.com/apache/iceberg/pull/10482> and the tests pass. >>>>>>>> So dropping Java 8 doesn't technically require removing the Hive 3 >>>>>>>> related modules, unless users cannot do anything useful with them >>>>>>>> (because >>>>>>>> e.g. they can only run Hive runtime with Java 8 for some reason). >>>>>>>> Peter, can you please confirm this is not the case? >>>>>>>> Then it seems we could proceed with JDK 8 drop and discuss what to >>>>>>>> do with Hive modules *separately*. >>>>>>>> >>>>>>>> re original question of adding JDK 21 support -- we seem to have >>>>>>>> strong consensus to add it. >>>>>>>> Eduard plans to merge the PR once 1.6.0 is out. So I think we no >>>>>>>> longer need to debate this topic, unless there are any new objections >>>>>>>> to be >>>>>>>> raised. >>>>>>>> >>>>>>>> >>>>>>>> Best >>>>>>>> Piotr >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Fri, 19 Jul 2024 at 13:49, Péter Váry < >>>>>>>> peter.vary.apa...@gmail.com> wrote: >>>>>>>> >>>>>>>>> Back to the main topic: >>>>>>>>> >>>>>>>>> Removing java8 support still allows us to use the old HMS >>>>>>>>> libraries, so iceberg-hive-metastore module is not affected in this >>>>>>>>> sense. >>>>>>>>> We still need to run the tests, to ensure that there is no changes in >>>>>>>>> the >>>>>>>>> behaviour, but it is unlikely. >>>>>>>>> >>>>>>>>> +1 in favor of adding java 21 support >>>>>>>>> +1 in favor of removing java 8 support - especially if it is not >>>>>>>>> bound to removing Hive support as well >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Peter >>>>>>>>> >>>>>>>>> Cheng Pan <pan3...@gmail.com> ezt írta (időpont: 2024. júl. 19., >>>>>>>>> P, 4:56): >>>>>>>>> >>>>>>>>>> A basic question, is iceberg-hive-metastore considered part of the >>>>>>>>>> "Hive module"? >>>>>>>>>> >>>>>>>>>> I suppose that HMS 2.x is still widely used. AFAIK, the current >>>>>>>>>> iceberg-hive-metastore is compatible with HMS 2.1+, based on >>>>>>>>>> Iceberg >>>>>>>>>> and Spark CI, I also suppose it works well with Java 8 to 21. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Cheng Pan >>>>>>>>>> >>>>>>>>>> On Fri, Jul 19, 2024 at 4:44 AM Ryan Blue >>>>>>>>>> <b...@databricks.com.invalid> wrote: >>>>>>>>>> > >>>>>>>>>> > Thanks for the context, Denys and Peter. Sounds like there's a >>>>>>>>>> good question here about where the Hive integration should live and >>>>>>>>>> the >>>>>>>>>> most recent decision was to maintain that support in Hive. I >>>>>>>>>> definitely >>>>>>>>>> hear the point about Hive 3 users depending on the Iceberg modules. >>>>>>>>>> I'm >>>>>>>>>> also glad to hear that some of the issues are expected to be fixed >>>>>>>>>> with the >>>>>>>>>> release of Hive 4.0.x. >>>>>>>>>> > >>>>>>>>>> > I think that we have two separate questions for how to move >>>>>>>>>> forward with Hive support, depending on the Hive version. There is a >>>>>>>>>> question about what we do with the current Hive modules and what to >>>>>>>>>> do with >>>>>>>>>> the Hive 4 support that has been developed externally. >>>>>>>>>> > >>>>>>>>>> > For Hive 2.x and 3.x, we have code in the Iceberg repo that is >>>>>>>>>> not being developed. Hive 2 is fairly easy since it is EOL. While >>>>>>>>>> Hive 3 is >>>>>>>>>> still used, I don't think it makes sense to keep releasing versions >>>>>>>>>> of it >>>>>>>>>> if it requires Java 8, which has not been publicly maintained for 5 >>>>>>>>>> years. >>>>>>>>>> We need to upgrade and that is at odds with keeping support for Hive >>>>>>>>>> 3. As >>>>>>>>>> Fokko and I both pointed out, people can still use older releases. >>>>>>>>>> > >>>>>>>>>> > For the question of how to maintain support for Hive 4, I think >>>>>>>>>> it's worth having a separate discussion (probably not on the thread >>>>>>>>>> about >>>>>>>>>> JDK versions) about where to maintain it. I think that it is best to >>>>>>>>>> maintain integration in engines and not in the Iceberg project; >>>>>>>>>> there are >>>>>>>>>> few implementations here and I think that it is a hard problem for >>>>>>>>>> Iceberg >>>>>>>>>> to maintain support for multiple versions (as you can see with >>>>>>>>>> support for >>>>>>>>>> so many different Flink, Hive, and Spark versions). >>>>>>>>>> > >>>>>>>>>> > Ryan >>>>>>>>>> > >>>>>>>>>> > On Thu, Jul 18, 2024 at 7:25 AM Denys Kuzmenko < >>>>>>>>>> dkuzme...@apache.org> wrote: >>>>>>>>>> >> >>>>>>>>>> >> In the following 1-2 months we plan to release HIVE-4.0.1 >>>>>>>>>> which includes bug fixes and then focus on HIVE-4.1.0 release with >>>>>>>>>> jdk17. >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > -- >>>>>>>>>> > Ryan Blue >>>>>>>>>> > Databricks >>>>>>>>>> >>>>>>>>> >>>>>> >>>>>> -- >>>>>> Ryan Blue >>>>>> Databricks >>>>>> >>>>> >>>> >>>> -- >>>> John Zhuge >>>> >>>
