Thanks Piotr for driving this, late +1 to add JDK 21 support and your plan for spotless.
It seems ok to me too to bite the bullet and move to newer spotless (disabling spotless for JDK8 builds) post 1.6, but looks like the discussion happened and I'm fine either way. Thanks! Szehon On Mon, Jul 22, 2024 at 6:38 AM Piotr Findeisen <piotr.findei...@gmail.com> wrote: > Thanks Fokko. > I like the idea. Started a new "Dropping JDK 8 support" thread to ensure > transparency. > > Best > Piotr > > > On Mon, 22 Jul 2024 at 15:24, Fokko Driesprong <fo...@apache.org> wrote: > >> Thanks for summarizing this, Piotr. >> >> I believe having a separate thread on dropping Java 8 is the right thing >> to do. We want to be as transparent about these changes as possible. >> >> Kind regards, >> Fokko Driesprong >> >> Op ma 22 jul 2024 om 14:37 schreef Piotr Findeisen < >> piotr.findei...@gmail.com>: >> >>> Thanks for this lively discussion, it is great to see so many great >>> people involved! >>> >>> We have unanimous agreement that we add support for JDK 21. >>> Partial support (without spotless) will be added after 1.6.0 release is >>> out (just not to mess up with the release). >>> Full support (with spotless) will be added as soon as JDK 8 is dropped. >>> For 21 we have clarity and we don't have preconditions, this is great. >>> And this is a non-destructive operation too. >>> >>> We do not have agreement for dropping Hive module. This will be >>> discussed separately on a new thread. >>> >>> We also seem to have unanimous agreement for dropping JDK 8. >>> As to the timeline, it was proposed to do this in 2.0 release, so let's >>> roll with this, unless there are new objections >>> Since dropping support for something can be seen as a destructive >>> operation, does it require a formal vote? >>> Or do we treat +1 and -1 on this thread as votes already cast? >>> >>> Best, >>> Piotr >>> >>> >>> >>> >>> >>> >>> On Sat, 20 Jul 2024 at 00:12, Jack Ye <yezhao...@gmail.com> wrote: >>> >>>> +1 for dropping JDK8 support and adding JDK21. >>>> >>>> > What does dropping Java 8 support mean to companies that are still >>>> using Java 8 for Iceberg in production? >>>> >>>> From the AWS side, AWS Corretto JDK8 end of life is July 2026, see: >>>> https://aws.amazon.com/corretto/faqs/#support_calendar. I would >>>> suggest at least migrate before that time. >>>> >>>> -Jack >>>> >>>> >>>> >>>> On Fri, Jul 19, 2024 at 3:02 PM John Zhuge <jzh...@apache.org> wrote: >>>> >>>>> +1 adding java 21 support >>>>> +1 removing java 8 support >>>>> >>>>> On Fri, Jul 19, 2024 at 1:33 PM Daniel Weeks <dwe...@apache.org> >>>>> wrote: >>>>> >>>>>> I'm also in favor of removing Java 8 support. Hive docs state Hive >>>>>> 3 requires java 8 >>>>>> <https://github.com/apache/hive?tab=readme-ov-file#java> and in >>>>>> prior cases there were potential correctness issues when running with >>>>>> newer >>>>>> Java versions (these may have been addressed). >>>>>> >>>>>> As long as we're not updating the target version, I think we should >>>>>> be ok as they can still run in Java 8 if that remains a requirement. >>>>>> >>>>>> +1 to removing Java 8 support >>>>>> +1 to adding Java 21 support. >>>>>> >>>>>> -Dan >>>>>> >>>>>> >>>>>> >>>>>> On Fri, Jul 19, 2024 at 1:04 PM Ryan Blue <b...@databricks.com.invalid> >>>>>> wrote: >>>>>> >>>>>>> I agree that if we can separate the discussion about how to support >>>>>>> Hive, then we should do that. >>>>>>> >>>>>>> +1 to removing Java 8 support >>>>>>> +1 to adding Java 21 support. >>>>>>> >>>>>>> On Fri, Jul 19, 2024 at 12:58 PM huaxin gao <huaxin.ga...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> +1 in favor of adding java 21 support >>>>>>>> +1 in favor of removing java 8 support >>>>>>>> >>>>>>>> I am currently working on Spark 4.0 / Iceberg integration >>>>>>>> <https://github.com/apache/iceberg/pull/10622>. Spark 4.0 runs on >>>>>>>> Java 17/21. >>>>>>>> >>>>>>>> On Fri, Jul 19, 2024 at 4:58 AM Piotr Findeisen < >>>>>>>> piotr.findei...@gmail.com> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> We recently started to test Hive3 with Java 11 and 17 >>>>>>>>> <https://github.com/apache/iceberg/pull/10482> and the tests pass. >>>>>>>>> So dropping Java 8 doesn't technically require removing the Hive 3 >>>>>>>>> related modules, unless users cannot do anything useful with them >>>>>>>>> (because >>>>>>>>> e.g. they can only run Hive runtime with Java 8 for some reason). >>>>>>>>> Peter, can you please confirm this is not the case? >>>>>>>>> Then it seems we could proceed with JDK 8 drop and discuss what to >>>>>>>>> do with Hive modules *separately*. >>>>>>>>> >>>>>>>>> re original question of adding JDK 21 support -- we seem to have >>>>>>>>> strong consensus to add it. >>>>>>>>> Eduard plans to merge the PR once 1.6.0 is out. So I think we no >>>>>>>>> longer need to debate this topic, unless there are any new objections >>>>>>>>> to be >>>>>>>>> raised. >>>>>>>>> >>>>>>>>> >>>>>>>>> Best >>>>>>>>> Piotr >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, 19 Jul 2024 at 13:49, Péter Váry < >>>>>>>>> peter.vary.apa...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Back to the main topic: >>>>>>>>>> >>>>>>>>>> Removing java8 support still allows us to use the old HMS >>>>>>>>>> libraries, so iceberg-hive-metastore module is not affected in this >>>>>>>>>> sense. >>>>>>>>>> We still need to run the tests, to ensure that there is no changes >>>>>>>>>> in the >>>>>>>>>> behaviour, but it is unlikely. >>>>>>>>>> >>>>>>>>>> +1 in favor of adding java 21 support >>>>>>>>>> +1 in favor of removing java 8 support - especially if it is not >>>>>>>>>> bound to removing Hive support as well >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Peter >>>>>>>>>> >>>>>>>>>> Cheng Pan <pan3...@gmail.com> ezt írta (időpont: 2024. júl. 19., >>>>>>>>>> P, 4:56): >>>>>>>>>> >>>>>>>>>>> A basic question, is iceberg-hive-metastore considered part of >>>>>>>>>>> the >>>>>>>>>>> "Hive module"? >>>>>>>>>>> >>>>>>>>>>> I suppose that HMS 2.x is still widely used. AFAIK, the current >>>>>>>>>>> iceberg-hive-metastore is compatible with HMS 2.1+, based on >>>>>>>>>>> Iceberg >>>>>>>>>>> and Spark CI, I also suppose it works well with Java 8 to 21. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Cheng Pan >>>>>>>>>>> >>>>>>>>>>> On Fri, Jul 19, 2024 at 4:44 AM Ryan Blue >>>>>>>>>>> <b...@databricks.com.invalid> wrote: >>>>>>>>>>> > >>>>>>>>>>> > Thanks for the context, Denys and Peter. Sounds like there's a >>>>>>>>>>> good question here about where the Hive integration should live and >>>>>>>>>>> the >>>>>>>>>>> most recent decision was to maintain that support in Hive. I >>>>>>>>>>> definitely >>>>>>>>>>> hear the point about Hive 3 users depending on the Iceberg modules. >>>>>>>>>>> I'm >>>>>>>>>>> also glad to hear that some of the issues are expected to be fixed >>>>>>>>>>> with the >>>>>>>>>>> release of Hive 4.0.x. >>>>>>>>>>> > >>>>>>>>>>> > I think that we have two separate questions for how to move >>>>>>>>>>> forward with Hive support, depending on the Hive version. There is a >>>>>>>>>>> question about what we do with the current Hive modules and what to >>>>>>>>>>> do with >>>>>>>>>>> the Hive 4 support that has been developed externally. >>>>>>>>>>> > >>>>>>>>>>> > For Hive 2.x and 3.x, we have code in the Iceberg repo that is >>>>>>>>>>> not being developed. Hive 2 is fairly easy since it is EOL. While >>>>>>>>>>> Hive 3 is >>>>>>>>>>> still used, I don't think it makes sense to keep releasing versions >>>>>>>>>>> of it >>>>>>>>>>> if it requires Java 8, which has not been publicly maintained for 5 >>>>>>>>>>> years. >>>>>>>>>>> We need to upgrade and that is at odds with keeping support for >>>>>>>>>>> Hive 3. As >>>>>>>>>>> Fokko and I both pointed out, people can still use older releases. >>>>>>>>>>> > >>>>>>>>>>> > For the question of how to maintain support for Hive 4, I >>>>>>>>>>> think it's worth having a separate discussion (probably not on the >>>>>>>>>>> thread >>>>>>>>>>> about JDK versions) about where to maintain it. I think that it is >>>>>>>>>>> best to >>>>>>>>>>> maintain integration in engines and not in the Iceberg project; >>>>>>>>>>> there are >>>>>>>>>>> few implementations here and I think that it is a hard problem for >>>>>>>>>>> Iceberg >>>>>>>>>>> to maintain support for multiple versions (as you can see with >>>>>>>>>>> support for >>>>>>>>>>> so many different Flink, Hive, and Spark versions). >>>>>>>>>>> > >>>>>>>>>>> > Ryan >>>>>>>>>>> > >>>>>>>>>>> > On Thu, Jul 18, 2024 at 7:25 AM Denys Kuzmenko < >>>>>>>>>>> dkuzme...@apache.org> wrote: >>>>>>>>>>> >> >>>>>>>>>>> >> In the following 1-2 months we plan to release HIVE-4.0.1 >>>>>>>>>>> which includes bug fixes and then focus on HIVE-4.1.0 release with >>>>>>>>>>> jdk17. >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > -- >>>>>>>>>>> > Ryan Blue >>>>>>>>>>> > Databricks >>>>>>>>>>> >>>>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Ryan Blue >>>>>>> Databricks >>>>>>> >>>>>> >>>>> >>>>> -- >>>>> John Zhuge >>>>> >>>>
