I'm also in favor of removing Java 8 support. Hive docs state Hive 3 requires java 8 <https://github.com/apache/hive?tab=readme-ov-file#java> and in prior cases there were potential correctness issues when running with newer Java versions (these may have been addressed).
As long as we're not updating the target version, I think we should be ok as they can still run in Java 8 if that remains a requirement. +1 to removing Java 8 support +1 to adding Java 21 support. -Dan On Fri, Jul 19, 2024 at 1:04 PM Ryan Blue <b...@databricks.com.invalid> wrote: > I agree that if we can separate the discussion about how to support Hive, > then we should do that. > > +1 to removing Java 8 support > +1 to adding Java 21 support. > > On Fri, Jul 19, 2024 at 12:58 PM huaxin gao <huaxin.ga...@gmail.com> > wrote: > >> +1 in favor of adding java 21 support >> +1 in favor of removing java 8 support >> >> I am currently working on Spark 4.0 / Iceberg integration >> <https://github.com/apache/iceberg/pull/10622>. Spark 4.0 runs on Java >> 17/21. >> >> On Fri, Jul 19, 2024 at 4:58 AM Piotr Findeisen < >> piotr.findei...@gmail.com> wrote: >> >>> Hi, >>> >>> We recently started to test Hive3 with Java 11 and 17 >>> <https://github.com/apache/iceberg/pull/10482> and the tests pass. >>> So dropping Java 8 doesn't technically require removing the Hive 3 >>> related modules, unless users cannot do anything useful with them (because >>> e.g. they can only run Hive runtime with Java 8 for some reason). >>> Peter, can you please confirm this is not the case? >>> Then it seems we could proceed with JDK 8 drop and discuss what to do >>> with Hive modules *separately*. >>> >>> re original question of adding JDK 21 support -- we seem to have strong >>> consensus to add it. >>> Eduard plans to merge the PR once 1.6.0 is out. So I think we no longer >>> need to debate this topic, unless there are any new objections to be raised. >>> >>> >>> Best >>> Piotr >>> >>> >>> >>> >>> >>> >>> >>> On Fri, 19 Jul 2024 at 13:49, Péter Váry <peter.vary.apa...@gmail.com> >>> wrote: >>> >>>> Back to the main topic: >>>> >>>> Removing java8 support still allows us to use the old HMS libraries, so >>>> iceberg-hive-metastore module is not affected in this sense. We still need >>>> to run the tests, to ensure that there is no changes in the behaviour, but >>>> it is unlikely. >>>> >>>> +1 in favor of adding java 21 support >>>> +1 in favor of removing java 8 support - especially if it is not bound >>>> to removing Hive support as well >>>> >>>> Thanks, >>>> Peter >>>> >>>> Cheng Pan <pan3...@gmail.com> ezt írta (időpont: 2024. júl. 19., P, >>>> 4:56): >>>> >>>>> A basic question, is iceberg-hive-metastore considered part of the >>>>> "Hive module"? >>>>> >>>>> I suppose that HMS 2.x is still widely used. AFAIK, the current >>>>> iceberg-hive-metastore is compatible with HMS 2.1+, based on Iceberg >>>>> and Spark CI, I also suppose it works well with Java 8 to 21. >>>>> >>>>> Thanks, >>>>> Cheng Pan >>>>> >>>>> On Fri, Jul 19, 2024 at 4:44 AM Ryan Blue <b...@databricks.com.invalid> >>>>> wrote: >>>>> > >>>>> > Thanks for the context, Denys and Peter. Sounds like there's a good >>>>> question here about where the Hive integration should live and the most >>>>> recent decision was to maintain that support in Hive. I definitely hear >>>>> the >>>>> point about Hive 3 users depending on the Iceberg modules. I'm also glad >>>>> to >>>>> hear that some of the issues are expected to be fixed with the release of >>>>> Hive 4.0.x. >>>>> > >>>>> > I think that we have two separate questions for how to move forward >>>>> with Hive support, depending on the Hive version. There is a question >>>>> about >>>>> what we do with the current Hive modules and what to do with the Hive 4 >>>>> support that has been developed externally. >>>>> > >>>>> > For Hive 2.x and 3.x, we have code in the Iceberg repo that is not >>>>> being developed. Hive 2 is fairly easy since it is EOL. While Hive 3 is >>>>> still used, I don't think it makes sense to keep releasing versions of it >>>>> if it requires Java 8, which has not been publicly maintained for 5 years. >>>>> We need to upgrade and that is at odds with keeping support for Hive 3. As >>>>> Fokko and I both pointed out, people can still use older releases. >>>>> > >>>>> > For the question of how to maintain support for Hive 4, I think it's >>>>> worth having a separate discussion (probably not on the thread about JDK >>>>> versions) about where to maintain it. I think that it is best to maintain >>>>> integration in engines and not in the Iceberg project; there are few >>>>> implementations here and I think that it is a hard problem for Iceberg to >>>>> maintain support for multiple versions (as you can see with support for so >>>>> many different Flink, Hive, and Spark versions). >>>>> > >>>>> > Ryan >>>>> > >>>>> > On Thu, Jul 18, 2024 at 7:25 AM Denys Kuzmenko <dkuzme...@apache.org> >>>>> wrote: >>>>> >> >>>>> >> In the following 1-2 months we plan to release HIVE-4.0.1 which >>>>> includes bug fixes and then focus on HIVE-4.1.0 release with jdk17. >>>>> > >>>>> > >>>>> > >>>>> > -- >>>>> > Ryan Blue >>>>> > Databricks >>>>> >>>> > > -- > Ryan Blue > Databricks >
