Moving to a newer JDK is a major milestone for the project and I am really thankful to everyone pushing this effort forward. There are always small hurdles and delays when pushing a big project forward so no one should feel bad or apologize when things do not go exactly as planned.
Most projects that are jumping to a new JDK usually do a major version bump and I think we should follow that paradigm and release Hive with JDK17 as Hive 5.0.0. Our last major version was released ~1 year ago so in terms of timing having 1 major release every year or so sounds reasonable. On a more general topic, having more frequent releases has numerous benefits so we should re-discuss and adopt a scheme that can allow us to move into this direction. This is independent of the JDK upgrade so let's discuss it on a separate thread. Best, Stamatis On Tue, Apr 8, 2025, 4:27 PM Butao Zhang <zhangbu...@apache.org> wrote: > The upgrade to JDK 17 is a significant change that introduces > compatibility and adaptation requirements for both Apache Tez and Apache > Hive. Relevant work is currently in progress—thanks to all community > contributors who have already been involved in adaptation and fixes. We > encourage everyone to actively report any issues encountered during usage > or integration, so that we can collaborate on discussions and resolutions, > and accelerate the overall transition to JDK 17. > > Thanks, > Butao Zhang > > On 2025/04/07 11:25:33 Shohei Okumiya wrote: > > Hi Ayush and all people working on the JDK upgrade, > > > > Thanks for sharing the timeline and detailing the current progress. I'm > > more than happy to hear the good news and look forward to seeing Hive > > running on Java 17! > > > > Mid-May is not so far away. I don't have a firm preference about how to > > package 4.1. If anyone is interested, let's discuss it in the thread for > > the Hive 4.1.x release. > > > > Finally, let me express my gratitude to everyone working on the > challenging > > upgrade project. It will definitely be one of the most outstanding > > achievements! > > > > Sincerely, > > Okumin > > > > On Mon, Apr 7, 2025 at 7:03 PM Ayush Saxena <ayush...@gmail.com> wrote: > > > > > Hi Okumin, > > > > > > I know I'm one of the people responsible for misjudging the timeline > > > around the JDK-17 upgrade, which has ultimately blocked the 4.1.0 > release. > > > I want to take a moment to share my thoughts and where things currently > > > stand. > > > > > > Q: Can we confirm the importance and ETA of the JDK upgrade and how we > > > should proceed? > > > > > > Ans: I think we all agree that moving to JDK-17 is a major milestone, > and > > > we're admittedly late in adopting it. Many projects have already moved > or > > > are actively working towards it. Not being on JDK-17 restricts us from > > > adopting modern third-party libraries like Iceberg and from leveraging > > > several improvements that come with it. > > > > > > As for the ETA: I realize I previously gave an inaccurate timeline, > and I > > > sincerely regret that. However, I now have a much clearer picture. I'm > in > > > active discussions with folks internally who are working on it. I > believe > > > we should have the code ready by mid-May, at which point we can start > > > gathering feedback from the community. > > > > > > Q: We've been working on this upgrade for over half a year. When will > it > > > merge? PR: https://github.com/apache/hive/pull/5404 > > > In my opinion, #5404 should be divided into multiple pull requests. I > > > really understand we want to test simultaneously rather than make > > > incremental progress. However, the consolidated PR makes it impossible > for > > > other developers to help the initiative, reviewers have to check huge > > > changes, and authors must occasionally resolve many conflicts. I don't > > > prefer any HOW as long as the delivery is predictable, meaning it is > okay > > > for me if we are confident with the agreed ETA. If it is still > uncertain, > > > we may try another project management, which is > incremental/predictable and > > > where we can invest other developers' resources in it. > > > > > > Ans: I completely agree with your observations. The original PR started > > > more as an experiment to see what changes were needed. At the time, I > don’t > > > think anyone anticipated how large it would become, and unfortunately, > it > > > kept growing incrementally over time. > > > That said, I’ve already asked the team to start breaking down the PR > into > > > smaller, more manageable parts. Dependency upgrades will be separated > out > > > soon, and changes like those related to SpotBugs (which account for a > large > > > number of line changes) will also be split. The goal is to make the PR > > > easier to review and more accessible for other contributors to jump > in. I > > > understand this is a genuine blocker, and folks are doing their best > to > > > address it. > > > > > > Q: [1] Should we ship 4.1 without JDK-17, given it's been a long time > > > since 4.0.1? > > > > > > Ans: Honestly, I’m not in a position to oppose this. The community > kindly > > > respected my request last time when I asked for more time for the > JDK-17 > > > upgrade, and unfortunately, I couldn’t follow through as expected. So, > with > > > sincere apologies for being one of the reasons for the delay, I’m okay > with > > > whatever direction the community chooses to take from here. > > > > > > -Ayush > > > > > > > > > On Sun, 6 Apr 2025 at 11:55, Shohei Okumiya <oku...@apache.org> wrote: > > > > > >> Hi, > > >> > > >> I appreciate everyone working on this big and challenging project! > > >> > > >> Can we confirm the importance and ETA of the JDK upgrade and how we > > >> should work on it? > > >> > > >> I recognize it is one of the most essential initiatives blocking the > > >> release of Hive 4.1.0[1] and the upgrade of Iceberg libraries[2]. > Users and > > >> developers would expect it to be shipped soon. > > >> > > >> We have to complete this effort to ship JDK 17. We've been working on > it > > >> for over half a year. Some of us might be interested in when we will > merge > > >> it. > > >> https://github.com/apache/hive/pull/5404 > > >> > > >> In my opinion, #5404 should be divided into multiple pull requests. I > > >> really understand we want to test simultaneously rather than make > > >> incremental progress. However, the consolidated PR makes it > impossible for > > >> other developers to help the initiative, reviewers have to check huge > > >> changes, and authors must occasionally resolve many conflicts. I don't > > >> prefer any HOW as long as the delivery is predictable, meaning it is > okay > > >> for me if we are confident with the agreed ETA. If it is still > uncertain, > > >> we may try another project management, which is > incremental/predictable and > > >> where we can invest other developers' resources in it. > > >> > > >> - [1] Honestly, I feel we should ship 4.1 without JDK 17, as it's > been a > > >> long time since the last release of 4.0.1 > > >> - [2] Iceberg 1.7 or later doesn't support Java 8 > > >> > > >> Best, > > >> Okumin > > >> > > >> On Wed, Dec 11, 2024 at 2:51 AM Ayush Saxena <ayush...@gmail.com> > wrote: > > >> > > >>> Regarding JDK-11, we didn't actively chased that, so its status > remains > > >>> uncertain. With lower compile-time support like JDK-8, we can > certainly > > >>> ensure runtime support for both JDK-11 and JDK-17, which I believe > should > > >>> work for Hive 4.0.1. As far as I know, this approach works > downstream for > > >>> us atleast. > > >>> > > >>> However, maintaining compile-time compatibility across JDK-17, > JDK-11, > > >>> and JDK-8 simultaneously may not be feasible. JDK-17 introduces > significant > > >>> changes, and I don’t believe it’s naive—or even possible—to compile > Hive > > >>> using both JDK-8 and JDK-17 & expect things to work properly. > > >>> > > >>> The initial plan appeared to be supporting JDK-8 for Hive 4.0.x > release > > >>> line, with potential tweaks if required to enable runtime support for > > >>> JDK-17. Starting with Hive 4.1.x, we could shift to JDK-17 for > compile-time > > >>> support, and later evaluate the feasibility of supporting JDK-21 or > newer > > >>> versions. > > >>> > > >>> Once we see a state when nobody want JDK-8 compile time support & > > >>> everyone has moved to JDK-17 or say using 4.1.x release line, We can > then > > >>> drop having bug fix releases for 4.0.x > > >>> > > >>> -Ayush > > >>> > > >>> On Mon, 9 Dec 2024 at 15:11, Butao Zhang <butaozha...@163.com> > wrote: > > >>> > > >>>> IMO, I think there two challenges if we want to maintain JDK8 & > JDK11 & > > >>>> JDK17 at the same time. > > >>>> > > >>>> - Different jdk syntactic sugar between multiple jdks, and > advanced > > >>>> feats in jdk17&jdk21, such as jdk vector api. If we want the > multiple jdks, > > >>>> we can not use the advanced feats in high jdk in the future. > Such as the > > >>>> discussion in https://issues.apache.org/jira/browse/HIVE-28614 > > >>>> - We need to run the jenkins CI process multiple times to run the > > >>>> multiple jdk env. I'm not sure we have enough jenkins resources > to do this. > > >>>> However, i want to say if we can use github actions to run the > whole CI? I > > >>>> have seen Apache Spark have done the good job which using the > forked > > >>>> repository github actions to run the CI. FRY > > >>>> https://lists.apache.org/thread/dmv1h2snqc7qtdjlzj4k8qxookclp720 > > >>>> > > >>>> > > >>>> Thanks, > > >>>> Butao Zhang > > >>>> ---- Replied Message ---- > > >>>> From Shohei Okumiya<oku...@apache.org> <undefined> > > >>>> Date 12/9/2024 17:06 > > >>>> To <dev@hive.apache.org> <dev@hive.apache.org> > > >>>> Subject Re: Move to JDK-11 > > >>>> Hi, > > >>>> > > >>>> I appreciate everyone's work on the JDK upgrade. It would be ideal > if > > >>>> we could migrate JDK like this. That will give us an > > >>>> incremental migration path. > > >>>> > > >>>> 1. (now) Hive 4.0 compiled by JDK 8, and run by JDK 8 > > >>>> 2. Hive 4.x compiled by JDK 8, and run by JDK 17 > > >>>> 3. Hive 4.x compiled by JDK 17, and run by JDK 17 > > >>>> > > >>>> I guess it is technically possible that Hive supports JDK 8 and 17 > at > > >>>> runtime. Apache Spark supports JDK 8, 11, and 17. What is the most > > >>>> challenging point? > > >>>> > > >>>> Regards, > > >>>> Okumin > > >>>> > > >>>> On Thu, Aug 29, 2024 at 10:48 PM Butao Zhang <butaozha...@163.com> > > >>>> wrote: > > >>>> > > >>>>> Hi Akshat, > > >>>>> > > >>>>> Thanks for your reply! I agree that supporting both JDK 8 and JDK > 17 > > >>>>> is not easy. I think we can safely to move to JDK 17 if we can > ensure *hive > > >>>>> application* can access to HiveServer2/Metastore without forcing > to > > >>>>> use the JDK 17. > > >>>>> For example, if a hive application is developed by JDK 8, it can > > >>>>> import the *hive-exec.jar & hive-jdbc.jar* which is compiled by JDK > > >>>>> 17 and can access to hs2/hms successfully. If this case can be > supported, > > >>>>> I think we can move to JDK 17 as soon as possible. > > >>>>> > > >>>>> BTW, If Apache Hive , Apache Tez and Apache Hadoop are all compiled > > >>>>> with JDK17 or higer version, i think the hive job will run more > faster. > > >>>>> Users will enjoy significant performance improvements due to JDK > version > > >>>>> upgrades. > > >>>>> I'm looking forward to the performance improvement. :) > > >>>>> > > >>>>> Thanks, > > >>>>> Butao Zhang > > >>>>> ---- Replied Message ---- > > >>>>> From Akshat m<akshatats...@gmail.com> <undefined> > > >>>>> Date 8/29/2024 18:44 > > >>>>> To <dev@hive.apache.org> <dev@hive.apache.org> > > >>>>> Subject Re: Move to JDK-11 > > >>>>> Hi Butao, > > >>>>> > > >>>>> It won’t be feasible to support both JDK 8 and JDK 17 in the same > Hive > > >>>>> version due to compatibility issues and behavior changes, as > detailed in > > >>>>> the PR. The internal changes in JDK implementations have made this > > >>>>> challenging. Additionally, many other libraries maintain separate > versions > > >>>>> for JDK 17+ and JDK 8 support. > > >>>>> > > >>>>> To align with this approach, I suggest we continue with Hive 4 > > >>>>> supporting JDK 8. Once the JDK 17 PR is merged, we could plan a > separate > > >>>>> release that specifically targets JDK 17+ support. This would > ensure that > > >>>>> Hive stays up-to-date while maintaining stability across different > > >>>>> environments. > > >>>>> > > >>>>> Would love to hear your thoughts or any additional suggestions the > > >>>>> community might have! > > >>>>> > > >>>>> Regards > > >>>>> > > >>>>> Akshat > > >>>>> > > >>>>> > > >>>>> > > >>>>> On Wed, Aug 28, 2024 at 8:00 AM Butao Zhang <butaozha...@163.com> > > >>>>> wrote: > > >>>>> > > >>>>>> Good job! Thanks everyone who put effort into the JDK upgrading. > > >>>>>> That's a big good news for Apache Hive community! > > >>>>>> Just a question: Given that many people are still using the lower > > >>>>>> jdk8 version. Do we consider to keeping compatibility with lower > jdk8 > > >>>>>> version? > > >>>>>> > > >>>>>> > > >>>>>> Thanks, > > >>>>>> Butao Zhang > > >>>>>> ---- Replied Message ---- > > >>>>>> From Akshat m<akshatats...@gmail.com> <undefined> > > >>>>>> Date 8/27/2024 23:12 > > >>>>>> To dev<dev@hive.apache.org> <dev@hive.apache.org> > > >>>>>> Cc kokila narayanan<kokilanarayana...@gmail.com>, > > >>>>>> <kokilanarayana...@gmail.com>Raghav > > >>>>>> Aggarwal<raghavaggarwal03...@gmail.com>, > > >>>>>> <raghavaggarwal03...@gmail.com>Ayush Saxena<ayush...@gmail.com>, > > >>>>>> <ayush...@gmail.com>tanishq chugh<tanishqchugh4...@gmail.com>, > > >>>>>> <tanishqchugh4...@gmail.com><aturo...@cloudera.com> > > >>>>>> <aturo...@cloudera.com> > > >>>>>> Subject Re: Move to JDK-11 > > >>>>>> > > >>>>>> Hi everyone, > > >>>>>> > > >>>>>> I’m excited to share that after months of dedicated work, we’ve > > >>>>>> finally achieved a passing PR[1] for running *Apache Hive on JDK > 17*! > > >>>>>> > > >>>>>> This was a challenging task, requiring to address various > > >>>>>> compatibility issues and ensure our comprehensive test suite > passed without > > >>>>>> a hitch. > > >>>>>> > > >>>>>> Thanks Kokila and Tanishq for joining me in the initiative and > Raghav > > >>>>>> for contributing fix for the errorProne issue, and thanks to > Attila, Ayush > > >>>>>> and other Hive PMC members for their guidance in the initiative > > >>>>>> > > >>>>>> It’s been a real team effort, and I’m grateful for the > collaboration > > >>>>>> and support from everyone involved. > > >>>>>> > > >>>>>> I’d appreciate it if you could take some time to review the PR. > > >>>>>> Community feedback is really important to make this happen. > > >>>>>> > > >>>>>> The PR #5404 addresses the integration of Apache Hive with JDK 17, > > >>>>>> introducing a range of changes aimed at ensuring compatibility and > > >>>>>> improving the codebase. Here’s a detailed breakdown: > > >>>>>> > > >>>>>> 1. Core Code Adjustments: > > >>>>>> > > >>>>>> - Compatibility Fixes: Various parts of the Hive codebase have > > >>>>>> been modified to ensure they work seamlessly with JDK 17. This > includes > > >>>>>> addressing language or API changes that JDK 17 introduces. > > >>>>>> > > >>>>>> - Reflection and Modules: Adjustments related to how Hive > > >>>>>> interacts with Java's reflection API, which has stricter module > system > > >>>>>> rules in JDK 17. > > >>>>>> > > >>>>>> 2. Dependency Management: > > >>>>>> > > >>>>>> - Version Upgrades: Several third-party dependencies have been > > >>>>>> updated to versions that are compatible with JDK 17. This is > critical > > >>>>>> because older versions of some libraries may not work properly or > might use > > >>>>>> deprecated features removed in JDK 17. > > >>>>>> > > >>>>>> - Data nucleus upgrade: To ensure compatibility with Java 17, > we > > >>>>>> upgraded DataNucleus to version 6.x. This upgrade, however, > introduced an > > >>>>>> issue with Hive schema creation. The issue stems from a commit in > > >>>>>> DataNucleus[2] that alters the handling of primary key column > names by > > >>>>>> enclosing them in double quotes. This change caused conflicts > during Hive > > >>>>>> schema generation. As an interim solution, we implemented a > patched > > >>>>>> DataNucleus-rdbms module to address the issue in Hive. The > necessary fix is > > >>>>>> now included in the DataNucleus repository[3]. Once DataNucleus > 6.0.8 is > > >>>>>> officially released, we will upgrade to this version and remove > the > > >>>>>> temporary patched module. > > >>>>>> > > >>>>>> - Dependency Cleanup: Unused or outdated dependencies were > removed, > > >>>>>> reducing potential conflicts and simplifying the build process. > > >>>>>> > > >>>>>> 3. Testing Enhancements: > > >>>>>> > > >>>>>> - Test Suite Updates: The PR includes updates to Hive’s test > suite > > >>>>>> to ensure all tests pass under JDK 17. This likely involved > modifying tests > > >>>>>> that needed adjustments to align with JDK 17’s behaviour. > > >>>>>> > > >>>>>> - New Tests: New tests may have been added to cover areas where > JDK > > >>>>>> 17 introduced new features or changes that could impact Hive. > > >>>>>> > > >>>>>> 4. Compatibility Checks: > > >>>>>> > > >>>>>> - JDK Version Verification: A new compatibility check has been > > >>>>>> introduced to verify the JDK version at runtime. This ensures > that Hive is > > >>>>>> running on an appropriate JDK version, preventing unexpected > errors or > > >>>>>> behaviour due to version mismatches. > > >>>>>> > > >>>>>> 5. Build Updates: > > >>>>>> > > >>>>>> - Build System Tweaks: The PR likely includes changes to the > Maven > > >>>>>> build configuration to accommodate JDK 17, ensuring that the > build process > > >>>>>> is smooth and compatible. > > >>>>>> > > >>>>>> Overall, this PR is a comprehensive effort to bring Apache Hive > up to > > >>>>>> date with the latest Java platform, ensuring long-term > maintainability and > > >>>>>> performance improvements. > > >>>>>> > > >>>>>> As the next steps, we’ll be focusing on further sanity checks and > > >>>>>> extensive testing to make sure everything functions smoothly. > Community > > >>>>>> involvement in this process will be crucial, so please do share > any issues > > >>>>>> or feedback as you test the changes. > > >>>>>> > > >>>>>> I’m looking forward to seeing the benefits this will bring to our > > >>>>>> community and users. > > >>>>>> > > >>>>>> > > >>>>>> [1] https://github.com/apache/hive/pull/5404 > > >>>>>> > > >>>>>> [2] > > >>>>>> > https://github.com/datanucleus/datanucleus-rdbms/commit/b99df3c803862f9e9feacc7891461bcb21160b38 > > >>>>>> > > >>>>>> [3] > > >>>>>> > https://github.com/datanucleus/datanucleus-rdbms/commit/b99df3c803862f9e9feacc7891461bcb21160b38 > > >>>>>> > > >>>>>> > > >>>>>> Best regards, > > >>>>>> > > >>>>>> Akshat Mathur > > >>>>>> > > >>>>>> > > >>>>>> On Tue, 11 Jul 2023 at 2:25 PM, Attila Turoczy < > > >>>>>> aturo...@cloudera.com.invalid> wrote: > > >>>>>> > > >>>>>>> Returning to this topic, I kindly request those who would like to > > >>>>>>> advocate > > >>>>>>> for the continued support of JDK8 to please share their > reasoning and > > >>>>>>> insights with us. Your input and perspective are greatly > appreciated! > > >>>>>>> Thank you. > > >>>>>>> > > >>>>>>> -Attila > > >>>>>>> > > >>>>>>> On Fri, Jun 2, 2023 at 12:43 PM Attila Turoczy < > > >>>>>>> aturo...@cloudera.com> > > >>>>>>> wrote: > > >>>>>>> > > >>>>>>> > Hi All, > > >>>>>>> > > > >>>>>>> > I know my opinion might not be the most popular, but I advocate > > >>>>>>> for using* > > >>>>>>> > JDK 17*. Here's why: > > >>>>>>> > > > >>>>>>> > Let's consider a scenario where a customer wants to use the > latest > > >>>>>>> version > > >>>>>>> > of Apache Hive. They would typically install it locally or on a > > >>>>>>> small > > >>>>>>> > cluster. In 2023, is it realistic to assume that this customer > > >>>>>>> won't be > > >>>>>>> > able to install JDK 17 on their cluster? Even in large > > >>>>>>> enterprises, it > > >>>>>>> > should be feasible to install an LTS JDK, especially > considering > > >>>>>>> the > > >>>>>>> > widespread adoption of cloud computing. Sungwoo Park's > > >>>>>>> measurements also > > >>>>>>> > support this recommendation to go with JDK 17. It outperforms > JDK > > >>>>>>> 11 by 8% > > >>>>>>> > in terms of runtime speed, and JDK 11 itself is 10+% faster > than > > >>>>>>> JDK 8. > > >>>>>>> > This is a significant value proposition. Who would be the > customer > > >>>>>>> that > > >>>>>>> > says, "I don't want faster query execution! I'd rather use JDK > 8 > > >>>>>>> and pay > > >>>>>>> > more for cloud or data center resources instead of using JDK > 17!" > > >>>>>>> It > > >>>>>>> > doesn't make sense to me. > > >>>>>>> > > > >>>>>>> > The tech industry has been evolving at an incredible pace, with > > >>>>>>> > improvements in serialization, IPC mechanisms, and parallelized > > >>>>>>> frameworks > > >>>>>>> > since the release of JDK 8 ten years ago. We should leverage > these > > >>>>>>> > advancements! Couple years ago, we invested a lot to improve > 1-2% > > >>>>>>> of the > > >>>>>>> > execution. We prayed for 3 gods, sacrificed 2 ships and > traveled > > >>>>>>> around the > > >>>>>>> > world to make it happen. :-) Now, the JDK itself provides a > > >>>>>>> substantial > > >>>>>>> > amount of improvement. So, why would we resist progress just > > >>>>>>> because there > > >>>>>>> > are a few lazy or conservative admins who don't want to spend > two > > >>>>>>> minutes > > >>>>>>> > installing a JDK? > > >>>>>>> > > > >>>>>>> > A platform needs to be modern and incorporate the latest > > >>>>>>> technologies to > > >>>>>>> > attract developers and users. I understand that some may > prefer to > > >>>>>>> stay > > >>>>>>> > with JDK 8 as it seems like the safest position, but I believe > in > > >>>>>>> taking > > >>>>>>> > bold bets to achieve big wins. Even if we decide to stick with > JDK > > >>>>>>> 8, I > > >>>>>>> > would still be happy since we are moving forward and not > dwelling > > >>>>>>> on a JDK > > >>>>>>> > that is a decade old. Personally, I think focusing on one thing > > >>>>>>> that brings > > >>>>>>> > more value to us and our users is the idealistic path forward. > > >>>>>>> > > > >>>>>>> > -Attila > > >>>>>>> > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> > On Thu, Jun 1, 2023 at 11:23 AM Stamatis Zampetakis < > > >>>>>>> zabe...@gmail.com> > > >>>>>>> > wrote: > > >>>>>>> > > > >>>>>>> >> Hey everyone, > > >>>>>>> >> > > >>>>>>> >> If we claim that Hive supports a certain JDK then we should > > >>>>>>> compile and > > >>>>>>> >> run > > >>>>>>> >> tests with it. > > >>>>>>> >> > > >>>>>>> >> The more JDKs we can support the better for everyone but this > > >>>>>>> comes at a > > >>>>>>> >> cost (resources mostly). We should have a precommit run for > every > > >>>>>>> >> supported > > >>>>>>> >> JDK (frequency to be determined once per day/week) that > compiles > > >>>>>>> and run > > >>>>>>> >> all tests. > > >>>>>>> >> > > >>>>>>> >> From my perspective, I would be pretty happy if we could cover > > >>>>>>> the two > > >>>>>>> >> edge > > >>>>>>> >> LTS releases at every point in time. > > >>>>>>> >> > > >>>>>>> >> Then we have to decide also which JDK shall we use for the > pull > > >>>>>>> requests > > >>>>>>> >> and local dev environment. I think it makes sense to use the > > >>>>>>> latest. > > >>>>>>> >> People > > >>>>>>> >> like working on modern stuff and also it makes sense that > newer > > >>>>>>> releases > > >>>>>>> >> will also use newer versions. It would be pretty awkward if > > >>>>>>> someone wants > > >>>>>>> >> to use the latest Hive version and it turns out that it can > only > > >>>>>>> run on > > >>>>>>> >> JDK8. > > >>>>>>> >> > > >>>>>>> >> Best, > > >>>>>>> >> Stamatis > > >>>>>>> >> > > >>>>>>> >> On Thu, Jun 1, 2023, 3:42 AM Sungwoo Park <glap...@gmail.com> > > >>>>>>> wrote: > > >>>>>>> >> > > >>>>>>> >> > Hi, everyone. > > >>>>>>> >> > > > >>>>>>> >> > I have not tested the master branch with Java 11/17 yet, > but I > > >>>>>>> would > > >>>>>>> >> like > > >>>>>>> >> > to share my experience with testing a fork of branch-3.1 > with > > >>>>>>> Java 11/17 > > >>>>>>> >> > (as part of developing Hive-MR3), in case that it can be > useful > > >>>>>>> for the > > >>>>>>> >> > discussion. I merged the patches listed in [1] HIVE-22415 > and > > >>>>>>> updated > > >>>>>>> >> the > > >>>>>>> >> > Maven configuration for Java 11. > > >>>>>>> >> > > > >>>>>>> >> > 1. Building Hive was fine and I was able to run it with > Java 11 > > >>>>>>> as well > > >>>>>>> >> as > > >>>>>>> >> > Java 17. So, it seems that the work reported in [1] is > indeed > > >>>>>>> complete > > >>>>>>> >> for > > >>>>>>> >> > upgrading to Java 11 (and Java 17) and getting Hive to work. > > >>>>>>> >> > > > >>>>>>> >> > 2. However, there was a problem with running tests, so this > can > > >>>>>>> be > > >>>>>>> >> > additional work for upgrading to Java 11. > > >>>>>>> >> > > > >>>>>>> >> > 3. For performance, Java 17 gives about 8 percent of (free) > > >>>>>>> performance > > >>>>>>> >> > improvement. When tested with 10TB TPC-DS, Java 8 takes 8074 > > >>>>>>> seconds, > > >>>>>>> >> > whereas Java 17 takes 7415 seconds. Considering the > maturity of > > >>>>>>> Hive, I > > >>>>>>> >> > think this is not a small improvement because almost every > > >>>>>>> query gets > > >>>>>>> >> some > > >>>>>>> >> > speedup. > > >>>>>>> >> > > > >>>>>>> >> > Thanks, > > >>>>>>> >> > > > >>>>>>> >> > --- Sungwoo > > >>>>>>> >> > > > >>>>>>> >> > [1] https://issues.apache.org/jira/browse/HIVE-22415 > > >>>>>>> >> > > > >>>>>>> >> > > > >>>>>>> >> > On Thu, Jun 1, 2023 at 3:53 AM Sai Hemanth Gantasala > > >>>>>>> >> > <saihema...@cloudera.com.invalid> wrote: > > >>>>>>> >> > > > >>>>>>> >> > > Hi All, > > >>>>>>> >> > > > > >>>>>>> >> > > I would strongly advocate keeping support for JDK8. > > >>>>>>> >> > > Between JDK11 and JDK17, Depending on the amount of > effort on > > >>>>>>> the > > >>>>>>> >> upgrade > > >>>>>>> >> > > I'm inclined towards JDK17 (JDK21 LTS will be released in > Sep > > >>>>>>> 2023). > > >>>>>>> >> > > > > >>>>>>> >> > > Thanks, > > >>>>>>> >> > > Sai. > > >>>>>>> >> > > > > >>>>>>> >> > > On Wed, May 31, 2023 at 5:39 AM László Bodor < > > >>>>>>> >> bodorlaszlo0...@gmail.com> > > >>>>>>> >> > > wrote: > > >>>>>>> >> > > > > >>>>>>> >> > > > *Hi!* > > >>>>>>> >> > > > > > >>>>>>> >> > > > > > >>>>>>> >> > > > *Should we support both JDK-11 & JDK-8?* > > >>>>>>> >> > > > IMO absolutely yes, let's not break up with JDK-8: > > >>>>>>> according to its > > >>>>>>> >> > > > lifecycle, it's going to stay with us for a long time. > > >>>>>>> >> > > > > > >>>>>>> >> > > > I believe > > >>>>>>> >> > > > a) we should be able to compile on JDK8, JDK11, and > JDK17 > > >>>>>>> (github > > >>>>>>> >> > actions > > >>>>>>> >> > > > can cover this conveniently in precommit time, like tez > > >>>>>>> >> > > > < > > >>>>>>> >> > > >>>>>>> > https://github.com/apache/tez/blob/master/.github/workflows/build.yml > > >>>>>>> >> > >) > > >>>>>>> >> > > > b) the release artifacts should be compatible with JDK8 > as > > >>>>>>> long as > > >>>>>>> >> it > > >>>>>>> >> > is > > >>>>>>> >> > > > with us. > > >>>>>>> >> > > > > > >>>>>>> >> > > > Regards, > > >>>>>>> >> > > > Laszlo Bodor > > >>>>>>> >> > > > > > >>>>>>> >> > > > > > >>>>>>> >> > > > Butao Zhang <butaozha...@163.com> ezt írta (időpont: > 2023. > > >>>>>>> máj. > > >>>>>>> >> 31., > > >>>>>>> >> > > Sze, > > >>>>>>> >> > > > 14:33): > > >>>>>>> >> > > > > > >>>>>>> >> > > > > Thanks Ayush for driving this! Good to know that Hive > is > > >>>>>>> getting > > >>>>>>> >> > ready > > >>>>>>> >> > > > for > > >>>>>>> >> > > > > newer JDK. > > >>>>>>> >> > > > > From my opinon, if we have more community energy to > put > > >>>>>>> into it, > > >>>>>>> >> we > > >>>>>>> >> > can > > >>>>>>> >> > > > > support both JDK-11 and JDK-17 like Spark[1]. If we > have > > >>>>>>> to make > > >>>>>>> >> a > > >>>>>>> >> > > > choice > > >>>>>>> >> > > > > between a JDK-11 and JDK-17, i would like to choose > the > > >>>>>>> relatively > > >>>>>>> >> > new > > >>>>>>> >> > > > > version JDK-17, meanwhile, we should maintain > > >>>>>>> compatibility with > > >>>>>>> >> > jdk8, > > >>>>>>> >> > > as > > >>>>>>> >> > > > > JDK-8 is still widely used in most big data platforms. > > >>>>>>> >> > > > > > > >>>>>>> >> > > > > > > >>>>>>> >> > > > > Thanks, > > >>>>>>> >> > > > > Butao Zhang > > >>>>>>> >> > > > > > > >>>>>>> >> > > > > > > >>>>>>> >> > > > > [1]https://issues.apache.org/jira/browse/SPARK-33772 > > >>>>>>> >> > > > > ---- Replied Message ---- > > >>>>>>> >> > > > > | From | Ayush Saxena<ayush...@gmail.com> | > > >>>>>>> >> > > > > | Date | 5/31/2023 18:39 | > > >>>>>>> >> > > > > | To | dev<dev@hive.apache.org> | > > >>>>>>> >> > > > > | Subject | Move to JDK-11 | > > >>>>>>> >> > > > > Hi Everyone, > > >>>>>>> >> > > > > Want to pull in the attention of folks towards moving > to > > >>>>>>> JDK-11 > > >>>>>>> >> > compile > > >>>>>>> >> > > > > time support in Hive. There was a ticket in the past > [1] > > >>>>>>> which > > >>>>>>> >> talks > > >>>>>>> >> > > > about > > >>>>>>> >> > > > > it and If I could decode it right, it was blocked > because > > >>>>>>> the > > >>>>>>> >> Hadoop > > >>>>>>> >> > > > > version used by Hive didn't had JDK-11 runtime > support, > > >>>>>>> But with > > >>>>>>> >> [2] > > >>>>>>> >> > in > > >>>>>>> >> > > > we > > >>>>>>> >> > > > > have upgraded the Hadoop version, so that problem is > > >>>>>>> sorted out. I > > >>>>>>> >> > > > couldn't > > >>>>>>> >> > > > > even see any unresolved tickets in the blocked state > > >>>>>>> either. > > >>>>>>> >> > > > > > > >>>>>>> >> > > > > I quickly tried* a mvn clean install -DskipTests > > >>>>>>> -Piceberg > > >>>>>>> >> -Pitests > > >>>>>>> >> > > > > -Dmaven.javadoc.skip=true > > >>>>>>> >> > > > > > > >>>>>>> >> > > > > And no surprises it failed with some weird exceptions > > >>>>>>> towards the > > >>>>>>> >> > end. > > >>>>>>> >> > > > But > > >>>>>>> >> > > > > I think that should be solvable. > > >>>>>>> >> > > > > > > >>>>>>> >> > > > > So, Questions? > > >>>>>>> >> > > > > > > >>>>>>> >> > > > > - What do folks think about this? Should we put in > some > > >>>>>>> effort > > >>>>>>> >> > towards > > >>>>>>> >> > > > > JDK-11 > > >>>>>>> >> > > > > - Should we support both JDK-11 & JDK-8? > > >>>>>>> >> > > > > - Ditch JDK-11 and directly shoot for JDK-17? > > >>>>>>> >> > > > > > > >>>>>>> >> > > > > Let me know your thoughts, In case anyone has some > > >>>>>>> experience in > > >>>>>>> >> this > > >>>>>>> >> > > > area > > >>>>>>> >> > > > > and have tried something in the context, feel free to > > >>>>>>> share or > > >>>>>>> >> may be > > >>>>>>> >> > > if > > >>>>>>> >> > > > > someone has any potential action plan or so > > >>>>>>> >> > > > > > > >>>>>>> >> > > > > -Ayush > > >>>>>>> >> > > > > > > >>>>>>> >> > > > > [1] https://issues.apache.org/jira/browse/HIVE-22415 > > >>>>>>> >> > > > > [2] https://issues.apache.org/jira/browse/HIVE-24484 > > >>>>>>> >> > > > > > > >>>>>>> >> > > > > * changed the maven.compiler.source & > > >>>>>>> maven.compiler.target to 11 > > >>>>>>> >> > > > > > > >>>>>>> >> > > > > > >>>>>>> >> > > > > >>>>>>> >> > > > >>>>>>> >> > > >>>>>>> > > > >>>>>>> > > >>>>>> > > >
