Congratulations guys! That's really awesome! Will test it out and let you know if I run into any bugs!
Thanks everyone ! Simhadri G On Sat, May 31, 2025, 12:28 AM Ayush Saxena <ayush...@gmail.com> wrote: > Hi Folks, > > The Hive master branch has now been upgraded to JDK 17. This means > that building or running the latest code from the master branch will > require JDK 17 going forward. > You're encouraged to try it out and share your feedback. If you > encounter any issues, please feel free to report them here or create a > ticket with the details. > > A big thank you to all the contributors who helped make this > transition possible! > > Looking ahead, Hive 4.1.0 is expected to be released by the end of > June or early July, and will officially support JDK 17. > > Best regards, > Ayush > > ---------- Forwarded message --------- > From: Vlad Rozov <vro...@apache.org> > Date: Tue, 6 May 2025 at 20:29 > Subject: Re: Move to JDK-11 > To: <dev@hive.apache.org> > > > I am looking to upgrade Apache Spark 4.x to use Apache Hive 4.x as > both compile and runtime dependencies. Spark 4.x requires JDK 17 and > while working on upgrade I have not encountered issues caused by Hive > using JDK 8. I will keep an eye and report any, though I don't expect > to see a problem till Spark upgrades to JDK 21. > > Thank you, > > Vlad > > On 2025/04/08 14:27:33 Butao Zhang wrote: > > The upgrade to JDK 17 is a significant change that introduces > compatibility and adaptation requirements for both Apache Tez and Apache > Hive. Relevant work is currently in progress—thanks to all community > contributors who have already been involved in adaptation and fixes. We > encourage everyone to actively report any issues encountered during usage > or integration, so that we can collaborate on discussions and resolutions, > and accelerate the overall transition to JDK 17. > > > > Thanks, > > Butao Zhang > > > > On 2025/04/07 11:25:33 Shohei Okumiya wrote: > > > Hi Ayush and all people working on the JDK upgrade, > > > > > > Thanks for sharing the timeline and detailing the current progress. I'm > > > more than happy to hear the good news and look forward to seeing Hive > > > running on Java 17! > > > > > > Mid-May is not so far away. I don't have a firm preference about how to > > > package 4.1. If anyone is interested, let's discuss it in the thread > for > > > the Hive 4.1.x release. > > > > > > Finally, let me express my gratitude to everyone working on the > challenging > > > upgrade project. It will definitely be one of the most outstanding > > > achievements! > > > > > > Sincerely, > > > Okumin > > > > > > On Mon, Apr 7, 2025 at 7:03 PM Ayush Saxena <ayush...@gmail.com> > wrote: > > > > > > > Hi Okumin, > > > > > > > > I know I'm one of the people responsible for misjudging the timeline > > > > around the JDK-17 upgrade, which has ultimately blocked the 4.1.0 > release. > > > > I want to take a moment to share my thoughts and where things > currently > > > > stand. > > > > > > > > Q: Can we confirm the importance and ETA of the JDK upgrade and how > we > > > > should proceed? > > > > > > > > Ans: I think we all agree that moving to JDK-17 is a major > milestone, and > > > > we're admittedly late in adopting it. Many projects have already > moved or > > > > are actively working towards it. Not being on JDK-17 restricts us > from > > > > adopting modern third-party libraries like Iceberg and from > leveraging > > > > several improvements that come with it. > > > > > > > > As for the ETA: I realize I previously gave an inaccurate timeline, > and I > > > > sincerely regret that. However, I now have a much clearer picture. > I'm in > > > > active discussions with folks internally who are working on it. I > believe > > > > we should have the code ready by mid-May, at which point we can start > > > > gathering feedback from the community. > > > > > > > > Q: We've been working on this upgrade for over half a year. When > will it > > > > merge? PR: https://github.com/apache/hive/pull/5404 > > > > In my opinion, #5404 should be divided into multiple pull requests. I > > > > really understand we want to test simultaneously rather than make > > > > incremental progress. However, the consolidated PR makes it > impossible for > > > > other developers to help the initiative, reviewers have to check huge > > > > changes, and authors must occasionally resolve many conflicts. I > don't > > > > prefer any HOW as long as the delivery is predictable, meaning it is > okay > > > > for me if we are confident with the agreed ETA. If it is still > uncertain, > > > > we may try another project management, which is > incremental/predictable and > > > > where we can invest other developers' resources in it. > > > > > > > > Ans: I completely agree with your observations. The original PR > started > > > > more as an experiment to see what changes were needed. At the time, > I don’t > > > > think anyone anticipated how large it would become, and > unfortunately, it > > > > kept growing incrementally over time. > > > > That said, I’ve already asked the team to start breaking down the PR > into > > > > smaller, more manageable parts. Dependency upgrades will be > separated out > > > > soon, and changes like those related to SpotBugs (which account for > a large > > > > number of line changes) will also be split. The goal is to make the > PR > > > > easier to review and more accessible for other contributors to jump > in. I > > > > understand this is a genuine blocker, and folks are doing their > best to > > > > address it. > > > > > > > > Q: [1] Should we ship 4.1 without JDK-17, given it's been a long time > > > > since 4.0.1? > > > > > > > > Ans: Honestly, I’m not in a position to oppose this. The community > kindly > > > > respected my request last time when I asked for more time for the > JDK-17 > > > > upgrade, and unfortunately, I couldn’t follow through as expected. > So, with > > > > sincere apologies for being one of the reasons for the delay, I’m > okay with > > > > whatever direction the community chooses to take from here. > > > > > > > > -Ayush > > > > > > > > > > > > On Sun, 6 Apr 2025 at 11:55, Shohei Okumiya <oku...@apache.org> > wrote: > > > > > > > >> Hi, > > > >> > > > >> I appreciate everyone working on this big and challenging project! > > > >> > > > >> Can we confirm the importance and ETA of the JDK upgrade and how we > > > >> should work on it? > > > >> > > > >> I recognize it is one of the most essential initiatives blocking the > > > >> release of Hive 4.1.0[1] and the upgrade of Iceberg libraries[2]. > Users and > > > >> developers would expect it to be shipped soon. > > > >> > > > >> We have to complete this effort to ship JDK 17. We've been working > on it > > > >> for over half a year. Some of us might be interested in when we > will merge > > > >> it. > > > >> https://github.com/apache/hive/pull/5404 > > > >> > > > >> In my opinion, #5404 should be divided into multiple pull requests. > I > > > >> really understand we want to test simultaneously rather than make > > > >> incremental progress. However, the consolidated PR makes it > impossible for > > > >> other developers to help the initiative, reviewers have to check > huge > > > >> changes, and authors must occasionally resolve many conflicts. I > don't > > > >> prefer any HOW as long as the delivery is predictable, meaning it > is okay > > > >> for me if we are confident with the agreed ETA. If it is still > uncertain, > > > >> we may try another project management, which is > incremental/predictable and > > > >> where we can invest other developers' resources in it. > > > >> > > > >> - [1] Honestly, I feel we should ship 4.1 without JDK 17, as it's > been a > > > >> long time since the last release of 4.0.1 > > > >> - [2] Iceberg 1.7 or later doesn't support Java 8 > > > >> > > > >> Best, > > > >> Okumin > > > >> > > > >> On Wed, Dec 11, 2024 at 2:51 AM Ayush Saxena <ayush...@gmail.com> > wrote: > > > >> > > > >>> Regarding JDK-11, we didn't actively chased that, so its status > remains > > > >>> uncertain. With lower compile-time support like JDK-8, we can > certainly > > > >>> ensure runtime support for both JDK-11 and JDK-17, which I believe > should > > > >>> work for Hive 4.0.1. As far as I know, this approach works > downstream for > > > >>> us atleast. > > > >>> > > > >>> However, maintaining compile-time compatibility across JDK-17, > JDK-11, > > > >>> and JDK-8 simultaneously may not be feasible. JDK-17 introduces > significant > > > >>> changes, and I don’t believe it’s naive—or even possible—to > compile Hive > > > >>> using both JDK-8 and JDK-17 & expect things to work properly. > > > >>> > > > >>> The initial plan appeared to be supporting JDK-8 for Hive 4.0.x > release > > > >>> line, with potential tweaks if required to enable runtime support > for > > > >>> JDK-17. Starting with Hive 4.1.x, we could shift to JDK-17 for > compile-time > > > >>> support, and later evaluate the feasibility of supporting JDK-21 > or newer > > > >>> versions. > > > >>> > > > >>> Once we see a state when nobody want JDK-8 compile time support & > > > >>> everyone has moved to JDK-17 or say using 4.1.x release line, We > can then > > > >>> drop having bug fix releases for 4.0.x > > > >>> > > > >>> -Ayush > > > >>> > > > >>> On Mon, 9 Dec 2024 at 15:11, Butao Zhang <butaozha...@163.com> > wrote: > > > >>> > > > >>>> IMO, I think there two challenges if we want to maintain JDK8 & > JDK11 & > > > >>>> JDK17 at the same time. > > > >>>> > > > >>>> - Different jdk syntactic sugar between multiple jdks, and > advanced > > > >>>> feats in jdk17&jdk21, such as jdk vector api. If we want the > multiple jdks, > > > >>>> we can not use the advanced feats in high jdk in the future. > Such as the > > > >>>> discussion in https://issues.apache.org/jira/browse/HIVE-28614 > > > >>>> - We need to run the jenkins CI process multiple times to run > the > > > >>>> multiple jdk env. I'm not sure we have enough jenkins > resources to do this. > > > >>>> However, i want to say if we can use github actions to run the > whole CI? I > > > >>>> have seen Apache Spark have done the good job which using the > forked > > > >>>> repository github actions to run the CI. FRY > > > >>>> > https://lists.apache.org/thread/dmv1h2snqc7qtdjlzj4k8qxookclp720 > > > >>>> > > > >>>> > > > >>>> Thanks, > > > >>>> Butao Zhang > > > >>>> ---- Replied Message ---- > > > >>>> From Shohei Okumiya<oku...@apache.org> <undefined> > > > >>>> Date 12/9/2024 17:06 > > > >>>> To <dev@hive.apache.org> <dev@hive.apache.org> > > > >>>> Subject Re: Move to JDK-11 > > > >>>> Hi, > > > >>>> > > > >>>> I appreciate everyone's work on the JDK upgrade. It would be > ideal if > > > >>>> we could migrate JDK like this. That will give us an > > > >>>> incremental migration path. > > > >>>> > > > >>>> 1. (now) Hive 4.0 compiled by JDK 8, and run by JDK 8 > > > >>>> 2. Hive 4.x compiled by JDK 8, and run by JDK 17 > > > >>>> 3. Hive 4.x compiled by JDK 17, and run by JDK 17 > > > >>>> > > > >>>> I guess it is technically possible that Hive supports JDK 8 and > 17 at > > > >>>> runtime. Apache Spark supports JDK 8, 11, and 17. What is the most > > > >>>> challenging point? > > > >>>> > > > >>>> Regards, > > > >>>> Okumin > > > >>>> > > > >>>> On Thu, Aug 29, 2024 at 10:48 PM Butao Zhang <butaozha...@163.com > > > > > >>>> wrote: > > > >>>> > > > >>>>> Hi Akshat, > > > >>>>> > > > >>>>> Thanks for your reply! I agree that supporting both JDK 8 and > JDK 17 > > > >>>>> is not easy. I think we can safely to move to JDK 17 if we can > ensure *hive > > > >>>>> application* can access to HiveServer2/Metastore without > forcing to > > > >>>>> use the JDK 17. > > > >>>>> For example, if a hive application is developed by JDK 8, it can > > > >>>>> import the *hive-exec.jar & hive-jdbc.jar* which is compiled by > JDK > > > >>>>> 17 and can access to hs2/hms successfully. If this case can be > supported, > > > >>>>> I think we can move to JDK 17 as soon as possible. > > > >>>>> > > > >>>>> BTW, If Apache Hive , Apache Tez and Apache Hadoop are all > compiled > > > >>>>> with JDK17 or higer version, i think the hive job will run more > faster. > > > >>>>> Users will enjoy significant performance improvements due to JDK > version > > > >>>>> upgrades. > > > >>>>> I'm looking forward to the performance improvement. :) > > > >>>>> > > > >>>>> Thanks, > > > >>>>> Butao Zhang > > > >>>>> ---- Replied Message ---- > > > >>>>> From Akshat m<akshatats...@gmail.com> <undefined> > > > >>>>> Date 8/29/2024 18:44 > > > >>>>> To <dev@hive.apache.org> <dev@hive.apache.org> > > > >>>>> Subject Re: Move to JDK-11 > > > >>>>> Hi Butao, > > > >>>>> > > > >>>>> It won’t be feasible to support both JDK 8 and JDK 17 in the > same Hive > > > >>>>> version due to compatibility issues and behavior changes, as > detailed in > > > >>>>> the PR. The internal changes in JDK implementations have made > this > > > >>>>> challenging. Additionally, many other libraries maintain > separate versions > > > >>>>> for JDK 17+ and JDK 8 support. > > > >>>>> > > > >>>>> To align with this approach, I suggest we continue with Hive 4 > > > >>>>> supporting JDK 8. Once the JDK 17 PR is merged, we could plan a > separate > > > >>>>> release that specifically targets JDK 17+ support. This would > ensure that > > > >>>>> Hive stays up-to-date while maintaining stability across > different > > > >>>>> environments. > > > >>>>> > > > >>>>> Would love to hear your thoughts or any additional suggestions > the > > > >>>>> community might have! > > > >>>>> > > > >>>>> Regards > > > >>>>> > > > >>>>> Akshat > > > >>>>> > > > >>>>> > > > >>>>> > > > >>>>> On Wed, Aug 28, 2024 at 8:00 AM Butao Zhang <butaozha...@163.com > > > > > >>>>> wrote: > > > >>>>> > > > >>>>>> Good job! Thanks everyone who put effort into the JDK upgrading. > > > >>>>>> That's a big good news for Apache Hive community! > > > >>>>>> Just a question: Given that many people are still using the > lower > > > >>>>>> jdk8 version. Do we consider to keeping compatibility with > lower jdk8 > > > >>>>>> version? > > > >>>>>> > > > >>>>>> > > > >>>>>> Thanks, > > > >>>>>> Butao Zhang > > > >>>>>> ---- Replied Message ---- > > > >>>>>> From Akshat m<akshatats...@gmail.com> <undefined> > > > >>>>>> Date 8/27/2024 23:12 > > > >>>>>> To dev<dev@hive.apache.org> <dev@hive.apache.org> > > > >>>>>> Cc kokila narayanan<kokilanarayana...@gmail.com>, > > > >>>>>> <kokilanarayana...@gmail.com>Raghav > > > >>>>>> Aggarwal<raghavaggarwal03...@gmail.com>, > > > >>>>>> <raghavaggarwal03...@gmail.com>Ayush Saxena<ayush...@gmail.com > >, > > > >>>>>> <ayush...@gmail.com>tanishq chugh<tanishqchugh4...@gmail.com>, > > > >>>>>> <tanishqchugh4...@gmail.com><aturo...@cloudera.com> > > > >>>>>> <aturo...@cloudera.com> > > > >>>>>> Subject Re: Move to JDK-11 > > > >>>>>> > > > >>>>>> Hi everyone, > > > >>>>>> > > > >>>>>> I’m excited to share that after months of dedicated work, we’ve > > > >>>>>> finally achieved a passing PR[1] for running *Apache Hive on > JDK 17*! > > > >>>>>> > > > >>>>>> This was a challenging task, requiring to address various > > > >>>>>> compatibility issues and ensure our comprehensive test suite > passed without > > > >>>>>> a hitch. > > > >>>>>> > > > >>>>>> Thanks Kokila and Tanishq for joining me in the initiative and > Raghav > > > >>>>>> for contributing fix for the errorProne issue, and thanks to > Attila, Ayush > > > >>>>>> and other Hive PMC members for their guidance in the initiative > > > >>>>>> > > > >>>>>> It’s been a real team effort, and I’m grateful for the > collaboration > > > >>>>>> and support from everyone involved. > > > >>>>>> > > > >>>>>> I’d appreciate it if you could take some time to review the PR. > > > >>>>>> Community feedback is really important to make this happen. > > > >>>>>> > > > >>>>>> The PR #5404 addresses the integration of Apache Hive with JDK > 17, > > > >>>>>> introducing a range of changes aimed at ensuring compatibility > and > > > >>>>>> improving the codebase. Here’s a detailed breakdown: > > > >>>>>> > > > >>>>>> 1. Core Code Adjustments: > > > >>>>>> > > > >>>>>> - Compatibility Fixes: Various parts of the Hive codebase > have > > > >>>>>> been modified to ensure they work seamlessly with JDK 17. This > includes > > > >>>>>> addressing language or API changes that JDK 17 introduces. > > > >>>>>> > > > >>>>>> - Reflection and Modules: Adjustments related to how Hive > > > >>>>>> interacts with Java's reflection API, which has stricter module > system > > > >>>>>> rules in JDK 17. > > > >>>>>> > > > >>>>>> 2. Dependency Management: > > > >>>>>> > > > >>>>>> - Version Upgrades: Several third-party dependencies have > been > > > >>>>>> updated to versions that are compatible with JDK 17. This is > critical > > > >>>>>> because older versions of some libraries may not work properly > or might use > > > >>>>>> deprecated features removed in JDK 17. > > > >>>>>> > > > >>>>>> - Data nucleus upgrade: To ensure compatibility with Java > 17, we > > > >>>>>> upgraded DataNucleus to version 6.x. This upgrade, however, > introduced an > > > >>>>>> issue with Hive schema creation. The issue stems from a commit > in > > > >>>>>> DataNucleus[2] that alters the handling of primary key column > names by > > > >>>>>> enclosing them in double quotes. This change caused conflicts > during Hive > > > >>>>>> schema generation. As an interim solution, we implemented a > patched > > > >>>>>> DataNucleus-rdbms module to address the issue in Hive. The > necessary fix is > > > >>>>>> now included in the DataNucleus repository[3]. Once DataNucleus > 6.0.8 is > > > >>>>>> officially released, we will upgrade to this version and remove > the > > > >>>>>> temporary patched module. > > > >>>>>> > > > >>>>>> - Dependency Cleanup: Unused or outdated dependencies were > removed, > > > >>>>>> reducing potential conflicts and simplifying the build process. > > > >>>>>> > > > >>>>>> 3. Testing Enhancements: > > > >>>>>> > > > >>>>>> - Test Suite Updates: The PR includes updates to Hive’s test > suite > > > >>>>>> to ensure all tests pass under JDK 17. This likely involved > modifying tests > > > >>>>>> that needed adjustments to align with JDK 17’s behaviour. > > > >>>>>> > > > >>>>>> - New Tests: New tests may have been added to cover areas > where JDK > > > >>>>>> 17 introduced new features or changes that could impact Hive. > > > >>>>>> > > > >>>>>> 4. Compatibility Checks: > > > >>>>>> > > > >>>>>> - JDK Version Verification: A new compatibility check has been > > > >>>>>> introduced to verify the JDK version at runtime. This ensures > that Hive is > > > >>>>>> running on an appropriate JDK version, preventing unexpected > errors or > > > >>>>>> behaviour due to version mismatches. > > > >>>>>> > > > >>>>>> 5. Build Updates: > > > >>>>>> > > > >>>>>> - Build System Tweaks: The PR likely includes changes to the > Maven > > > >>>>>> build configuration to accommodate JDK 17, ensuring that the > build process > > > >>>>>> is smooth and compatible. > > > >>>>>> > > > >>>>>> Overall, this PR is a comprehensive effort to bring Apache Hive > up to > > > >>>>>> date with the latest Java platform, ensuring long-term > maintainability and > > > >>>>>> performance improvements. > > > >>>>>> > > > >>>>>> As the next steps, we’ll be focusing on further sanity checks > and > > > >>>>>> extensive testing to make sure everything functions smoothly. > Community > > > >>>>>> involvement in this process will be crucial, so please do share > any issues > > > >>>>>> or feedback as you test the changes. > > > >>>>>> > > > >>>>>> I’m looking forward to seeing the benefits this will bring to > our > > > >>>>>> community and users. > > > >>>>>> > > > >>>>>> > > > >>>>>> [1] https://github.com/apache/hive/pull/5404 > > > >>>>>> > > > >>>>>> [2] > > > >>>>>> > https://github.com/datanucleus/datanucleus-rdbms/commit/b99df3c803862f9e9feacc7891461bcb21160b38 > > > >>>>>> > > > >>>>>> [3] > > > >>>>>> > https://github.com/datanucleus/datanucleus-rdbms/commit/b99df3c803862f9e9feacc7891461bcb21160b38 > > > >>>>>> > > > >>>>>> > > > >>>>>> Best regards, > > > >>>>>> > > > >>>>>> Akshat Mathur > > > >>>>>> > > > >>>>>> > > > >>>>>> On Tue, 11 Jul 2023 at 2:25 PM, Attila Turoczy < > > > >>>>>> aturo...@cloudera.com.invalid> wrote: > > > >>>>>> > > > >>>>>>> Returning to this topic, I kindly request those who would like > to > > > >>>>>>> advocate > > > >>>>>>> for the continued support of JDK8 to please share their > reasoning and > > > >>>>>>> insights with us. Your input and perspective are greatly > appreciated! > > > >>>>>>> Thank you. > > > >>>>>>> > > > >>>>>>> -Attila > > > >>>>>>> > > > >>>>>>> On Fri, Jun 2, 2023 at 12:43 PM Attila Turoczy < > > > >>>>>>> aturo...@cloudera.com> > > > >>>>>>> wrote: > > > >>>>>>> > > > >>>>>>> > Hi All, > > > >>>>>>> > > > > >>>>>>> > I know my opinion might not be the most popular, but I > advocate > > > >>>>>>> for using* > > > >>>>>>> > JDK 17*. Here's why: > > > >>>>>>> > > > > >>>>>>> > Let's consider a scenario where a customer wants to use the > latest > > > >>>>>>> version > > > >>>>>>> > of Apache Hive. They would typically install it locally or > on a > > > >>>>>>> small > > > >>>>>>> > cluster. In 2023, is it realistic to assume that this > customer > > > >>>>>>> won't be > > > >>>>>>> > able to install JDK 17 on their cluster? Even in large > > > >>>>>>> enterprises, it > > > >>>>>>> > should be feasible to install an LTS JDK, especially > considering > > > >>>>>>> the > > > >>>>>>> > widespread adoption of cloud computing. Sungwoo Park's > > > >>>>>>> measurements also > > > >>>>>>> > support this recommendation to go with JDK 17. It > outperforms JDK > > > >>>>>>> 11 by 8% > > > >>>>>>> > in terms of runtime speed, and JDK 11 itself is 10+% faster > than > > > >>>>>>> JDK 8. > > > >>>>>>> > This is a significant value proposition. Who would be the > customer > > > >>>>>>> that > > > >>>>>>> > says, "I don't want faster query execution! I'd rather use > JDK 8 > > > >>>>>>> and pay > > > >>>>>>> > more for cloud or data center resources instead of using JDK > 17!" > > > >>>>>>> It > > > >>>>>>> > doesn't make sense to me. > > > >>>>>>> > > > > >>>>>>> > The tech industry has been evolving at an incredible pace, > with > > > >>>>>>> > improvements in serialization, IPC mechanisms, and > parallelized > > > >>>>>>> frameworks > > > >>>>>>> > since the release of JDK 8 ten years ago. We should leverage > these > > > >>>>>>> > advancements! Couple years ago, we invested a lot to improve > 1-2% > > > >>>>>>> of the > > > >>>>>>> > execution. We prayed for 3 gods, sacrificed 2 ships and > traveled > > > >>>>>>> around the > > > >>>>>>> > world to make it happen. :-) Now, the JDK itself provides a > > > >>>>>>> substantial > > > >>>>>>> > amount of improvement. So, why would we resist progress just > > > >>>>>>> because there > > > >>>>>>> > are a few lazy or conservative admins who don't want to > spend two > > > >>>>>>> minutes > > > >>>>>>> > installing a JDK? > > > >>>>>>> > > > > >>>>>>> > A platform needs to be modern and incorporate the latest > > > >>>>>>> technologies to > > > >>>>>>> > attract developers and users. I understand that some may > prefer to > > > >>>>>>> stay > > > >>>>>>> > with JDK 8 as it seems like the safest position, but I > believe in > > > >>>>>>> taking > > > >>>>>>> > bold bets to achieve big wins. Even if we decide to stick > with JDK > > > >>>>>>> 8, I > > > >>>>>>> > would still be happy since we are moving forward and not > dwelling > > > >>>>>>> on a JDK > > > >>>>>>> > that is a decade old. Personally, I think focusing on one > thing > > > >>>>>>> that brings > > > >>>>>>> > more value to us and our users is the idealistic path > forward. > > > >>>>>>> > > > > >>>>>>> > -Attila > > > >>>>>>> > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>>>> > On Thu, Jun 1, 2023 at 11:23 AM Stamatis Zampetakis < > > > >>>>>>> zabe...@gmail.com> > > > >>>>>>> > wrote: > > > >>>>>>> > > > > >>>>>>> >> Hey everyone, > > > >>>>>>> >> > > > >>>>>>> >> If we claim that Hive supports a certain JDK then we should > > > >>>>>>> compile and > > > >>>>>>> >> run > > > >>>>>>> >> tests with it. > > > >>>>>>> >> > > > >>>>>>> >> The more JDKs we can support the better for everyone but > this > > > >>>>>>> comes at a > > > >>>>>>> >> cost (resources mostly). We should have a precommit run for > every > > > >>>>>>> >> supported > > > >>>>>>> >> JDK (frequency to be determined once per day/week) that > compiles > > > >>>>>>> and run > > > >>>>>>> >> all tests. > > > >>>>>>> >> > > > >>>>>>> >> From my perspective, I would be pretty happy if we could > cover > > > >>>>>>> the two > > > >>>>>>> >> edge > > > >>>>>>> >> LTS releases at every point in time. > > > >>>>>>> >> > > > >>>>>>> >> Then we have to decide also which JDK shall we use for the > pull > > > >>>>>>> requests > > > >>>>>>> >> and local dev environment. I think it makes sense to use the > > > >>>>>>> latest. > > > >>>>>>> >> People > > > >>>>>>> >> like working on modern stuff and also it makes sense that > newer > > > >>>>>>> releases > > > >>>>>>> >> will also use newer versions. It would be pretty awkward if > > > >>>>>>> someone wants > > > >>>>>>> >> to use the latest Hive version and it turns out that it can > only > > > >>>>>>> run on > > > >>>>>>> >> JDK8. > > > >>>>>>> >> > > > >>>>>>> >> Best, > > > >>>>>>> >> Stamatis > > > >>>>>>> >> > > > >>>>>>> >> On Thu, Jun 1, 2023, 3:42 AM Sungwoo Park < > glap...@gmail.com> > > > >>>>>>> wrote: > > > >>>>>>> >> > > > >>>>>>> >> > Hi, everyone. > > > >>>>>>> >> > > > > >>>>>>> >> > I have not tested the master branch with Java 11/17 yet, > but I > > > >>>>>>> would > > > >>>>>>> >> like > > > >>>>>>> >> > to share my experience with testing a fork of branch-3.1 > with > > > >>>>>>> Java 11/17 > > > >>>>>>> >> > (as part of developing Hive-MR3), in case that it can be > useful > > > >>>>>>> for the > > > >>>>>>> >> > discussion. I merged the patches listed in [1] HIVE-22415 > and > > > >>>>>>> updated > > > >>>>>>> >> the > > > >>>>>>> >> > Maven configuration for Java 11. > > > >>>>>>> >> > > > > >>>>>>> >> > 1. Building Hive was fine and I was able to run it with > Java 11 > > > >>>>>>> as well > > > >>>>>>> >> as > > > >>>>>>> >> > Java 17. So, it seems that the work reported in [1] is > indeed > > > >>>>>>> complete > > > >>>>>>> >> for > > > >>>>>>> >> > upgrading to Java 11 (and Java 17) and getting Hive to > work. > > > >>>>>>> >> > > > > >>>>>>> >> > 2. However, there was a problem with running tests, so > this can > > > >>>>>>> be > > > >>>>>>> >> > additional work for upgrading to Java 11. > > > >>>>>>> >> > > > > >>>>>>> >> > 3. For performance, Java 17 gives about 8 percent of > (free) > > > >>>>>>> performance > > > >>>>>>> >> > improvement. When tested with 10TB TPC-DS, Java 8 takes > 8074 > > > >>>>>>> seconds, > > > >>>>>>> >> > whereas Java 17 takes 7415 seconds. Considering the > maturity of > > > >>>>>>> Hive, I > > > >>>>>>> >> > think this is not a small improvement because almost every > > > >>>>>>> query gets > > > >>>>>>> >> some > > > >>>>>>> >> > speedup. > > > >>>>>>> >> > > > > >>>>>>> >> > Thanks, > > > >>>>>>> >> > > > > >>>>>>> >> > --- Sungwoo > > > >>>>>>> >> > > > > >>>>>>> >> > [1] https://issues.apache.org/jira/browse/HIVE-22415 > > > >>>>>>> >> > > > > >>>>>>> >> > > > > >>>>>>> >> > On Thu, Jun 1, 2023 at 3:53 AM Sai Hemanth Gantasala > > > >>>>>>> >> > <saihema...@cloudera.com.invalid> wrote: > > > >>>>>>> >> > > > > >>>>>>> >> > > Hi All, > > > >>>>>>> >> > > > > > >>>>>>> >> > > I would strongly advocate keeping support for JDK8. > > > >>>>>>> >> > > Between JDK11 and JDK17, Depending on the amount of > effort on > > > >>>>>>> the > > > >>>>>>> >> upgrade > > > >>>>>>> >> > > I'm inclined towards JDK17 (JDK21 LTS will be released > in Sep > > > >>>>>>> 2023). > > > >>>>>>> >> > > > > > >>>>>>> >> > > Thanks, > > > >>>>>>> >> > > Sai. > > > >>>>>>> >> > > > > > >>>>>>> >> > > On Wed, May 31, 2023 at 5:39 AM László Bodor < > > > >>>>>>> >> bodorlaszlo0...@gmail.com> > > > >>>>>>> >> > > wrote: > > > >>>>>>> >> > > > > > >>>>>>> >> > > > *Hi!* > > > >>>>>>> >> > > > > > > >>>>>>> >> > > > > > > >>>>>>> >> > > > *Should we support both JDK-11 & JDK-8?* > > > >>>>>>> >> > > > IMO absolutely yes, let's not break up with JDK-8: > > > >>>>>>> according to its > > > >>>>>>> >> > > > lifecycle, it's going to stay with us for a long time. > > > >>>>>>> >> > > > > > > >>>>>>> >> > > > I believe > > > >>>>>>> >> > > > a) we should be able to compile on JDK8, JDK11, and > JDK17 > > > >>>>>>> (github > > > >>>>>>> >> > actions > > > >>>>>>> >> > > > can cover this conveniently in precommit time, like > tez > > > >>>>>>> >> > > > < > > > >>>>>>> >> > > > >>>>>>> > https://github.com/apache/tez/blob/master/.github/workflows/build.yml > > > >>>>>>> >> > >) > > > >>>>>>> >> > > > b) the release artifacts should be compatible with > JDK8 as > > > >>>>>>> long as > > > >>>>>>> >> it > > > >>>>>>> >> > is > > > >>>>>>> >> > > > with us. > > > >>>>>>> >> > > > > > > >>>>>>> >> > > > Regards, > > > >>>>>>> >> > > > Laszlo Bodor > > > >>>>>>> >> > > > > > > >>>>>>> >> > > > > > > >>>>>>> >> > > > Butao Zhang <butaozha...@163.com> ezt írta (időpont: > 2023. > > > >>>>>>> máj. > > > >>>>>>> >> 31., > > > >>>>>>> >> > > Sze, > > > >>>>>>> >> > > > 14:33): > > > >>>>>>> >> > > > > > > >>>>>>> >> > > > > Thanks Ayush for driving this! Good to know that > Hive is > > > >>>>>>> getting > > > >>>>>>> >> > ready > > > >>>>>>> >> > > > for > > > >>>>>>> >> > > > > newer JDK. > > > >>>>>>> >> > > > > From my opinon, if we have more community energy to > put > > > >>>>>>> into it, > > > >>>>>>> >> we > > > >>>>>>> >> > can > > > >>>>>>> >> > > > > support both JDK-11 and JDK-17 like Spark[1]. If we > have > > > >>>>>>> to make > > > >>>>>>> >> a > > > >>>>>>> >> > > > choice > > > >>>>>>> >> > > > > between a JDK-11 and JDK-17, i would like to choose > the > > > >>>>>>> relatively > > > >>>>>>> >> > new > > > >>>>>>> >> > > > > version JDK-17, meanwhile, we should maintain > > > >>>>>>> compatibility with > > > >>>>>>> >> > jdk8, > > > >>>>>>> >> > > as > > > >>>>>>> >> > > > > JDK-8 is still widely used in most big data > platforms. > > > >>>>>>> >> > > > > > > > >>>>>>> >> > > > > > > > >>>>>>> >> > > > > Thanks, > > > >>>>>>> >> > > > > Butao Zhang > > > >>>>>>> >> > > > > > > > >>>>>>> >> > > > > > > > >>>>>>> >> > > > > [1] > https://issues.apache.org/jira/browse/SPARK-33772 > > > >>>>>>> >> > > > > ---- Replied Message ---- > > > >>>>>>> >> > > > > | From | Ayush Saxena<ayush...@gmail.com> | > > > >>>>>>> >> > > > > | Date | 5/31/2023 18:39 | > > > >>>>>>> >> > > > > | To | dev<dev@hive.apache.org> | > > > >>>>>>> >> > > > > | Subject | Move to JDK-11 | > > > >>>>>>> >> > > > > Hi Everyone, > > > >>>>>>> >> > > > > Want to pull in the attention of folks towards > moving to > > > >>>>>>> JDK-11 > > > >>>>>>> >> > compile > > > >>>>>>> >> > > > > time support in Hive. There was a ticket in the > past [1] > > > >>>>>>> which > > > >>>>>>> >> talks > > > >>>>>>> >> > > > about > > > >>>>>>> >> > > > > it and If I could decode it right, it was blocked > because > > > >>>>>>> the > > > >>>>>>> >> Hadoop > > > >>>>>>> >> > > > > version used by Hive didn't had JDK-11 runtime > support, > > > >>>>>>> But with > > > >>>>>>> >> [2] > > > >>>>>>> >> > in > > > >>>>>>> >> > > > we > > > >>>>>>> >> > > > > have upgraded the Hadoop version, so that problem is > > > >>>>>>> sorted out. I > > > >>>>>>> >> > > > couldn't > > > >>>>>>> >> > > > > even see any unresolved tickets in the blocked state > > > >>>>>>> either. > > > >>>>>>> >> > > > > > > > >>>>>>> >> > > > > I quickly tried* a mvn clean install -DskipTests > > > >>>>>>> -Piceberg > > > >>>>>>> >> -Pitests > > > >>>>>>> >> > > > > -Dmaven.javadoc.skip=true > > > >>>>>>> >> > > > > > > > >>>>>>> >> > > > > And no surprises it failed with some weird > exceptions > > > >>>>>>> towards the > > > >>>>>>> >> > end. > > > >>>>>>> >> > > > But > > > >>>>>>> >> > > > > I think that should be solvable. > > > >>>>>>> >> > > > > > > > >>>>>>> >> > > > > So, Questions? > > > >>>>>>> >> > > > > > > > >>>>>>> >> > > > > - What do folks think about this? Should we put in > some > > > >>>>>>> effort > > > >>>>>>> >> > towards > > > >>>>>>> >> > > > > JDK-11 > > > >>>>>>> >> > > > > - Should we support both JDK-11 & JDK-8? > > > >>>>>>> >> > > > > - Ditch JDK-11 and directly shoot for JDK-17? > > > >>>>>>> >> > > > > > > > >>>>>>> >> > > > > Let me know your thoughts, In case anyone has some > > > >>>>>>> experience in > > > >>>>>>> >> this > > > >>>>>>> >> > > > area > > > >>>>>>> >> > > > > and have tried something in the context, feel free > to > > > >>>>>>> share or > > > >>>>>>> >> may be > > > >>>>>>> >> > > if > > > >>>>>>> >> > > > > someone has any potential action plan or so > > > >>>>>>> >> > > > > > > > >>>>>>> >> > > > > -Ayush > > > >>>>>>> >> > > > > > > > >>>>>>> >> > > > > [1] > https://issues.apache.org/jira/browse/HIVE-22415 > > > >>>>>>> >> > > > > [2] > https://issues.apache.org/jira/browse/HIVE-24484 > > > >>>>>>> >> > > > > > > > >>>>>>> >> > > > > * changed the maven.compiler.source & > > > >>>>>>> maven.compiler.target to 11 > > > >>>>>>> >> > > > > > > > >>>>>>> >> > > > > > > >>>>>>> >> > > > > > >>>>>>> >> > > > > >>>>>>> >> > > > >>>>>>> > > > > >>>>>>> > > > >>>>>> > > > > > >
