Hi all, I would like to confirm that, with the current Hive 2.3.6 SNAPSHOT, Spark can pass all tests with JDK 8 and JDK 11 with each build. Therefore, if this was only the one that blocks Hive 2.3.6, it should be good to release now.
Here are official CI build/test results against JDK 11: https://github.com/apache/spark/pull/25443#issuecomment-522295117 https://github.com/apache/spark/pull/25443#issuecomment-522235276 https://github.com/apache/spark/pull/25443#issuecomment-522206143 I checked with other Spark people in case - https://github.com/apache/spark/pull/25443#issuecomment-522207828. There are some more minor changes to fix but they are specific to Spark itself and looks fix-able. 2019년 8월 14일 (수) 오전 10:03, Hyukjin Kwon <gurwls...@gmail.com>님이 작성: > Thank you so much, Alan. > > For a quick followup update, seems this is available at > https://repository.apache.org/content/repositories/orgapachehive-1096/ > FYI, Spark CI is almost ready to trigger JDK 11 in PR. Should be able to > verify and check the official CI test results. > > > 2019년 8월 14일 (수) 오전 5:23, Alan Gates <alanfga...@gmail.com>님이 작성: > >> I've committed the patch for Hive 22096 and created a snapshot for Hive >> 2.3.6-rc0, which should be in the Apache repo. Let me know if this works, >> and if so I'll push out the 2.3.6 release. >> >> Alan. >> >> On Sun, Aug 11, 2019 at 10:28 PM Hyukjin Kwon <gurwls...@gmail.com> >> wrote: >> >>> I sincerely appreciate, Alan, for a quick positive response. >>> Thanks Yuming for quickly starting to work on that. >>> >>> For quick updates: >>> - Seems Yuming already started to investigate it at >>> https://issues.apache.org/jira/browse/HIVE-22096 - I will try to reach >>> him and get synced. >>> (He works in a different company and lives in a different country >>> from me so some delays are expected) >>> - Spark side, it's being investigated under >>> https://issues.apache.org/jira/browse/SPARK-28684 >>> - .. and .. actually I am now synced with Yuming, almost of all tests >>> pass only with HIVE-22096 >>> >>> Alan, would you mind if I ask to make a snapshot after HIVE-22096 fix if >>> it makes sense? >>> >>> Sounds like, in his local, almost all Spark test cases pass except one >>> test case with HIVE-22096 (with JDK 11). >>> Looks like he wants to verify via Spark CI as well to show official test >>> results to show up which can be easily done if there's a snapshot. >>> >>> >>> 2019년 8월 10일 (토) 오전 6:35, Alan Gates <alanfga...@gmail.com>님이 작성: >>> >>>> I think it's fine to backport this and do a release. Are we sure >>>> that's enough to make it run on JDK11? As noted in the bug there isn't an >>>> umbrella issue to make it JDK11 compatible. I don't know if anyone has >>>> tested Hive 2 on JDK11 or not. >>>> >>>> Are you available to do the backport? If so, and we don't find any >>>> other JDK11 related issues, I can create a 2.3.6 release once you're done. >>>> >>>> Alan. >>>> >>>> On Thu, Aug 8, 2019 at 6:23 PM Hyukjin Kwon <gurwls...@gmail.com> >>>> wrote: >>>> >>>>> Hi all, >>>>> >>>>> >>>>> I am from Spark dev and had a question about feasibility of porting >>>>> HIVE-21584 >>>>> and minor release 2.3.x. >>>>> >>>>> Just to share full context, please take a look at >>>>> >>>>> https://issues.apache.org/jira/browse/HIVE-21639?focusedCommentId=16822802&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16822802 >>>>> TL;DR: with Hive 2.3.5, Spark is now very close to get through the >>>>> complicated >>>>> situation that has been locked for years. Thank you Yuming Wang and >>>>> Alan Gates >>>>> - it would have been impossible without all the efforts from you guys. >>>>> >>>>> >>>>> One problem found lately though, >>>>> Spark is trying to support JDK 11 but this seems blocked by >>>>> HIVE-21584. So, I and Spark >>>>> community are trying to find a way to get through. >>>>> >>>>> Firstly, I (and presumably some Spark community guys) thought Spark >>>>> should find its >>>>> own workaround or try to upgrade Hive to 4.0.0 in the upcoming release. >>>>> This possibility was checked and seems difficult. >>>>> It was already pretty a radical change to upgrade Hive 1.2.x to 2.3.5, >>>>> seems it's difficult to target upgrade to 4.0.0 in this release at >>>>> this moment. >>>>> >>>>> I understand HIVE-21584 was fixed in 4.0.0 and it might be difficult >>>>> to backport >>>>> through branch-3.x and 2.x; however, wanted to at least ask and now >>>>> because >>>>> I thought porting HIVE-21584 into Hive 2.3.x branch is an option that >>>>> needs the >>>>> minimised efforts to permanently resolve all related issues, >>>>> apparently, blocked >>>>> for multiple years .. >>>>> >>>>> >>>>> Thanks for consideration in advance. >>>>> >>>>
