> > It is my understanding that removing the hive-metastore module is NOT > under consideration; is that correct? >
Correct, the modules under discussion here are specifically iceberg-mr and iceberg-hive3. I don't see any problems for other modules to upgrade to Hive 3. On Thu, Nov 21, 2024 at 4:01 AM Wing Yew Poon <wyp...@cloudera.com.invalid> wrote: > Also to clarify -- > It is my understanding that removing the hive-metastore module is NOT > under consideration; is that correct? > We still need a Hive version to depend on for the hive-metastore module. > In https://github.com/apache/iceberg/pull/10996, this is Hive 3. Does > this present any problem? > > > On Tue, Nov 19, 2024 at 10:26 PM Manu Zhang <owenzhang1...@gmail.com> > wrote: > >> To clarify, the changes discussed here don't affect hive connectors in >> engines, which either use the built-in hive version (Spark) or can be >> upgraded to hive 3 (Flink). >> >> On Wed, Nov 20, 2024 at 2:19 PM Manu Zhang <owenzhang1...@gmail.com> >> wrote: >> >>> Okay, let me add this option >>> >>> D. Drop Hive 2 & 3 support and suggest to use built-in Iceberg support >>> of Hive 4 >>> >>> On Wed, Nov 20, 2024 at 2:00 PM Cheng Pan <pan3...@gmail.com> wrote: >>> >>>> Hive 4 brings built-in support for Iceberg format, duplicated >>>> implementation in both sides look a redundant stuff. >>>> >>>> As Hive 2 and 3 do not support Java 11+, and Iceberg 1.8 requires Java >>>> 11+, the combination is invalid. How about simply dropping support for Hive >>>> 2&3 and suggesting the Hive user upgrade Hive 4 to gain the built-in >>>> Iceberg support? >>>> >>>> Thanks, >>>> Cheng Pan >>>> >>>> >>>> >>>> On Nov 20, 2024, at 12:47, Manu Zhang <owenzhang1...@gmail.com> wrote: >>>> >>>> Hi all, >>>> >>>> We previously reached consensus[1] to deprecate Hive 2 in 1.7 and drop >>>> in 1.8. However, when working on the removal PR[2], multiple tests failed >>>> in Hive 3 due to not supporting JDK11[3]. The fix has been back-ported to >>>> branch-3.1[4] but not released yet. As announced on Hive website, Hive 3.x >>>> is declared as End of Life so there will be no more Hive 3 release. >>>> Peter(@pvary) suggested upgrading to Hive 4 instead. On the other hand, >>>> iceberg-hive3 tests are already broken after we dropped JDK 8 support. It's >>>> not caught previously due to tests not running[6]. >>>> >>>> Based on the current situation, here are the options I can think of to >>>> move forward >>>> >>>> A. Continue to remove Hive 2 in the current PR and upgrade to Hive 4 in >>>> a separate PR. >>>> B. Hold on removing Hive 2 until we upgrade to Hive 4 >>>> C. Add source dependency[7] on Hive branch-3.1 or make a Hive 3.1 >>>> release from a forked repo. >>>> >>>> 1. https://lists.apache.org/thread/zg14b8cor4lnbyd3t4n1297y2bwb1fsg >>>> 2. https://github.com/apache/iceberg/pull/10996 >>>> 3. https://issues.apache.org/jira/browse/HIVE-21584 >>>> 4. https://github.com/apache/hive/commits/branch-3.1/ >>>> 5. https://hive.apache.org/general/downloads/ >>>> 6. https://github.com/apache/iceberg/pull/11584 >>>> 7. https://blog.gradle.org/introducing-source-dependencies >>>> >>>> Which option do you prefer? Any better alternative? >>>> >>>> Thanks, >>>> Manu >>>> >>>> >>>>
