Oh, I think I see. The upgrade to Hive 4 is just for the Hive metastore support? When I read the thread, I thought that we weren't going to change the metastore. That seems reasonable to me. Sorry for the confusion.
On Fri, Dec 13, 2024 at 10:24 AM rdb...@gmail.com <rdb...@gmail.com> wrote: > Sorry, I must have missed something. I don't think that we should upgrade > anything in Iceberg to Hive 4. Why not simply remove the Hive support > entirely? Why would anyone need Hive 4 support from Iceberg when it is > built into Hive 4? > > On Thu, Dec 12, 2024 at 11:03 AM Daniel Weeks <dwe...@apache.org> wrote: > >> Hey Manu, >> >> I agree with the direction here, but we should probably hold a quick >> procedural vote just to confirm since this is a significant change in >> support for Hive. >> >> -Dan >> >> On Wed, Dec 11, 2024 at 5:19 PM Manu Zhang <owenzhang1...@gmail.com> >> wrote: >> >>> Thanks all for sharing your thoughts. It looks there's a consensus on >>> upgrading to Hive 4 and dropping hive-runtime. >>> I've submitted a PR[1] as the first step. Please help review. >>> >>> 1. https://github.com/apache/iceberg/pull/11750 >>> >>> Thanks, >>> Manu >>> >>> On Thu, Nov 28, 2024 at 11:26 PM Shohei Okumiya <oku...@apache.org> >>> wrote: >>> >>>> Hi all, >>>> >>>> I also prefer option 1. I have some initiatives[1] to improve >>>> integrations between Hive and Iceberg. The current style allows us to >>>> develop both Hive's core and HiveIcebergStorageHandler simultaneously. >>>> That would help us enhance integrations. >>>> >>>> - [1] https://issues.apache.org/jira/browse/HIVE-28410 >>>> >>>> Regards, >>>> Okumin >>>> >>>> On Thu, Nov 28, 2024 at 4:17 AM Fokko Driesprong <fo...@apache.org> >>>> wrote: >>>> > >>>> > Hey Cheng, >>>> > >>>> > Thanks for the suggestion. The nightly snapshots are available: >>>> https://repository.apache.org/content/groups/snapshots/org/apache/iceberg/iceberg-core/, >>>> which might help when working on features that are not released yet (eg >>>> Nanosecond timestamps). Besides that, we should run RCs against Hive to >>>> check if everything works as expected. >>>> > >>>> > I'm leaning toward removing Hive 2 and 3 as well. >>>> > >>>> > Kind regards, >>>> > Fokko >>>> > >>>> > Op wo 27 nov 2024 om 20:05 schreef rdb...@gmail.com <rdb...@gmail.com >>>> >: >>>> >> >>>> >> I think that we should remove Hive 2 and Hive 3. We already agreed >>>> to remove Hive 2, but Hive 3 is not compatible with the project anymore and >>>> is already EOL and will not see a release to update it so that it can be >>>> compatible. Anyone using the existing Hive 3 support should be able to >>>> continue using older releases. >>>> >> >>>> >> In general, I think it's a good idea to let people use older >>>> releases when these situations happen. It is difficult for the project to >>>> continue to support libraries that are EOL and I don't think there's a >>>> great justification for it, considering Iceberg support in Hive 4 is native >>>> and much better! >>>> >> >>>> >> On Wed, Nov 27, 2024 at 7:12 AM Cheng Pan <pan3...@gmail.com> wrote: >>>> >>> >>>> >>> That said, it would be helpful if they continue running >>>> >>> tests against the latest stable Hive releases to ensure that any >>>> >>> changes don’t unintentionally break something for Hive, which would >>>> be >>>> >>> beyond our control. >>>> >>> >>>> >>> >>>> >>> I believe we should continue maintaining a Hive Iceberg runtime >>>> test suite with the latest version of Hive in the Iceberg repository. >>>> >>> >>>> >>> >>>> >>> i think we can keep some basic Hive4 tests in iceberg repo >>>> >>> >>>> >>> >>>> >>> Instead of running basic tests on the Iceberg repo, maybe let >>>> Iceberg publish daily snapshot jars to Nexus, and have a daily CI in Hive >>>> to consume those jars and run full Iceberg tests makes more sense? >>>> >>> >>>> >>> Thanks, >>>> >>> Cheng Pan >>>> >>> >>>> >>>
