Given that the Hive folks also leaning towards keeping the hive-runtime code in the Hive repo, I think we should move forward as Cheng Pan suggested: - Upgrade to Hive 4 - Remove hive-runtime code and tests - Make sure that a nightly build is available, so Hive folks could run integration tests, and could raise an issue if something breaks with the integration
Thanks, Peter On Thu, Nov 28, 2024, 06:46 Ajantha Bhat <ajanthab...@gmail.com> wrote: > +1 to remove support for both Hive2 and Hive3 in the latest Iceberg > release as it has reached EOL. > > Hive4 is natively managing Iceberg integration, similar to how Trino > handles its Iceberg integration. Therefore, in my opinion, it would be > better for engines to manage the integration aspect, allowing the Iceberg > community to focus on the specification and table format. > > - Ajantha > > On Thu, Nov 28, 2024 at 12:47 AM Fokko Driesprong <fo...@apache.org> > wrote: > >> Hey Cheng, >> >> Thanks for the suggestion. The nightly snapshots are available: >> https://repository.apache.org/content/groups/snapshots/org/apache/iceberg/iceberg-core/, >> which might help when working on features that are not released yet (eg >> Nanosecond timestamps). Besides that, we should run RCs against Hive to >> check if everything works as expected. >> >> I'm leaning toward removing Hive 2 and 3 as well. >> >> Kind regards, >> Fokko >> >> Op wo 27 nov 2024 om 20:05 schreef rdb...@gmail.com <rdb...@gmail.com>: >> >>> I think that we should remove Hive 2 and Hive 3. We already agreed to >>> remove Hive 2, but Hive 3 is not compatible with the project anymore and is >>> already EOL and will not see a release to update it so that it can be >>> compatible. Anyone using the existing Hive 3 support should be able to >>> continue using older releases. >>> >>> In general, I think it's a good idea to let people use older releases >>> when these situations happen. It is difficult for the project to continue >>> to support libraries that are EOL and I don't think there's a great >>> justification for it, considering Iceberg support in Hive 4 is native and >>> much better! >>> >>> On Wed, Nov 27, 2024 at 7:12 AM Cheng Pan <pan3...@gmail.com> wrote: >>> >>>> That said, it would be helpful if they continue running >>>> tests against the latest stable Hive releases to ensure that any >>>> changes don’t unintentionally break something for Hive, which would be >>>> beyond our control. >>>> >>>> >>>> I believe we should continue maintaining a Hive Iceberg runtime test >>>> suite with the latest version of Hive in the Iceberg repository. >>>> >>>> >>>> i think we can keep some basic Hive4 tests in iceberg repo >>>> >>>> >>>> Instead of running basic tests on the Iceberg repo, maybe let Iceberg >>>> publish daily snapshot jars to Nexus, and have a daily CI in Hive to >>>> consume those jars and run full Iceberg tests makes more sense? >>>> >>>> Thanks, >>>> Cheng Pan >>>> >>>>
