Cheng, You are right that Spark's IsolatedClientLoader is only used by its HiveExternalCatalog. In any case, that only adds to the argument to continue supporting iceberg-hive-metastore on Hive 2. In yesterday's community sync, there seems to be agreement to support publishing iceberg-hive-metastore jars that are built against different versions of Hive, and to remove the iceberg-hive-metastore classes from the runtime jars (for engines). However, Ryan Blue suggested that the latter step should only be done in an Iceberg 2.0 release, as it would be a breaking change for users. - Wing Yew
On Tue, May 20, 2025 at 7:19 PM Cheng Pan <pan3...@gmail.com> wrote: > > Spark can be configured to use a different version > > It’s not true. IsolatedClassLoader only supports Spark Hive session > catalog and Iceberg always uses Spark built-in Hive jars, namely, Hive > 2.3.9 for Spark 3.x and Hive 2.3.10 for Spark 4.0 > > Thanks, > Cheng Pan > > > > On May 21, 2025, at 03:06, Denys Kuzmenko <dkuzme...@apache.org> wrote: > > I don't believe continuing to build iceberg-hive-metastore jars for Hive 2 > is well justified. > Hive 2 is deprecated and has known security vulnerabilities, which should > encourage users to upgrade. > As you pointed out, Spark can be configured to use a different version, > but it's ultimately up to the Iceberg community to make that decision. > > >
