Re: [DISCUSS] Hive Support

2025-01-06 Thread Wing Yew Poon
FYI -- It looks like the built-in Hive version in the master branch of Apache Spark is 2.3.10 (https://issues.apache.org/jira/browse/SPARK-47018), and https://issues.apache.org/jira/browse/SPARK-44114 (upgrade built-in Hive to 3+) is an open issue. On Mon, Jan 6, 2025 at 1:07 PM Wing Yew Poon

Re: [DISCUSS] Hive Support

2025-01-06 Thread Wing Yew Poon
Hi Peter, In Spark, you can specify the Hive version of the metastore that you want to use. There is a configuration, spark.sql.hive.metastore.version, which currently (as of Spark 3.5) defaults to 2.3.9, and the jars supporting this default version are shipped with Spark as built-in. You can speci

Re: [DISCUSS] Hive Support

2024-11-25 Thread Wing Yew Poon
For the Hive runtime, would it be feasible for the Hive community to contribute a suite of tests to the Iceberg repo that can be run with dependencies from the latest Hive release (Hive 4.x), and then update them from time to time as appropriate? The purpose of this suite would be to test integrati