pan3793 commented on PR #46521: URL: https://github.com/apache/spark/pull/46521#issuecomment-2104175525
Hmm, let me clear my view. In short, I think the current CI is sufficient. Spark uses Hive in two cases: 1. As an HMS client. To support different HMS versions, it allows to use IsolatedClassLoder to load a different Hive class. It calls `Hive.getWithoutRegisterFns(hiveConf)` or `Hive.get(hiveConf)` to create the Hive instance, and there is a chance to trigger the Hive built-in UDF registration, for older Hive, e.g. 2.1.1, some built-in Hive UDF may trigger CodeHaus Jackson classes loading. 2. As an execution library. Spark always used the built-in Hive jars to read/write Hive tables, execute Hive UDFs. For case 1, the CI already covers that(any older HMS client initialization triggers built-in UDF registration). For case 2, there is no chance to invoke CodeHaus Jackson classes since Hive 2.3.10 totally removed it in the codebase. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
