yaooqinn opened a new pull request, #50182:
URL: https://github.com/apache/spark/pull/50182
### What changes were proposed in this pull request?
```
25/03/06 08:25:17 WARN HiveExternalCatalog: Hive incompatible types found:
timestamp_ntz. Persisting data source table `spark_catalog`.`default`.`c` into
Hive metastore in Spark SQL specific format, which is NOT compatible with Hive.
org.apache.spark.sql.AnalysisException:
org.apache.hadoop.hive.ql.metadata.HiveException:
InvalidObjectException(message:Invalid partition column type: timestamp_ntz)
```
The partition columns are duplicated and stored both in the HMS column meta
and the table properties. If they contain incompatible data types, the HMS Meta
API will fail the process.
We can rely on the table properties to read/write
### Why are the changes needed?
bugfix, otherwise, newly added spark data types are not able to be used as
partition columns
### Does this PR introduce _any_ user-facing change?
Yes, More type cases are supported for partitioned datasouce tables stored
in HMS
### How was this patch tested?
new tests
### Was this patch authored or co-authored using generative AI tooling?
no
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
