LuciferYang opened a new issue, #14291: URL: https://github.com/apache/gravitino/issues/14291
### Version main branch ### Describe what's wrong `HiveGravitinoOperationOperator` builds Hive partition names for the Spark connector with three problems: - A null partition value becomes the literal `null` (`dt=null`). Hive and Spark write a null partition value as `__HIVE_DEFAULT_PARTITION__`, so `DROP PARTITION (dt = null)` does not find a partition written by `INSERT ... SELECT ..., null`. - `ADD PARTITION (dt = null)` sends a null literal, and the server stores the string `null` as the value. On a non-string column, `SHOW PARTITIONS` then fails to convert `null`. - For a partial spec such as `SHOW PARTITIONS t PARTITION (b = 'x')`, Spark passes only the given columns, but the connector looks the columns up by position. With `PARTITIONED BY (a INT, b STRING)` it reads the string value as an INT and throws `ClassCastException`. ### Error message and/or stacktrace ``` [PARTITIONS_NOT_FOUND] The partition(s) PARTITION (`dt` = null) cannot be found in table ... java.lang.UnsupportedOperationException: Failed to convert partition value 'null' to type IntegerType java.lang.ClassCastException: class org.apache.spark.unsafe.types.UTF8String cannot be cast to class java.lang.Integer ``` ### How to reproduce ```sql CREATE TABLE t (id INT) PARTITIONED BY (dt INT) STORED AS PARQUET; INSERT INTO t SELECT 1, null; ALTER TABLE t DROP PARTITION (dt = null); -- PARTITIONS_NOT_FOUND CREATE TABLE p (id INT) PARTITIONED BY (a INT, b STRING); ALTER TABLE p ADD PARTITION (a = 1, b = 'x'); SHOW PARTITIONS p PARTITION (b = 'x'); -- ClassCastException ``` ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
