jarredhj0214 opened a new issue, #13549: URL: https://github.com/apache/gravitino/issues/13549
### Version main branch ### Describe what's wrong Hive Metastore may store `<derived from deserializer>` as the column type when the schema must be resolved by the table SerDe. This commonly occurs on external TextFile tables using `org.apache.hadoop.hive.serde2.OpenCSVSerde`, especially after schema alteration. Gravitino currently passes this marker directly to the Hive type parser. Since the marker is not a valid Hive type, loading the table fails. Operations that depend on loading the table, including GRANT, also fail. For affected tables, Gravitino should ask Hive Metastore to resolve the storage columns through `getFields` before converting the table schema. ### Error message and/or stacktrace ``` Error: type expected at the position 0 of '<derived from deserializer>' ``` Example affected tables: - `hive_catalog.dm.dm_vom_algo_media_play_df` - `hive_catalog.ods.ods_gio_ads_track_activation` A typical schema returned by Hive contains fields such as: ``` play_name <derived from deserializer> COMMENT 'from deserializer' ``` ### How to reproduce 1. Create an external TextFile table using `org.apache.hadoop.hive.serde2.OpenCSVSerde`. 2. Perform a schema alteration that causes Hive Metastore to persist the storage column type as `<derived from deserializer>`. 3. Load the table through a Gravitino Hive catalog. 4. Observe that table loading fails in Hive type parsing. 5. Attempting GRANT on the table fails because Gravitino cannot load it. ### Additional context Hive uses this value as a sentinel rather than an actual type. Hive Metastore's `getFields` API initializes the SerDe and returns the resolved storage columns. Partition columns are stored separately and should continue to use the existing Gravitino conversion path. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
