jarredhj0214 opened a new pull request, #13550: URL: https://github.com/apache/gravitino/pull/13550
### What changes were proposed in this pull request? Detect Hive storage columns whose type is `<derived from deserializer>` and resolve them through Hive Metastore's `getFields` API before converting the table schema. - Use `getFields(database, table)` for Hive 2. - Use the catalog-aware `getFields(catalog, database, table)` for Hive 3. - Replace only the in-memory storage columns returned by `getTable`. - Keep the existing partition-column handling unchanged. - Report a clear error if Hive Metastore still returns unresolved types. The additional HMS request is made only for affected tables. Loading ordinary Hive tables does not add an extra RPC. ### Why are the changes needed? `<derived from deserializer>` is a Hive sentinel and is not a valid Hive type. Passing it to Gravitino's type parser causes `loadTable` to fail, which also blocks operations such as GRANT. Fix: #13549 ### Does this PR introduce _any_ user-facing change? Yes. Hive tables with SerDe-derived columns can now be loaded when Hive Metastore can resolve their schema. This also allows dependent authorization operations to proceed. There are no API or configuration changes, and normal Hive tables retain the existing behavior. ### How was this patch tested? Added unit tests covering: - Normal Hive 2 tables skip field resolution. - Hive 2 derived columns are resolved through `getFields`. - Hive 3 uses the catalog-aware `getFields` overload. - Unresolved SerDe-derived types produce a clear error. Commands executed: - `./gradlew :catalogs:hive-metastore-common:test :catalogs:hive-metastore3-libs:test -PskipITs` - `./gradlew :catalogs:catalog-hive:test -PskipITs` - `./gradlew :catalogs:hive-metastore-common:spotlessApply :catalogs:hive-metastore3-libs:spotlessApply` All tests passed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
