bharos commented on issue #5574: URL: https://github.com/apache/gravitino/issues/5574#issuecomment-3336780476
I feel this has two parts: One is "When we do the migration of the metadata service at the bottom of iceberg, users should not be aware of it, and they should not be asked to modify the iceberg catalog name in their sql, otherwise it will be very difficult to implement." For this one, can't we solve it in spark using the SparkSessionCatalog, which can point to Gravitino IRC, and the fallback connector will point to metastore, and users don't need to specify a catalog name. And the other part is incremental/migration behind the scenes transparent to the users. For this part is where the "multi-backend" catalog support helps in my opinion, so that tables can be migrated to primary catalog from secondary when they are being read/written to WDYT cc @FANNG1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
