morningman opened a new issue, #17413: URL: https://github.com/apache/doris/issues/17413
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Description ## Background Doris supports [multi catalog ](https://doris.apache.org/docs/dev/lakehouse/multi-catalog/) to connect to external datasource. For any catalog, Doris will sync the metadata, such as database, table, schema, partition, file from external datasource into FE's memory, so that when accessing the catalog, we don' t need to access the remote data source' meta service. But it is a one-time operation, after sync, if the external data source change its metadata, such as adding new table, partition, Doris can't be aware of it and the newly created metadata can not be accessed from Doris. So we need a way refresh the metadata of the catalog, to sync the metadata at fix interval. ## How to First, we should set new properties of catalog when creating it, eg: ``` create catalog hive properties ( ... "metadata_refresh_interval_sec" = "7200" ... ); ``` which means for this `hive` catalog, its metadata will be refreshed every 7200s. And there will be a daemon thread on Master FE to do this for all catalogs. ## Tips 1. Related classes: See `makeSureInitialized()` of `ExternalCatalog` class to learn how to refresh catalog. 2. See [Refresh](https://doris.apache.org/docs/dev/sql-manual/sql-reference/Utility-Statements/REFRESH) command to learn how to refresh catalog manually. ### Use case _No response_ ### Related issues _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
