yuqi1129 commented on PR #6488: URL: https://github.com/apache/gravitino/pull/6488#issuecomment-2674402379
> My concern is that creating and closing the fs each time will introduce unnecessary overheads. I remembered that 1: Hadoop FS has cache mechanism. 2: If we don't leverage Hadoop's cache, shall we maintain it on our own, so that we can close them when the catalog is closed? No need to close the fs each time. What do you think? > Hadoop FS has cache mechanism Since we would support FileSystem credentials, each different fileset(name identifier) will have a different filesystem instance. > If we don't leverage Hadoop's cache, shall we maintain it on our own, so that we can close them when the catalog is closed? In the GVFS, we do have a cache to manage file system instances and the key is `nameIdentier` and disable cache in the Filesystem level. For the Gravitino server side, there is no cache, so I'm inclined to close it to free up memory quickly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@gravitino.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
