morningman commented on code in PR #18102:
URL: https://github.com/apache/doris/pull/18102#discussion_r1147777671
##########
docs/zh-CN/docs/lakehouse/multi-catalog/hive.md:
##########
@@ -152,6 +152,22 @@ CREATE CATALOG hive WITH RESOURCE hms_resource PROPERTIES(
);
```
+在 2.0 版本之后，我们可以在创建 Catalog 时可以采用参数 `file.meta.cache.ttl-second` 来设置 File Cache
自动失效时间，也可以将该值设置为 0 来禁用 File Cache。时间单位为：秒。示例如下：
Review Comment:
```
<version since="dev"></version>
创建 Catalog 时可以采用参数 `file.meta.cache.ttl-second` 来设置 File Cache
自动失效时间，也可以将该值设置为 0 来禁用 File Cache。时间单位为：秒。示例如下：
```
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HiveMetaStoreCache.java:
##########
@@ -120,14 +121,22 @@ public HivePartition load(PartitionCacheKey key) throws
Exception {
}
}, executor));
- fileCache =
CacheBuilder.newBuilder().maximumSize(Config.max_external_file_cache_num)
-
.expireAfterAccess(Config.external_cache_expire_time_minutes_after_access,
TimeUnit.MINUTES)
- .build(CacheLoader.asyncReloading(new
CacheLoader<FileCacheKey, ImmutableList<InputSplit>>() {
- @Override
- public ImmutableList<InputSplit> load(FileCacheKey key)
throws Exception {
- return loadFiles(key);
- }
- }, executor));
+ // if the file.meta.cache.ttl-second is equal or greater than 0, the
cache expired will be set to that value
+ int fileMetaCacheTtlSecond = NumberUtils.toInt(
+
(catalog.getProperties().getOrDefault(HMSExternalCatalog.FILE_META_CACHE_TTL_SECOND,
"")), -1);
+
+ CacheBuilder<Object, Object> fileCacheBuilder =
CacheBuilder.newBuilder().maximumSize(Config.max_external_file_cache_num);
+ if (fileMetaCacheTtlSecond >= 0) {
+ fileCacheBuilder.expireAfterWrite(fileMetaCacheTtlSecond,
TimeUnit.SECONDS);
+ }
Review Comment:
We also need to let user disable the file cache.
Maybe we can use `0` as a special value?
`0` means file cache is disabled.
`>0` means file cache with ttl
`-1` means file cache no ttl
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
