github-actions[bot] commented on code in PR #68842:
URL: https://github.com/apache/doris/pull/68842#discussion_r4234105307
##########
fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergCatalogFactory.java:
##########
@@ -161,14 +163,17 @@ public static Map<String, String>
buildBaseCatalogProperties(Map<String, String>
* {@code io.manifest.cache-enabled} directly — derive it to {@code
"true"} from the FE meta-cache
* spec ({@code meta.cache.iceberg.manifest.*}) using the same {@code
enable && ttl != 0 &&
* capacity != 0} rule. Default-disabled (legacy {@code
DEFAULT_ICEBERG_MANIFEST_CACHE_ENABLE}).
+ * Nothing is derived under a metadata cache weight limit: the SDK cache
keeps up to
+ * {@code io.manifest.cache.max-total-bytes} of manifest content per
FileIO outside that limit.
*/
- private static void appendManifestCacheProperties(Map<String, String>
props, Map<String, String> opts) {
+ private static void appendManifestCacheProperties(Map<String, String>
props, Map<String, String> opts,
+ boolean metaCacheWeightLimited) {
boolean hasExplicitEnabled =
StringUtils.isNotBlank(props.get(CatalogProperties.IO_MANIFEST_CACHE_ENABLED));
copyIfPresent(props, opts,
CatalogProperties.IO_MANIFEST_CACHE_ENABLED);
copyIfPresent(props, opts,
CatalogProperties.IO_MANIFEST_CACHE_EXPIRATION_INTERVAL_MS);
copyIfPresent(props, opts,
CatalogProperties.IO_MANIFEST_CACHE_MAX_TOTAL_BYTES);
copyIfPresent(props, opts,
CatalogProperties.IO_MANIFEST_CACHE_MAX_CONTENT_LENGTH);
- if (!hasExplicitEnabled) {
+ if (!hasExplicitEnabled && !metaCacheWeightLimited) {
Review Comment:
[P2] Keep the SDK manifest cache off after REST config merging. With a Doris
weight limit and no explicit `io.manifest.cache-enabled`, this branch omits the
key, but an Iceberg REST `/config` default of `true` fills it in
`ConfigResponse.merge`. `RESTSessionCatalog` then initializes its FileIO with
that value, and `ManifestFiles` can keep up to 100 MB of manifest content per
FileIO outside the Doris limit. Enforce `false` on the effective FileIO
properties, or reject a server-supplied `true`, unless Doris explicitly opted
in; setting only the client map would still lose to server overrides or
per-table config.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
