github-actions[bot] commented on code in PR #65625:
URL: https://github.com/apache/doris/pull/65625#discussion_r3585091295
##########
fe/fe-common/src/main/java/org/apache/doris/common/Config.java:
##########
@@ -3071,7 +3071,7 @@ public static int metaServiceRpcRetryTimes() {
public static int drop_rpc_retry_num = 200;
@ConfField
- public static int default_get_version_from_ms_timeout_second = 3;
+ public static int default_get_version_from_ms_timeout_second = 30;
Review Comment:
This value is now large enough to consume the cloud MoW delete-bitmap lock
lifetime before the commit reaches delete-bitmap calculation. In
`CloudGlobalTransactionMgr.commitTransactionWithoutLock()` and the
sub-transaction path, FE first gets the delete-bitmap update lock, whose
default expiration is 60s, and only then calls `getCalcDeleteBitmapInfo()`.
That method fetches partition versions via
`CloudPartition.getSnapshotVisibleVersionFromMs(...)`, so with this PR two slow
get-version attempts can spend roughly 60s before `sendCalcDeleteBitmaptask()`
even starts. During that window the MetaService lock can expire and another
transaction can take it, causing the current delete-bitmap update/commit to
fail and retry under load. Please either fetch the needed versions before
acquiring the delete-bitmap lock, refresh/revalidate the lock after the version
RPC, or use a much smaller total get-version deadline once this lock is held.
##########
fe/fe-common/src/main/java/org/apache/doris/common/Config.java:
##########
@@ -3071,7 +3071,7 @@ public static int metaServiceRpcRetryTimes() {
public static int drop_rpc_retry_num = 200;
@ConfField
- public static int default_get_version_from_ms_timeout_second = 3;
+ public static int default_get_version_from_ms_timeout_second = 30;
Review Comment:
This new 30s default also applies to `SHOW PARTITIONS` / partitions TVF in
cloud mode while metadata read locks are held.
`PartitionsProcDir.getPartitionInfosInrernal()` takes
`MetaLockUtils.readLockTables(needLocked)` and only unlocks in the final
`finally`, but inside that scope it calls `getPartitionVersions()`. In cloud
mode that reaches `OlapTable.getVisibleVersionFromMeta()` /
`CloudPartition.getSnapshotVisibleVersion*()`, which route through
`VersionHelper.getVisibleVersion()` and can wait this timeout on each of the
default 200 retries. A slow MetaService can therefore hold the table read lock
for the enlarged retry budget, blocking DDL or other writers; this is exactly
the RPC-under-metadata-lock pattern the FE core guide tells us to avoid. Please
snapshot the needed partition metadata under the lock and perform the
MetaService version RPC after releasing it, or use a short bounded timeout for
this lock-held path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
