aishikbh commented on code in PR #13232:
URL: https://github.com/apache/pinot/pull/13232#discussion_r1618179836
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/retention/RetentionManager.java:
##########
@@ -55,7 +55,7 @@
*/
public class RetentionManager extends ControllerPeriodicTask<Void> {
public static final long OLD_LLC_SEGMENTS_RETENTION_IN_MILLIS =
TimeUnit.DAYS.toMillis(5L);
- private static final RetryPolicy DEFAULT_RETRY_POLICY =
RetryPolicies.exponentialBackoffRetryPolicy(5, 1000L, 2.0f);
+ private static final RetryPolicy DEFAULT_RETRY_POLICY =
RetryPolicies.randomDelayRetryPolicy(20, 100L, 200L);
Review Comment:
The reason for changing the retry policy is simply to make it more resilient
along with serialising the update for idealState during deletion, currently
when a lot of segments are being uploaded and the retention manger is trying to
delete segments, the the retry is applied 5 times using exponential backoff and
it fails.
For example :
```
2024/05/16 01:59:39.041 ERROR [RetentionManager] [pool-20-thread-4] Failed
to clean up the segment lineage. (tableName = tableXXX_OFFLINE)
org.apache.pinot.spi.utils.retry.AttemptsExceededException: Operation failed
after 5 attempts
at
org.apache.pinot.spi.utils.retry.BaseRetryPolicy.attempt(BaseRetryPolicy.java:65)
~[startree-pinot-all-1.2.0-ST.10.1-jar-with-dependencies.jar:1.2.0-ST.10.1-8711a0aa760c824774f3ceb1a2913878a31071ec]
at
org.apache.pinot.controller.helix.core.retention.RetentionManager.manageSegmentLineageCleanupForTable(RetentionManager.java:195)
~[startree-pinot-all-1.2.0-ST.10.1-jar-with-dependencies.jar:1.2.0-ST.10.1-8711a0aa760c824774f3ceb1a2913878a31071ec]
at
org.apache.pinot.controller.helix.core.retention.RetentionManager.processTable(RetentionMan
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
