FrankChen021 commented on code in PR #19374:
URL: https://github.com/apache/druid/pull/19374#discussion_r3154599816
##########
server/src/main/java/org/apache/druid/server/http/DataSourcesResource.java:
##########
@@ -918,11 +918,24 @@ public Response isHandOffComplete(
final Interval theInterval = Intervals.of(interval);
final SegmentDescriptor descriptor = new SegmentDescriptor(theInterval,
version, partitionNumber);
final DateTime now = DateTimes.nowUtc();
+ // Look up the segment in the metadata snapshot so the rule cascade can
be evaluated against the real segment
+ // (necessary for partial load rules whose matcher inspects the
segment's projection list). If the cached
+ // snapshot is missing the segment, force a refresh and re-check before
declaring it never-handed-off, since
+ // the cache may simply not have caught up to a recent publish.
+ final SegmentId segmentId = SegmentId.of(dataSourceName, theInterval,
version, partitionNumber);
+ DataSegment segment =
lookupSegment(segmentsMetadataManager.getRecentDataSourcesSnapshot(),
segmentId);
+ if (segment == null) {
+ segment =
lookupSegment(segmentsMetadataManager.forceUpdateDataSourcesSnapshot(),
segmentId);
Review Comment:
[P2] Avoid blocking handoff checks on every metadata miss
When the recent snapshot does not contain the segment, this endpoint now
calls forceUpdateDataSourcesSnapshot() before returning true. In the v2
incremental metadata manager that call waits for the next cache sync with a
timeout of 2 * druid.manager.segments.pollDuration, so a genuinely absent or
removed segment can make each handoffComplete request block for up to the poll
interval before returning the same true result. Realtime handoff polling is
synchronous and iterates callbacks sequentially, so one missing segment can
delay completion checks for others. Consider avoiding the forced refresh on
every miss, using a cheaper targeted check, or bounding this path so absent
segments still complete promptly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
