klsince commented on a change in pull request #7319:
URL: https://github.com/apache/pinot/pull/7319#discussion_r704813575
##########
File path:
pinot-server/src/main/java/org/apache/pinot/server/starter/helix/HelixInstanceDataManager.java
##########
@@ -311,6 +310,47 @@ private void reloadSegment(String tableNameWithType,
SegmentMetadata segmentMeta
}
}
+ @Override
+ public void addOrReplaceSegment(String tableNameWithType, String
segmentName, boolean forceDownload)
+ throws Exception {
+ LOGGER.info("Adding or replacing segment: {} for table: {}
downloadIsForced: {}", segmentName, tableNameWithType,
+ forceDownload);
+
+ // Get updated table config and segment metadata from Zookeeper.
+ TableConfig tableConfig =
ZKMetadataProvider.getTableConfig(_propertyStore, tableNameWithType);
+ Preconditions.checkNotNull(tableConfig);
+ SegmentZKMetadata zkMetadata =
+ ZKMetadataProvider.getSegmentZKMetadata(_propertyStore,
tableNameWithType, segmentName);
+ Preconditions.checkNotNull(zkMetadata);
+
+ // This method might modify the file on disk. Use segment lock to prevent
race condition
+ Lock segmentLock = SegmentLocks.getSegmentLock(tableNameWithType,
segmentName);
+ try {
+ // Lock the segment to get its metadata, so that no other threads are
modifying
+ // the disk files of this segment.
+ segmentLock.lock();
+ SegmentMetadata localMetadata = getSegmentMetadata(tableNameWithType,
segmentName);
+
+ _tableDataManagerMap.computeIfAbsent(tableNameWithType, k ->
createTableDataManager(k, tableConfig))
+ .addOrReplaceSegment(segmentName, new
IndexLoadingConfig(_instanceDataManagerConfig, tableConfig),
+ localMetadata, zkMetadata, forceDownload);
+ LOGGER.info("Added or replaced segment: {} of table: {}", segmentName,
tableNameWithType);
+ } finally {
+ segmentLock.unlock();
+ }
+ }
+
+ @Override
+ public void addOrReplaceAllSegments(String tableNameWithType, boolean
forceDownload)
+ throws Exception {
+ LOGGER.info("Adding or replacing all segments in table: {}
downloadIsForced: {}", tableNameWithType, forceDownload);
+ List<SegmentMetadata> segMds = getAllSegmentsMetadata(tableNameWithType);
+ for (SegmentMetadata segMd : segMds) {
+ addOrReplaceSegment(tableNameWithType, segMd.getName(), forceDownload);
Review comment:
after refactoring reloadSegment with forceDownload flag, there is no
need for this method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
