peterxcli commented on code in PR #8360:
URL: https://github.com/apache/ozone/pull/8360#discussion_r2066180844
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/DownloadAndImportReplicator.java:
##########
@@ -81,15 +81,6 @@ public void replicate(ReplicationTask task) {
try {
targetVolume = containerImporter.chooseNextVolume();
- // Increment committed bytes and verify if it doesn't cross the space
left.
- targetVolume.incCommittedBytes(containerSize * 2);
Review Comment:
This step(increase commit space the check usable space) has been done in the
VolumeChoosingPolicy#chooseVolume, so removed, too
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/SendContainerRequestHandler.java:
##########
@@ -87,16 +86,6 @@ public void onNext(SendContainerRequest req) {
if (containerId == -1) {
containerId = req.getContainerID();
volume = importer.chooseNextVolume();
- // Increment committed bytes and verify if it doesn't cross the space
left.
- volume.incCommittedBytes(importer.getDefaultContainerSize() * 2);
- StorageLocationReport volumeReport = volume.getReport();
- // Already committed bytes increased above, so required space is not
required here in AvailableSpaceFilter
- if (volumeReport.getUsableSpace() <= 0) {
Review Comment:
This step(increase commit space the check usable space) has been done in the
`VolumeChoosingPolicy#chooseVolume`, so removed
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/replication/ContainerImporter.java:
##########
@@ -106,11 +106,6 @@ public void importContainer(long containerID, Path
tarFilePath,
ContainerProtos.Result.CONTAINER_EXISTS);
}
- HddsVolume targetVolume = hddsVolume;
- if (targetVolume == null) {
Review Comment:
Not sure if we should some `Precondition` here to make sure the volume won't
be null.
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/ozoneimpl/ContainerReader.java:
##########
@@ -202,50 +202,52 @@ public void verifyAndFixupContainerData(ContainerData
containerData)
throws IOException {
switch (containerData.getContainerType()) {
case KeyValueContainer:
- if (containerData instanceof KeyValueContainerData) {
- KeyValueContainerData kvContainerData = (KeyValueContainerData)
- containerData;
- containerData.setVolume(hddsVolume);
- KeyValueContainerUtil.parseKVContainerData(kvContainerData, config);
- KeyValueContainer kvContainer = new KeyValueContainer(kvContainerData,
- config);
- if (kvContainer.getContainerState() == RECOVERING) {
- if (shouldDelete) {
- // delete Ratis replicated RECOVERING containers
- if (kvContainer.getContainerData().getReplicaIndex() == 0) {
- cleanupContainer(hddsVolume, kvContainer);
- } else {
- kvContainer.markContainerUnhealthy();
- LOG.info("Stale recovering container {} marked UNHEALTHY",
- kvContainerData.getContainerID());
- containerSet.addContainer(kvContainer);
- }
- }
- return;
- }
- if (kvContainer.getContainerState() == DELETED) {
- if (shouldDelete) {
- cleanupContainer(hddsVolume, kvContainer);
- }
- return;
- }
- try {
- containerSet.addContainer(kvContainer);
- } catch (StorageContainerException e) {
- if (e.getResult() != ContainerProtos.Result.CONTAINER_EXISTS) {
- throw e;
- }
- if (shouldDelete) {
- resolveDuplicate((KeyValueContainer) containerSet.getContainer(
- kvContainer.getContainerData().getContainerID()), kvContainer);
- }
- }
- } else {
+ if (!(containerData instanceof KeyValueContainerData)) {
throw new StorageContainerException("Container File is corrupted. " +
"ContainerType is KeyValueContainer but cast to " +
"KeyValueContainerData failed. ",
ContainerProtos.Result.CONTAINER_METADATA_ERROR);
}
+
+ KeyValueContainerData kvContainerData = (KeyValueContainerData)
+ containerData;
+ containerData.setVolume(hddsVolume);
+ KeyValueContainerUtil.parseKVContainerData(kvContainerData, config);
+ KeyValueContainer kvContainer = new KeyValueContainer(kvContainerData,
+ config);
+ if (kvContainer.getContainerState() == RECOVERING) {
+ if (shouldDelete) {
+ // delete Ratis replicated RECOVERING containers
+ if (kvContainer.getContainerData().getReplicaIndex() == 0) {
+ cleanupContainer(hddsVolume, kvContainer);
+ } else {
+ kvContainer.markContainerUnhealthy();
+ LOG.info("Stale recovering container {} marked UNHEALTHY",
+ kvContainerData.getContainerID());
+ containerSet.addContainer(kvContainer);
+ }
+ }
+ return;
+ } else if (kvContainer.getContainerState() == DELETED) {
+ if (shouldDelete) {
+ cleanupContainer(hddsVolume, kvContainer);
+ }
+ return;
+ }
+
+ try {
+ containerData.commitSpace();
Review Comment:
Sorry this diff may hard to read, but I'm doing here is just:
1. Move the `containerData` instance check up to reduce the indent level of
the block.
2. Commit space before adding container. **But I have one question here,
IIRC, only containers in RECOVERING or OPEN will receive new write, so would it
be better to only commit the space for RECOVERING/OPEN containers?**
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@ozone.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@ozone.apache.org
For additional commands, e-mail: issues-h...@ozone.apache.org
