Gargi-jais11 commented on PR #11676:
URL: https://github.com/apache/ozone/pull/11676#issuecomment-6460765693
>
- Reconciliation needs to check for a container's DELETED state before
reporting progress in logs or ICR.
- If a container is moved to DELETED during the reconciliation process,
reconciler should abort and log a message indicating this instead of a summary
of the reconciliation work. This is a similar pattern to what the container
data scanner does already.
- Reconciler should not send an ICR if the container it is working on is
deleted, since it did not end up modifying the live replica. The process that
issued the delete is responsible for the ICR. This also mirrors the scanner.
- Disk balancer needs to clear the container's last scanned timestamp and
trigger an on-demand data scan. This on-demand scan will send an ICR when it
completes.
- A copy is a new container, so information about the original is no longer
valid.
- This is similar to what container import already does.
Thanks @errose28 and @ChenSammi. I'll rework the PR with new approach
mentioned above.
Few things to note is:
The new replica's last scan timestamp is already empty after a DiskBalancer
import, because the new ContainerData is built with the copy constructor, which
doesn't carry the timestamp. So as per Ethan's approach only the scan trigger
is needed.
**One corner case which will still be left after the above approach is:**
if reconciliation finishes after the copy but before the swap (or before the
old replica is marked DELETED), the reconcile ICR reports the repaired replica.
The post-move scan won't correct SCM, because the copy's stored checksum
matches its data. Should DiskBalancer send one ICR for the new replica after
the move, as container import does?
@ChenSammi "ICR is not needed": it isn't needed for the move itself, but it
is when a reconcile report landed during the move. So on seeing the cases
should we send ICR?
Also, the on-demand scanner is single-threaded at 5 MB/s per datanode, while
DiskBalancer moves up to 10 MB/s by default, so long balancing runs will build
a scan backlog.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
