errose28 commented on PR #11676:
URL: https://github.com/apache/ozone/pull/11676#issuecomment-6370295305
The issues mentioned here are valid. Since disk balancer does a delete of
the original replica, these issues are a subset of what HDDS-10929 intended to
cover.
Reconciliation is designed as an eventually consistent conflict-free and
lock-free process backed by a CRDT. The only requirement when reconciliation
finishes is that the system is moved to an equal or safer state. A partial or
no-op reconcile alone does not violate this, but the inaccurate state reported
to users and SCM described here does.
With this in mind, I believe we can solve all the issues mentioned here with
minor changes to disk balancer and reconciler instead of the blocking approach:
- Reconciliation needs to check for a container's DELETED state before
reporting progress in logs or ICR.
- If a container is moved to DELETED during the reconciliation process,
reconciler should abort and log a message indicating this instead of a summary
of the reconciliation work. This is a similar pattern to what the container
data scanner does already.
- Reconciler should not send an ICR if the container it is working on is
deleted, since it did not end up modifying the live replica. The process that
issued the delete is responsible for the ICR. This also mirrors the scanner.
- Disk balancer needs to clear the container's last scanned timestamp and
trigger an on-demand data scan. This on-demand scan will send an ICR when it
completes.
- A copy is a new container, so information about the original is no
longer valid.
- This is similar to what container import already does.
Let me know if there are more corner cases, but I believe we can re-use
these existing patterns from the scanner and import processes to solve the
issues here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
