devmadhuu commented on PR #6178: URL: https://github.com/apache/ozone/pull/6178#issuecomment-1933454645
> @szetszwo @ArafatKhan2198 we've seen problems in the past with `ContainerInfo#usedBytes` going negative on the datanodes when block deletes happen. Block counts had a similar issue. We did some fixes in the past to try to get them more accurate but there may still be issues, or the issues are left over from when this cluster was running an older version. We ended up just not using these for anything requiring exact values. For example, container delete uses the `isEmpty` flag now, but container close and DU approximations still use usedBytes since those can have some margin of error. > > I checked one of our clusters with the issue and SCM sees the negative counts too so it is coming from the datanodes, not Recon. I think Recon should be tolerant of the inaccuracies and probably not log anything alarming like warn/error in this case. Thanks @errose28 for your explanation. As you mentioned that `isEmpty` flag being used in SCM now to know if container bceomes empty, however Recon have a different use case where Recon has a task to categorize containers on different size ranges based on `usedBytes`, so Recon has to rely on this attribute, but since you mentioned this attribute has still some underlying margin of error, can Recon use something else to know the actual used space in a container. Any suggestions ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
