priyeshkaratha opened a new pull request, #11855:
URL: https://github.com/apache/ozone/pull/11855
## What changes were proposed in this pull request?
Recon's `/pendingDeletion?component=om` currently reports one pending size
from Recon's own OM DB copy. That number can't separate deleted data OM can
actually purge from data still held by snapshots. HDDS-17502 added the
`GetBucketDeletedBytes` OM API, which returns the purgeable vs. snapshot-held
split per bucket. This PR uses that API to report OM pending deletion in Recon.
- Adds `BucketDeletedBytesMetricsService`. It lists the buckets from Recon's
OM DB copy and then sends one `getBucketDeletedBytes` admin RPC per bucket to
OM, in parallel on a bounded thread pool. It sums purgeable and snapshot-held
bytes, purgeable key/dir counts and the purgeable key/dir byte split. It is
rate-limited and has a timeout, and any bucket whose RPC fails or times out is
counted in `totalBucketQueriesFailed`.
- `/pendingDeletion?component=om` now reads from this service. It returns
`200` with the summed totals once collection has finished, and `202` with a
status/message while collection is in progress or after it failed.
- `/buckets` now returns per-bucket `trappedBytes` / `trappedKeys` (`-1`
when not yet known) and a `trappedBytesCollectionStatus`.
- If an older OM doesn't report the key/dir byte split, the split totals are
reported as `-1`.
- Adds new configs (in `ozone-default.xml`):
- `ozone.recon.bucket.deleted.bytes.collection.minimum.api.delay` (default
`30s`)
- `ozone.recon.bucket.deleted.bytes.collection.timeout` (default `10m`)
- `ozone.recon.bucket.deleted.bytes.collection.thread.count` (default `0`
= 2 × available processors)
## What is the link to the Apache JIRA
HDDS-17503
## How was this patch tested?
Added test cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
