errose28 commented on PR #7957:
URL: https://github.com/apache/ozone/pull/7957#issuecomment-2701764640
I can see use cases for both online and offline compaction. An online tool
may be easier to use and would not have the problem of missed compactions for
snapdiff. However, it could not be used on RocksDB instances for other services
without implementing custom commands for each one. Particularly datanode and
Recon DBs would be difficult to access in this way. I suggest the following:
- We have different Jiras/PRs for OM online (`ozone repair om compact`) and
generic offline (`ozone repair ldb compact`)
- Offline compaction can work as initially proposed in this change, with a
disclaimer in the help message about OM snapshots.
- Online compaction needs some more design discussion on how to implement
the protocol:
- Should we use Ratis to get the request to all 3 OMs, should the client
send the request to all 3 automatically, or should the user have to specify the
hosts manually?
- The requirements for this command and other online repair commands are
similar to those used for dynamic reconfiguration, see
[ReconfigureProtocolClientSideTranslatorPB](https://github.com/apache/ozone/blob/194077acbb9672ca05662fca84123938cae17851/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/protocolPB/ReconfigureProtocolClientSideTranslatorPB.java#L63).
- Can we unify this into one interface for giving online commands that
are outside of Ratis and the main data path?
- Existing
[protos](https://github.com/apache/ozone/blob/3d648466ed53e784b0fd34399cadaaf53894f10b/hadoop-hdds/interface-client/src/main/proto/ReconfigureProtocol.proto)
would need to stay the same unless we want to break this in 2.0 (which might
be an option), but we might be able to do this with just client/server side
code changes and leaving the reconfig protos as they are.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
