deadlinefen opened a new pull request, #22447: URL: https://github.com/apache/doris/pull/22447
## Proposed changes GC BE binlog metas when tablet is dropped. Add a timed GC binlog metas task in the timed GC task of BE. When cleaning, it will traverse all keys starting with binlog_meta_ in the metadata, and get its corresponding rowset meta and check whether the tablet belonging to the rowset meta exists. If it does not exist (that is, it has been dropped), clean up all binlog_meta and binlog_data corresponding to this tablet. In the implementation details, because the key of rocksdb is internally ordered, the same tablet will be traversed continuously, so if the tablet uuid obtained this time is the same as the last time, you can skip the check directly and execute the last operation (cleaned up or skipped). At the same time, the deletion of binlog_meta and binlog_data corresponding to the same tablet-rowset must be in the same transaction, in order to ensure that metadata leakage does not occur. ## Note If there are existing binlog meta and tombstone in the cluster, they need to be GCed before upgrading to this version. This upgrade is not a smooth upgrade. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
