github-actions[bot] commented on code in PR #64560:
URL: https://github.com/apache/doris/pull/64560#discussion_r3457209506
##########
be/src/storage/rowset/beta_rowset_writer.cpp:
##########
@@ -913,6 +948,27 @@ Status BetaRowsetWriter::_wait_flying_segcompaction() {
return Status::OK();
}
+Status BetaRowsetWriter::_finish_flying_segcompaction(bool
need_final_segcompaction_retry) {
+ if (need_final_segcompaction_retry) {
+ RETURN_NOT_OK_STATUS_WITH_WARN(_wait_flying_segcompaction(),
+ "segcompaction failed when build new
rowset");
+ RETURN_NOT_OK_STATUS_WITH_WARN(_segcompaction_if_necessary(),
+ "segcompaction failed when build new
rowset");
Review Comment:
The final retry still only catches up one segcompaction batch after the
delete-bitmap token has drained. If the new bitmap gate blocked every
`add_segment()` trigger while segment 0 was still pending,
`_finish_flying_segcompaction(true)` runs `_segcompaction_if_necessary()` once,
which can compact at most `segcompaction_batch_size` segments (default 10), and
then `_segcompaction_rename_last_segments()` just renames all remaining
segments individually. A fast MoW load that produced, for example, 1009 small
segments still ends above `max_segment_num_per_rowset` with the defaults, even
though all delete bitmaps are ready by this point and repeated segcompaction
could have reduced it. Please keep retrying/rescheduling while enough
un-compacted segments remain, or wait/reschedule the pending-bitmap case
earlier so the final build path is not limited to one batch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
