luwei16 commented on code in PR #68057:
URL: https://github.com/apache/doris/pull/68057#discussion_r4024004946
##########
be/src/cloud/cloud_storage_engine.cpp:
##########
@@ -681,7 +681,7 @@ void
CloudStorageEngine::_binlog_compaction_tasks_producer_callback() {
int64_t interval = config::generate_compaction_tasks_interval_ms;
do {
int64_t cur_time = UnixMillis();
- if (config::enable_feature_binlog && !config::disable_auto_compaction)
{
+ if (!config::disable_auto_compaction) {
Review Comment:
[major] Avoid a 100 ms full-tablet scan when no Row Binlog tablet exists
With the CCR gate removed, the default configuration now enters this branch
even on BEs that have no Row Binlog tablets, then resets the loop to
`generate_compaction_tasks_interval_ms` (100 ms). The Cloud generator
copies/traverses every cached tablet each round while `TabletMap::traverse`
holds its global mutex; the parallel non-Cloud change loops every data
directory and then visits every tablet shard, so an idle round is `O(data_dirs
* tablets)`. Role filtering prevents a wrong compaction but happens
during/after enumeration and does not avoid this cost. Before this change, the
false gate selected a 5-second wait; unlike the ordinary local producer, this
binlog producer has no empty-result backoff.
Please keep scheduling independent of `enable_feature_binlog`, but retain
the fast cadence only while work is found and use a bounded empty-round backoff
(or a cheap Row Binlog presence signal plus wakeup). Add coverage for both the
no-Row-Binlog steady state and discovery of a tablet that appears later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
