github-actions[bot] commented on code in PR #63504:
URL: https://github.com/apache/doris/pull/63504#discussion_r3369755588
##########
be/src/io/cache/cache_lru_dumper.cpp:
##########
@@ -298,25 +298,53 @@ void CacheLRUDumper::dump_queue(const std::string&
queue_name, bool force) {
if (force || _recorder->get_lru_queue_update_cnt_from_last_dump(type) >
config::file_cache_background_lru_dump_update_cnt_threshold) {
LRUQueue& queue = _recorder->get_shadow_queue(type);
- do_dump_queue(queue, queue_name);
- _recorder->reset_lru_queue_update_cnt_from_last_dump(type);
+ std::vector<LruDumpEntry> elements;
+ size_t dump_update_cnt = 0;
+ {
+ std::lock_guard<std::mutex>
lru_log_lock(_recorder->_mutex_lru_log);
+ // Drain logs counted for this dump before updating the counter,
otherwise restore may
+ // see a stale LRU tail after a crash.
+ _recorder->replay_queue_event_locked(type, 0, lru_log_lock);
Review Comment:
This now mutates the shadow LRU queue from the dump thread while holding
only `_mutex_lru_log`, but `LRUQueueRecorder::evaluate_queue_diff()` still
reads the same shadow queue without that mutex (it is called from
`run_background_lru_log_replay()` under only the cache mutex when
`enable_evaluate_shadow_queue_diff=true`). Before this change the dump path
only collected the shadow queue under `_mutex_lru_log`; after this replay call,
dump can concurrently erase/splice/add entries while evaluation
copies/traverses `target.queue` and `target.map`, causing a data race and
possible iterator/container corruption. Please make the shadow queue locking
consistent, e.g. take `_mutex_lru_log` around `evaluate_queue_diff()`'s access
or move the diff calculation onto a snapshot collected under that mutex.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
