Re: [PR] [improvement](memory) Storage page cache use LRU-K cache, K=2 [doris]

[via GitHub]

xinyiZzz commented on code in PR #45719:
URL: https://github.com/apache/doris/pull/45719#discussion_r1896304938


##########
be/src/olap/lru_cache.cpp:
##########
@@ -289,21 +289,35 @@ void LRUCache::_lru_append(LRUHandle* list, LRUHandle* e) 
{
 }
 
 Cache::Handle* LRUCache::lookup(const CacheKey& key, uint32_t hash) {
-    std::lock_guard l(_mutex);
-    ++_lookup_count;
-    LRUHandle* e = _table.lookup(key, hash);
-    if (e != nullptr) {
-        // we get it from _table, so in_cache must be true
-        DCHECK(e->in_cache);
-        if (e->refs == 1) {
-            // only in LRU free list, remove it from list
-            _lru_remove(e);
+    LRUHandle* e;
+    {
+        std::lock_guard l(_mutex);
+        ++_lookup_count;
+        e = _table.lookup(key, hash);
+        if (e != nullptr) {
+            // we get it from _table, so in_cache must be true
+            DCHECK(e->in_cache);
+            if (e->refs == 1) {
+                // only in LRU free list, remove it from list
+                _lru_remove(e);
+            }
+            e->refs++;
+            ++_hit_count;
+            e->last_visit_time = UnixMillis();
+        } else {
+            ++_miss_count;
+        }
+    }
+    // If key not exist in cache, and is lru k cache, and key in visits list,
+    // then move the key to beginning of the visits list.
+    // key in visits list indicates that the key has been inserted once after 
the cache is full.
+    if (e == nullptr && _is_lru_k) {
+        std::lock_guard l(_visits_lru_cache_mutex);

Review Comment:
   最好2个锁，现在 lru cache 锁的开销在某些场景下已经很大了，比如 prune 淘汰的时候，
   _visits_lru_cache_list 是个完全独立的数据结构，这样第一次insert的时候可以不用拿 cache _mutex
   
![image](https://github.com/user-attachments/assets/87f989e0-65ec-43f4-8aa1-9427d6b3ac89)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org