[
https://issues.apache.org/jira/browse/KUDU-613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17854204#comment-17854204
]
ASF subversion and git services commented on KUDU-613:
------------------------------------------------------
Commit f49a548fe4bac91679744b4dd0efa895ff9c09e2 in kudu's branch
refs/heads/master from Mahesh Reddy
[ https://gitbox.apache.org/repos/asf?p=kudu.git;h=f49a548fe ]
KUDU-613: Introduce SLRU cache
This patch introduces the SLRU cache that has two internal segments,
the probationary and protected, to protect the cache from long/random
reads. The SLRU cache has a parameter named 'lookups_threshold_' that
determines the minimum amount of times an entry can be accessed
before it's upgraded to the protected segment. Any random scans would
then only evict entries from the probationary segment.
Both the protected and probationary segment have their own
configurable capacities. When the protected segment is at capacity,
any entries evicted will be added to the MRU end of the
probationary segment.
Metrics will be added in a follow-up patch.
Ran cache benchmarks for RELEASE build.
Used default flag values in cache-bench.
Build ran locally on macOS, 6 cores and 2.6GHz.
Here are some benchmark numbers for SLRU cache:
Test case | Algorithm | Lookups/sec | Hit rate
ZIPFIAN ratio=1.00x | LRU | 11.01M | 99.8%
ZIPFIAN ratio=1.00x | SLRU | 10.62M | 99.9%
ZIPFIAN ratio=3.00x | LRU | 11.06M | 95.9%
ZIPFIAN ratio=3.00x | SLRU | 9.69M | 95.9%
UNIFORM ratio=1.00x | LRU | 8.54M | 99.7%
UNIFORM ratio=1.00x | SLRU | 6.18M | 99.7%
UNIFORM ratio=3.00x | LRU | 6.53M | 33.3%
UNIFORM ratio=3.00x | SLRU | 4.99M | 33.3%
Change-Id: I45531534a2049dd38c002f4dc7982df9fd46e0bb
Reviewed-on: http://gerrit.cloudera.org:8080/20607
Tested-by: Alexey Serbin <ale...@apache.org>
Reviewed-by: Alexey Serbin <ale...@apache.org>
> Scan-resistant cache replacement algorithm for the block cache
> --------------------------------------------------------------
>
> Key: KUDU-613
> URL: https://issues.apache.org/jira/browse/KUDU-613
> Project: Kudu
> Issue Type: Improvement
> Components: perf
> Affects Versions: M4.5
> Reporter: Andrew Wang
> Assignee: Mahesh Reddy
> Priority: Major
> Labels: performance, roadmap-candidate
>
> The block cache currently uses LRU, which is vulnerable to large scan
> workloads. It'd be good to implement something like 2Q.
> ARC (patent encumbered, but good for ideas):
> https://www.usenix.org/conference/fast-03/arc-self-tuning-low-overhead-replacement-cache
> HBase (2Q like):
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
