[
https://issues.apache.org/jira/browse/HBASE-20352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reid Chan updated HBASE-20352:
------------------------------
Release Note:
HBASE-20352 aims to speed up process of cleaning oldWALs in two aspects,
1. Use multiple threads to scan oldWALs directories through config
hbase.cleaner.scan.dir.concurrent.size, which supports both integer (meaning
the concrete size, but no more than available cpu cores) and double (between
>0.0 and <=1.0, meaning ratio of available cpu cores) value, 0.25 by default.
Pay attention that 1.0 is different from 1, the former indicates it will use
100% of cores, while the latter will use only 1 thread for chore to scan dir.
2. Use multiple threads to clean wals under oldWALs directory through
hbase.oldwals.cleaner.thread.size, 2 by default.
3. In addition, two configs, hbase.cleaner.scan.dir.concurrent.size and
hbase.oldwals.cleaner.thread.size both support online re-configuration.
was:
After HBASE-20352,
1. we could use multiple threads to scan oldWALs directories through config
hbase.cleaner.scan.dir.concurrent.size, which supports both integer (meaning
the concrete size, but no more than available cpu cores) and double (between
>0.0 and <=1.0, meaning ratio of available cpu cores) value, 0.25 by default.
Pay attention that 1.0 is different from 1, the former indicates it will use
100% of cores, while the latter will use only 1 thread for chore to scan dir.
2. We also support using multiple threads to clean wals under oldWALs directory
through hbase.oldwals.cleaner.thread.size, 2 by default.
3. In addition, hbase.cleaner.scan.dir.concurrent.size and
hbase.oldwals.cleaner.thread.size both support online re-configuration.
> [Chore] Backport HBASE-18309 to branch-1
> ----------------------------------------
>
> Key: HBASE-20352
> URL: https://issues.apache.org/jira/browse/HBASE-20352
> Project: HBase
> Issue Type: Improvement
> Reporter: Reid Chan
> Assignee: Reid Chan
> Priority: Major
> Attachments: HBASE-20352.branch-1.001.patch
>
>
> Using multiple threads to scan directory and to clean old WALs
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
