Hi list，
I am testing cache tier in writeback mode.
The test resutl is confusing.The write performance is worse than without a
cache tier.
The hot storage pool is an all ssd pool and the cold storage pool is an all hdd
pool. I also created a hddpool and a ssdpool with the same crush rule as the
cache tier pools for comparison.
The pool config:
tierOSDcap.(TB)pgOSDcap.(TB)pg
hot-pool204.81024ssd-pool204.81024
cold-pool14014002048hdd-pool14014002048
The cache tier config:
# ceph osd tier add cold-pool hot-pool
pool 'hot-pool' is now (or already was) a tier of 'cold-pool'
#
# ceph osd tier cache-mode hot-pool writeback
set cache-mode for pool 'hot-pool' to writeback
#
# ceph osd tier set-overlay cold-pool hot-pool
overlay for 'cold-pool' is now (or already was) 'hot-pool'
#
# ceph osd pool set hot-pool hit_set_type bloom
set pool 39 hit_set_type to bloom
#
# ceph osd pool set hot-pool hit_set_count 10
set pool 39 hit_set_count to 10
#
# ceph osd pool set hot-pool hit_set_period 3600
set pool 39 hit_set_period to 3600
#
# ceph osd pool set hot-pool target_max_bytes 2400000000000
set pool 39 target_max_bytes to 2400000000000
#
# ceph osd pool set hot-pool target_max_objects 300000
set pool 39 target_max_objects to 300000
#
# ceph osd pool set hot-pool cache_target_dirty_ratio 0.4
set pool 39 cache_target_dirty_ratio to 0.4
#
# ceph osd pool set hot-pool cache_target_dirty_high_ratio 0.6
set pool 39 cache_target_dirty_high_ratio to 0.6
#
# ceph osd pool set hot-pool cache_target_full_ratio 0.8
set pool 39 cache_target_full_ratio to 0.8
#
# ceph osd pool set hot-pool cache_min_flush_age 600
set pool 39 cache_min_flush_age to 600
#
# ceph osd pool set hot-pool cache_min_evict_age 1800
set pool 39 cache_min_evict_age to 1800
Write Test
cold-pool(tier) write test for 10s
# rados bench -p cold-pool 10 write --no-cleanup
hdd-pool write test for 10s
# rados bench -p hdd-pool 10 write --no-cleanup
ssd-pool write test for 10s
# rados bench -p ssd-pool 10 write --no-cleanup
result:
tier hdd ssd
objects 695 737 2550
bandwith(MB/s) 272 289 1016
avg latency (s) 0.23 0.22 0.06
Read Test
# rados bench -p cold-pool 10 seq
# rados bench -p cold-pool 10 rand
# rados bench -p hdd-pool 10 seq
# rados bench -p hdd-pool 10 rand
# rados bench -p ssd-pool 10 seq
# rados bench -p ssd-pool 10 rand
seq result:
tier hdd ssd
bandwith(MB/s) 806 789 1113
avg latency (s) 0.074 0.079 0.056
rand result:
tier hdd ssd
bandwith(MB/s) 1106 790 1113
avg latency (s) 0.056 0.079 0.056
For my understanding the pool with cache tier in writeback mode should
performace like all ssd pool(client get ack after data write to hot storage)
if the cache dosen't need to be flushed.
But In wirte test,the pool with cache tier has poorer performance than even all
hdd pool.
And I inspect the pool stat to find out that there is only 244 objects in the
hot-pool and 695 objects in the cold pool(the write test wrote 695 objects).But
for my setting 695 objects shouldn't trigger the flush.
Is there any setting or concept I wrongly understood ?
2018-02-09
lin.yunfan_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
