Hi Behnam,
On 2/12/2018 4:06 PM, Behnam Loghmani wrote:
Hi there,
I am using ceph Luminous 12.2.2 with:
3 osds (each osd is 100G) - no WAL/DB separation.
3 mons
1 rgw
cluster size 3
I stored lots of thumbnails with very small size on ceph with radosgw.
Actual size of files is something about 32G but it filled 70G of each osd.
what's the reason of this high disk usage?
Most probably the major reason is BlueStore allocation granularity. E.g.
an object of 1K bytes length needs 64K of disk space if default
bluestore_min_alloc_size_hdd (=64K) is applied.
Additional inconsistency in space reporting might also appear since
BlueStore adds up DB volume space when accounting total store space.
While free space is taken from Block device only. is As a result when
reporting "Used" space always contain that total DB space part ( i.e.
Used = Total(Block+DB) - Free(Block) ). That correlates to other
comments in this thread about RockDB space usage.
There is a pending PR to fix that:
https://github.com/ceph/ceph/pull/19454/commits/144fb9663778f833782bdcb16acd707c3ed62a86
You may look for "Bluestore: inaccurate disk usage statistics problem"
in this mail list for previous discussion as well.
should I change "bluestore_min_alloc_size_hdd"? and If I change it and
set it to smaller size, does it impact on performance?
Unfortunately I haven't benchmark "small writes over hdd" cases much
hence don't have exacts answer here. Indeed these 'min_alloc_size'
family of parameters might impact the performance quite significantly.
what is the best practice for storing small files on bluestore?
Best regards,
Behnam Loghmani
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
