Yes, as Jason suggests - 27 IOPS doing 4k blocks is:
27*4/1024 MB/s = 0.1 MB/s
While the RBD volume is composed of 4MB objects - many of the
(presumably) random IOs of 4k blocks can reside in the same 4MB object,
so it is tricky to estimate how many 4MB objects are needing to be
rewritten each second (i.e your write amplification).
Generally 27 IOPS is not particularly good, even with a client
connecting via 1Gbit (after all *it* can send or receive the 4k blocks
at a max of about 100MB/s) [1].
To be able to provide any more useful help, we'd need to know a bit more
about your ceph cluster (version, how many machines, osd's, ceph.conf)
and how your client is setup (vm using librbd or kernel rbd etc). Also
your fio job file would be handy to see.
Cheers
Mark
[1] I guess it might be worthwhile to check that the client *can*
actually achieve 100MB/s, if not there is some networking glitch
somewhere in your system that needs to be fixed before you retest.
On 01/09/14 14:51, Jason King wrote:
Guess you should multiply 27 by bs=4k?
Jason
2014-08-29 15:52 GMT+08:00 lixue...@chinacloud.com.cn
<mailto:lixue...@chinacloud.com.cn> <lixue...@chinacloud.com.cn
<mailto:lixue...@chinacloud.com.cn>>:
guys:
There's a ceph cluster working and nodes were connected with
10Gb cable. We defined fio's bs=4k and the object size of rbd is
4MB.
Client node was connect with the cluster via 1000Mb cable. Finally
the IOPS is 27 ,when we control the latency of fio under 20ms.
4MB*27=108MB nearly 1000Mb,so is it reasonable?
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
