You don't have results that include the added network latency of having replica 3 replicating across multiple hosts. The reads would be very similar as the primary is the only thing that is read from, but writes will not return until after all 3 copies are written.
On Sat, Apr 29, 2017, 9:46 PM Babu Shanmugam <b...@aalam.io> wrote: > Hi, > I did some basic experiments with mysql and measured the time taken by a > set of operations on CephFS and RBD. The RBD measurements are taken on a > 1GB RBD disk with ext4 filesystem. Following are my observation. The time > listed below are in seconds. > > > *Plain file system* *CephFS* *RBD* > Mysql install db 7.9 38.3 36.4 > Create table 0.43 4.2 2.5 > Drop table 0.14 0.21 0.40 > Create table + 1000 recs 2.76 4.69 5.07 > Create table + 10000 recs > 7.69 11.96 > Create table + 100K recs > 12.06 29.65 > > > From the above numbers, CephFS seems to fare very well while creating > records whereas RBD does well while creating a table. I tried measuring the > syscalls of ceph-osd, ceph-mds and the mysqld while creating a table on > CephFS and RBD. Following is how the key syscalls of mysqld performed while > creating a table (time includes wait time as well). > > *Syscalls of MYSQLD* *CephFS* *RBD* > fsync 338.237 ms 183.697 ms > fdatasync 75.635 ms 96.359 ms > io_submit 50 us 151 us > open 2266 us 61 us > close 1186 us 33 us > write 115 us 51 us > > From the above numbers, open, close and fsync syscalls take too much time > on CephFs as compared to RBD. > > Sysbench results are below; > > > *Sysbence 100K records in 60 secs* *CephFS* *RBD* > Read Queries performed 631876 501690 > Other Queries performed 90268 71670 > No. of transactions 45134 35835 > No. of transactions per sec 752.04 597.17 > R/W requests per sec 10528.55 8360.37 > Other operations per sec 1504.08 1194.34 > Above numbers seems to indicate the CephFS does very well with MYSQL > transactions, better than RBD. > > > Following is my setup; > > Num MONs : 1 > Num OSDs : 1 > Num MDSs : 1 > Disk : 10 GB Qemu disk file (Both journal and data in the > same disk) > Ceph version : 10.2.5 (Built from source) > <http://download.ceph.com/tarballs/ceph-10.2.5.tar.gz> > Build config : ./configure --without-debug --without-fuse --with-libaio > \ > --without-libatomic-ops --without-hadoop --with-nss > --without-cryptopp \ > --without-gtk2 --disable-static --with-jemalloc \ > --without-libzfs --without-lttng --without-babeltrace \ > --with-eventfd --with-python -without-kinetic > --without-librocksdb \ > --without-openldap \ > CFLAGS="-g -O2 -fPIC" CXXFLAGS="-g -O2 -std=c++11 -fPIC > > Ceph conf : Apart from host and network settings nothing else is > configured > CephFS mount options: rw,relatime,name=cephfs,secret=<hidden>,acl > RBD mount options: rw,relatime,stripe=1024,data=ordered > > All the processes were run in a Qemu virtual machine with Linux 4.4.18 > kernel > > Searching for "Mysql on CephFS" in google does not give any useful > results. If this kind of experiments had been done previously and shared > publicly, kindly share a link to it. > > If you are aware of anything that I can do to optimise this, kindly let me > know. I am willing to continue this experiment to see how well we can > optimise CephFs for mysql. > > > > Thank you, > Babu Shanmugam > www.aalam.io > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
