Hi,
I did some basic experiments with mysql and measured the time taken by a
set of operations on CephFS and RBD. The RBD measurements are taken on a
1GB RBD disk with ext4 filesystem. Following are my observation. The
time listed below are in seconds.
*Plain file system* *CephFS* *RBD*
Mysql install db 7.9 38.3 36.4
Create table 0.43 4.2 2.5
Drop table 0.14 0.21 0.40
Create table + 1000 recs 2.76 4.69 5.07
Create table + 10000 recs
7.69 11.96
Create table + 100K recs
12.06 29.65
From the above numbers, CephFS seems to fare very well while creating
records whereas RBD does well while creating a table. I tried measuring
the syscalls of ceph-osd, ceph-mds and the mysqld while creating a table
on CephFS and RBD. Following is how the key syscalls of mysqld performed
while creating a table (time includes wait time as well).
*Syscalls of MYSQLD* *CephFS* *RBD*
fsync 338.237 ms 183.697 ms
fdatasync 75.635 ms 96.359 ms
io_submit 50 us 151 us
open 2266 us 61 us
close 1186 us 33 us
write 115 us 51 us
From the above numbers, open, close and fsync syscalls take too much
time on CephFs as compared to RBD.
Sysbench results are below;
*Sysbence 100K records in 60 secs* *CephFS* *RBD*
Read Queries performed 631876 501690
Other Queries performed 90268 71670
No. of transactions 45134 35835
No. of transactions per sec 752.04 597.17
R/W requests per sec 10528.55 8360.37
Other operations per sec 1504.08 1194.34
Above numbers seems to indicate the CephFS does very well with MYSQL
transactions, better than RBD.
Following is my setup;
Num MONs : 1
Num OSDs : 1
Num MDSs : 1
Disk : 10 GB Qemu disk file (Both journal and data in the
same disk)
Ceph version : 10.2.5 (Built from source)
<http://download.ceph.com/tarballs/ceph-10.2.5.tar.gz>
Build config : ./configure --without-debug --without-fuse --with-libaio \
--without-libatomic-ops --without-hadoop --with-nss
--without-cryptopp \
--without-gtk2 --disable-static --with-jemalloc \
--without-libzfs --without-lttng --without-babeltrace \
--with-eventfd --with-python -without-kinetic
--without-librocksdb \
--without-openldap \
CFLAGS="-g -O2 -fPIC" CXXFLAGS="-g -O2 -std=c++11 -fPIC
Ceph conf : Apart from host and network settings nothing else is configured
CephFS mount options: rw,relatime,name=cephfs,secret=<hidden>,acl
RBD mount options: rw,relatime,stripe=1024,data=ordered
All the processes were run in a Qemu virtual machine with Linux 4.4.18
kernel
Searching for "Mysql on CephFS" in google does not give any useful
results. If this kind of experiments had been done previously and shared
publicly, kindly share a link to it.
If you are aware of anything that I can do to optimise this, kindly let
me know. I am willing to continue this experiment to see how well we can
optimise CephFs for mysql.
Thank you,
Babu Shanmugam
www.aalam.io <https://www.aalam.io>
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
