On Wednesday, 20 November 2013, Dimitri Maziuk wrote: > On 11/18/2013 01:19 AM, YIP Wai Peng wrote: > > Hi Dima, > > > > Benchmark FYI. > > > > $ /usr/sbin/bonnie++ -s 0 -n 5:1m:4k > > Version 1.97 ------Sequential Create------ --------Random > Create-------- > > altair -Create-- --Read--- -Delete-- -Create-- --Read--- > -Delete-- > > files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > /sec %CP > > 5:1048576:4096 18 1 609 11 604 9 18 1 436 10 > 686 9 > > Latency 1187ms 70907us 261ms 2352ms 205ms > 111ms > > Is this on an FS on RBD on the NFS server, mounted and exported?
> Yes it is. And benchmarked on the client nfs machine. > I get > > > Version 1.96 ------Sequential Create------ --------Random > Create-------- > > nautilus -Create-- --Read--- -Delete-- -Create-- --Read--- > -Delete-- > > files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > /sec %CP > > 5:1048576:4096 19 1 5422 99 123 2 41 3 4525 99 > 127 2 > > Latency 86381us 746us 44125us 303ms 746us > 60720us > > on an ext4 filesystem mounted on top of DRBD device backed by an LSI > raid (10, battery-backed) on the main node and an mdadm raid-0 on the > passive node. This is on the server with he FS exported and mounted (but > not getting much use -- I ran it after hours). Hm, so maybe this nfsceph is not _that_ bad after all! :) Your read clearly wins, so I'm guessing the drdb write is the slow one. Which drdb mode are you using? > On a client & during > working hours I get > > > Version 1.96 ------Sequential Create------ --------Random > Create-------- > > stingray -Create-- --Read--- -Delete-- -Create-- --Read--- > -Delete-- > > files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > /sec %CP > > 5:1048576:4096 3 0 1952 29 1385 13 2 0 1553 18 > 575 5 > > Latency 16383ms 19662us 153ms 8482ms 1935us > 4467ms > > -- > Dimitri Maziuk > Programmer/sysadmin > BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu > >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
