Thanks Somnath, You are correct. I enabled rbd cache and things are working fine for me. I belived your other solution will also work as your logic is correct, though have not tried yet.
Regards sumit On Mon, Feb 2, 2015 at 2:41 PM, Somnath Roy <somnath....@sandisk.com> wrote: > Sumit, > > I think random read/write will always outperform sequential read/write in > Ceph if we don’t have any kind of cache in front or you have proper > striping enabled in the image. The reason is the following. > > > > 1. If you are trying with the default image option, the object size is 4 > MB and the stripe size = 4MB and stripe unit = 1. > > > > 2. You didn’t mention your write size, so, if it is less than 4 MB , 2 seq > write will always land on a same PG and it will be serialized within the > OSD. > > > > 3. But, if we have 2 random writes it will always (more probable) to land > on different PGs and it will be processed in parallel. > > > > 4. Same will happen in case of random vs seq read as well. Increasing > read_ahead_kb to a reasonable big number will improve the seq read speed. > If you are using librbd, rbd_cache will help you both for read/write I > guess. > > > > 5. Another option you may want to try to set the > strip_size/object_size/stripe_unit to your io_size so that seq read/write > can land on different object and in that case the difference should go away. > > > > Hope this is helpful. > > > > Thanks & Regards > > Somnath > > > > > > *From:* ceph-users [mailto:ceph-users-boun...@lists.ceph.com] *On Behalf > Of *Sumit Gaur > *Sent:* Sunday, February 01, 2015 6:55 PM > *To:* Florent MONTHEL > *Cc:* ceph-users@lists.ceph.com > *Subject:* Re: [ceph-users] ceph Performance random write is more then > sequential > > > > Hi All, > > What I saw after enabling RBD cache it is working as expected, means > sequential write has better MBps than random write. can somebody explain > this behaviour ? Is RBD cache setting must for ceph cluster to behave > normally ? > > > > Thanks > > sumit > > > > On Mon, Feb 2, 2015 at 9:59 AM, Sumit Gaur <sumitkg...@gmail.com> wrote: > > Hi Florent, > > Cache tiering , No . > > > > ** Our Architecture : > > > > vdbench/FIO inside VM <--> RBD without cache <-> Ceph Cluster (6 OSDs + 3 > Mons) > > > > > > Thanks > > sumit > > > > [root@ceph-mon01 ~]# ceph -s > > cluster 47b3b559-f93c-4259-a6fb-97b00d87c55a > > health HEALTH_WARN clock skew detected on mon.ceph-mon02, > mon.ceph-mon03 > > monmap e1: 3 mons at {ceph-mon01= > 192.168.10.19:6789/0,ceph-mon02=192.168.10.20:6789/0,ceph-mon03=192.168.10.21:6789/0}, > election epoch 14, quorum 0,1,2 ceph-mon01,ceph-mon02,ceph-mon03 > > osdmap e603: 36 osds: 36 up, 36 in > > pgmap v40812: 5120 pgs, 2 pools, 179 GB data, 569 kobjects > > 522 GB used, 9349 GB / 9872 GB avail > > 5120 active+clean > > > > > > On Mon, Feb 2, 2015 at 12:21 AM, Florent MONTHEL <fmont...@flox-arts.net> > wrote: > > Hi Sumit > > Do you have cache pool tiering activated ? > Some feed-back regarding your architecture ? > Thanks > > Sent from my iPad > > > On 1 févr. 2015, at 15:50, Sumit Gaur <sumitkg...@gmail.com> wrote: > > > > Hi > > I have installed 6 node ceph cluster and to my surprise when I ran rados > bench I saw that random write has more performance number then sequential > write. This is opposite to normal disk write. Can some body let me know if > I am missing any ceph Architecture point here ? > > _______________________________________________ > > ceph-users mailing list > > ceph-users@lists.ceph.com > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > > ------------------------------ > > PLEASE NOTE: The information contained in this electronic mail message is > intended only for the use of the designated recipient(s) named above. If > the reader of this message is not the intended recipient, you are hereby > notified that you have received this message in error and that any review, > dissemination, distribution, or copying of this message is strictly > prohibited. If you have received this communication in error, please notify > the sender by telephone or e-mail (as shown above) immediately and destroy > any and all copies of this message in your possession (whether hard copies > or electronically stored copies). > >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
