In my belief, lot of sequential small reads will be aggregated after all when targeting filestore contents (of course if the moment of issuing next one is not dependent on status of previous read, otherwise they`ll be separated in time in such way that the rotating media scheduler will not be able to combine requests), am I wrong? If so, this case only affects OSD CPU consumption (on very large scale). Ke-fei, is there any real reasons behind staying on IDE and not LSI SCSI emulation/virtio?
On Fri, Jun 13, 2014 at 8:11 PM, Sage Weil <s...@inktank.com> wrote: > Right now, no. > > We could add a minimum read size to librbd when caching is enabled... > that would not be particularly difficult. > > sage > > > On Fri, 13 Jun 2014, Ke-fei Lin wrote: > >> 2014-06-13 22:04 GMT+08:00 Andrey Korolyov <and...@xdel.ru>: >> > >> > On Fri, Jun 13, 2014 at 5:50 PM, Ke-fei Lin <k...@kfei.net> wrote: >> > > Thanks, Andrey. >> > > >> > > I tried `virsh domblkstat <vm> hda` (only this VM in whole cluster) and >> got >> > > these values: >> > > >> > > hda rd_req 70682 >> > > hda rd_bytes 229894656 >> > > hda wr_req 1067 >> > > hda wr_bytes 12645888 >> > > hda flush_operations 0 >> > > >> > > (These values became stable after ~2 mins) >> > > >> > > While the output of `ceph -w` is attached at: >> http://pastebin.com/Uhdj9drV >> > > >> > > Any advices? >> > >> > >> > Thanks, poor man`s analysis shows that it can be true - assuming >> > median heartbeat value as 1.2s, overall read ops are about 40k, which >> > is close enough to what qemu stats saying, regarding floating >> > heartbeat interval. Because ceph -w never had such value as a precise >> > measurement tool, I may suggest to measure block stats difference on >> > smaller intervals, about 1s or so, and compare values then. By the >> > way, what driver do you use in qemu for a block device? >> >> OK, this time I capture the blkstat difference in a smaller interval (less >> than 1s). >> And a simple calculation gives me some result: >> >> (19531264-19209216)/(38147-37518) = 512 >> ... >> (20158976-19531264)/(39373-38147) = 512 >> >> Which means in the beginning of boot phase, every read request from VM is >> just *512 byte*. >> Maybe this is why `ceph -w` shows me every operation is about 1KB (in my >> first post)? >> >> So seems this is the inherent problem of Windows VM, but can I do something >> in my Ceph >> cluster's configuration to improve this? >> >> By the way the related part of my VM definition are: >> >> <emulator>/usr/bin/kvm</emulator> >> <disk type='network' device='disk'> >> <driver name='qemu' type='rbd' cache='writeback'/> >> <source protocol='rbd' name='libvirt-pool/test-rbd-1'> >> <host name='10.0.0.5' port='6789'/> >> </source> >> <target dev='hda' bus='ide'/> >> <address type='drive' controller='0' bus='0' unit='0'/> >> </disk> >> >> Thanks. >> >> _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
