On Mon, 2012-03-26 at 12:47 +0200, Conrad Wood wrote: > > >> Unless you actually want to test writing dirty pages, and I have seen > > >> a kernel bug where flushing dirty pages was slow, but then you should > > >> also time a 'sync': > > >> time dd if=/dev/zero of=/mnt/point/file bs=1M count=1000 && time sync > > >> > > That is part of the timing sequence, yes. > > > > > Conrad> Mostly I wonder if it is atall possible to get such speeds > > Conrad> over QDR. Are you in a position where you could perhaps run a > > Conrad> "cp" followed by "sync" on an infiniband attached storage > > Conrad> system? > > > > Absolutely, Infiniband is fast, and QDR should be insanely fast, and > > low low latency as well, which is it's primary metric. > > > > Do you have any comparative values for this particular sequence of > commands? > > > > > But can we go back and look at your system again? What OS, what > > hardware, what filesystem, are you using LVM? How is the array RAID > > setup, or is it not using RAID on the hardware end? > > sure. > on the storage server > gentoo, lvm, md raid 10, 512k chunks, 3.2.8 kernel with in-tree SRP > > on the remote server > OS = debian, 3.2.8 kernel, intree SRP, > > > > > > Have you tried using 'fio' the Flexible IO Tester to run your tests > > instead? > > yes I did, the results were inconclusive though (to me at least) ;) > > Conrad >
for the record on this list: blktrace provided the relevant clues, the blocks came in much smaller chunks with SRP than with iSer. The SCST Target software limits the SRP maximum rdma size to 64k. With iSer it goes up to 1 MByte. Adjusting the size works much better. ;) Thanks for the help. Conrad _______________________________________________ Tech mailing list [email protected] https://lists.lopsa.org/cgi-bin/mailman/listinfo/tech This list provided by the League of Professional System Administrators http://lopsa.org/
