Hello Ceph Community, I used libradosstriper api to test the striping feature, it doesn't seem to improve the performance at all, can anyone advise what's wrong with my settings:
The rados object store testbed at my center has osd: 48 oss: 4 monitor:2 pg number: 1024 replicated size: 3 I have implemented a benchmark code <https://github.com/NERSC/object-store/blob/master/tests/ceph/vpic_io/librados_test.c> with libradosstriper api. I then used 1 processes 1 thread to do the test, by varying a few settings: - stripe count from 1 to 48, - and object size from 1MB to 128 MB (with stripe size 1MB, stripe size needs to be smaller than the rados object size), - and file size from 100MB to 1.6GB, The peak bandwidth among all tests is only 130MB/s, no difference in different tests. I suspect that the IO got serialized in the rados layer, with some uncertain evidence in the libradosstriper source code (note the *for* loop): ... > ·Striper::file_to_extents(cct(), format.c_str(), &l, off, len, 0, > extents); > *·for* (vector<ObjectExtent>::iterator p = extents.begin(); p != extents. > end(); ++p) { > r = m_ioCtx.aio_write(p->oid.name, rados_completion, oid_bl, > p->length, p->offset); > } > ... Could you please correct me if I misused or misunderstood any things? Thanks much. Best, Jialin NERSC/LBNL
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
