Hi David, On 20/06/2014 14:15, David Z wrote:> > Hi Loic, > > We are evaluating erasure coding and we want to tolerate 3 chunks failure. > Then we choose cauchy_orig because RS's performance should be no better than > cauchy_orig and other algorithms are optimized for raid6 mode. > > For cauchy_orig, we need to tune w and packet size, because of 1) the padding > size which will result in storage overhead and 2) encoding and decoding > computing performance. > > We tested the encoding and decoding performance under different w and packet > size with different file sizes, see the attachment which shows an example of > 2MB file. And w=4 and packet size=512 seems to be good as the trade off > between the computing performance and storage overhead. Does this make sense > to you, because default value in ceph is w=8 and packet size=2048?
It is better to use Reed Solomon Vandermonde (reed_sol_van) at this point in time (Ceph Firefly v0.80.1): there is no need to worry about the packet size and it performs better than Reed Solomon Cauchy (cauchy_good). It would be nice to analyze the performance differences, using tools mentioned in http://dachary.org/?p=3042 or any other. I think there is room for improvement but I recently focused on locally repairable codes instead. > Another issue is that during our write testing, under the same load, we found > the OSD cpu usage between w=4,packet size=512 and w=6,packet size=2048 has no > much difference, but the disk util of w=4,packet size=512 will be a little > higher than w=6,packet size=2048. We thought OSD would write journal > according to chunk size, but we don't see any code about that and it > shouldn't be. So do you have any idea about it? And can you think of any side > effect of choosing w=4 and packet size=512? It would be great to get scripts and results to comment on. Modifying the packet size has an impact that is measurable but there are no measures regarding the word size at this point. Cheers > > Thanks. > > Regards, > David Z > -- Loïc Dachary, Artisan Logiciel Libre
signature.asc
Description: OpenPGP digital signature
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
