Hi Loic, Thanks for your reply.
I actually used the tool you mentioned for our evaluation on cauchy_orig. I didn't compare reed_sol_van with cauchy_orig. I plan to do that now. But we still care about padding size because there are storage overhead. As for reed_sol_van, I don't think we can have much to do about it because the stripe width is k*w*sizeof(int) and minimal w is 8. I am just curious that if you take this into consideration and may have some future plan to reduce storage overhead. Regards, David Z On Saturday, June 21, 2014 12:22 AM, Loic Dachary <l...@dachary.org> wrote: Hi David, On 20/06/2014 14:15, David Z wrote:> > Hi Loic, > > We are evaluating erasure coding and we want to tolerate 3 chunks failure. > Then we choose cauchy_orig because RS's performance should be no better than > cauchy_orig and other algorithms are optimized for raid6 mode. > > For cauchy_orig, we need to tune w and packet size, because of 1) the padding > size which will result in storage overhead and 2) encoding and decoding > computing performance. > > We tested the encoding and decoding performance under different w and packet > size with different file sizes, see the attachment which shows an example of > 2MB file. And w=4 and packet size=512 seems to be good as the trade off > between the computing performance and storage overhead. Does this make sense > to you, because default value in ceph is w=8 and packet size=2048? It is better to use Reed Solomon Vandermonde (reed_sol_van) at this point in time (Ceph Firefly v0.80.1): there is no need to worry about the packet size and it performs better than Reed Solomon Cauchy (cauchy_good). It would be nice to analyze the performance differences, using tools mentioned in http://dachary.org/?p=3042 or any other. I think there is room for improvement but I recently focused on locally repairable codes instead. > Another issue is that during our write testing, under the same load, we found > the OSD cpu usage between w=4,packet size=512 and w=6,packet size=2048 has no > much difference, but the disk util of w=4,packet size=512 will be a little > higher than w=6,packet size=2048. We thought OSD would write journal > according to chunk size, but we don't see any code about that and it > shouldn't be. So do you have any idea about it? And can you think of any side > effect of choosing w=4 and packet size=512? It would be great to get scripts and results to comment on. Modifying the packet size has an impact that is measurable but there are no measures regarding the word size at this point. Cheers > > Thanks. > > Regards, > David Z > -- Loïc Dachary, Artisan Logiciel Libre _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
