Thanks runzhiwang for your explanation! It makes a lot of sense!
-Rui On Mon, Dec 14, 2020 at 4:39 PM runzhiwang <runzhiwan...@gmail.com> wrote: > Hi Rui, > I did not post the comparison result of async api and streaming, because > FileStore's asyncApi's implementation is different from Ozone. > FileStore's asyncApi does not have any optimization, such as Ozone > implement dataStateMachineCache to improve readStateMachineData, > but FileStore's asyncApi need to read disk to read StateMachineData, so > FileStore's asyncApi is slower than Ozone, and it's meaningless to > compare FileStore's asyncApi and FileStore's streaming. Even though > FileStore's streaming is faster than FileStore's asyncApi, we can not say > FileStore's streaming is faster than Ozone. So I compare Ozone and > FileStore's streaming directly, because the improvement is great, We > can make sure FileStore's streaming is going to be a greate improvement on > Ozone. > > Thanks, > runzhiwang > > > Rui Wang <amaliu...@apache.org> 于2020年12月15日周二 上午4:58写道： > > > The improvements of Ratis streaming is solid based on the doc! Thanks > > runzhiwang for your work! > > > > After walking through the performance doc, I think the comparison seemed > to > > happen between an Ozone cluster and a filestore? > > > > If so, my question is, will the Ozone cluster be too heavy to compare > with > > filestore, which is a simple implementation to write files? Do we need a > > comparison for filesstore only between async api and streaming? > > > > > > Hi Liyun, > > > > Currently the Ratis streaming write feature should be ready to use > (though > > improvements could keep happening). The read feature is under > development. > > > > -Rui > > > > On Mon, Dec 14, 2020 at 8:02 AM Lin, Yiqun <yiq...@ebay.com.invalid> > > wrote: > > > > > Hi runzhiwang/ Tsz-Wo, > > > > > > Seems a very good improvement, I am interested for this. > > > As I see the Ratis Streaming feature (RATIS-979) is still in progress > > > state. Do you know current status of this feature and if it's ready and > > > stable for Ozone to use now? > > > > > > Thanks, > > > Yiqun > > > > > > On 2020/12/14, 5:40 PM, "Tsz Wo Sze" <szets...@gmail.com> wrote: > > > > > > External Email > > > > > > runzhiwang, > > > > > > Thanks a lot for working hard on Ratis Streaming and running the > > > benchmarks! Ratis Streaming is going to be a big improvement in > > Ozone. > > > > > > It is anticipated that Ratis Streaming can > > > - improve the performance as shown in the benchmark results; > > > - improve the scalability of Ozone since Streaming can reduce the > > > number of > > > pipelines in Ozone; and > > > - reduce the memory requirement in datanodes. > > > > > > Regards, > > > Tsz-Wo > > > > > > On Mon, Dec 14, 2020 at 4:35 PM runzhiwang <runzhiwan...@gmail.com > > > > > wrote: > > > > > > > Hi all, > > > > I test the performance of ratis streaming. As the image shows, > > Raits > > > > streaming can improve Ozone's throughput, memory usage, and > > stability > > > > greatly. > > > > > > > > [image: screenshot-1.png] > > > > > > > > If you are interested with the test and want to reproduce it, > > please > > > find > > > > the details here: > > > > > > > > > > > > > > https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.google.com%2Fdocument%2Fd%2F1TkR4qN1ymm9UY2nRLGlvL_TjI_gTabByDN5uCUEpJiM%2Fedit&data=04%7C01%7Cyiqlin%40ebay.com%7C4a8c74e9a369402a3c8f08d8a014502e%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C637435356398333595%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=moz51OUIBm98n%2B%2FYdUtLT4Az0CE%2BgQ%2FnhpU47G5HWOY%3D&reserved=0 > > > > > > > > > > > > Thanks, > > > > runzhiwang > > > > > > > > > > > > >
