Re: Ozone caching feature like HDFS

2021-01-07 Thread Michel Sumbul
Hi guys, I think it really depends on the application that you are using. In my case on HDFS, we have quite often thousands of custom applications that read the same data/block at more or less the same time. I can see that it saves us a lot of IOPS because after some time it reads straight from th

Re: Ozone caching feature like HDFS

2021-01-07 Thread Wei-Chiu Chuang
To be honest, the HDFS caching didn't take off because somehow it didn't improve application performance a lot (speaking from an Impala perspective, I've been told it seems to help with Solr). As a matter of fact we did some performance tests recently comparing HDFS and Ozone. Depending on the set

Re: Ozone caching feature like HDFS

2021-01-06 Thread Lin, Yiqun
Hi Michel, As far as I know, currently Ozone hasn't a feature like HDFS cache block. Not sure if someone else has done the performance test between Ozone read and HDFS read. But anyway, cache block on Ozone seems a good idea if it can greatly speed the data read. Thanks, Yiqun On 2021/1/6, 1

Ozone caching feature like HDFS

2021-01-06 Thread Michel Sumbul
Hi guys, I would like to know if Ozone has a caching feature like HDFS? ( https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/CentralizedCacheManagement.html ) I didn’t find any doc on it for Ozone, but caching blocks frequently access really improve the performance on HDFS and

Ozone caching feature like HDFS

2021-01-06 Thread Michel Sumbul
Hi guys, I would like to know if Ozone has a caching feature like HDFS? ( https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/CentralizedCacheManagement.html ) I didn’t find any doc on it for Ozone, but caching blocks frequently access really improve the performance on HDFS and