+1, Impressive performance achievement on OzoneManager, let's move to Beta.
Bests, Sammi Chen On Thu, Feb 20, 2020 at 4:17 AM Anu Engineer <aengin...@apache.org> wrote: > Hi All, > > > I would like to propose moving Ozone from 'Alpha' tags to 'Beta' tags when > we do future releases. Here are a couple of reasons why I think we should > make this move. > > > > 1. Ozone Manager or the Namenode for Ozone scales to more than 1 billion > keys. We tested this in our labs in an organic fashion; that is, we were > able to create more than 1 billion keys from external clients with no > loss > in performance. > 2. The ozone Manager meets the performance and resource constraints that > we set out to achieve. We were able to sustain the same throughput at > Ozone > manager for over three days that took us to get this 1 billion keys. > That > is, we did not have to shut down or resize memory for the namenode as we > went through this exercise. > 3. The most critical, we did this experiment with 64GB of memory > allocation in JVM and 64 GB of RAM off-heap allocation. That is, the > Ozone > Manager was able to achieve this scale with far less memory footprint > than > HDFS. > 4. Ozone's performance is at par with HDFS when running workloads like > Hive ( > > https://blog.cloudera.com/benchmarking-ozone-clouderas-next-generation-storage-for-cdp/ > ) > 5. We have been able to run long-running clusters with Ozone. > > > Having achieved these goals, I propose that we move from the planned > 0.4.2-Alpha release to 0.5.0-Beta as our next release. If we hear no > concerns about this, we would like to move Ozone from Alpha to Beta > releases. > > > Thanks > > Anu > > > P.S. I am CC-ing HDFS dev since many people who are interested in Ozone > still have not subscribed to Ozone dev lists. My apologies if it feels like > spam, I promise that over time we will become less noisy in the HDFS > channel. > > > PPS. I know lots of you will want to know more specifics; Our blog presses > are working overtime and I promise you that you will get to see all the > details pretty soon. >
