+1 for the Beta. Data pipeline and Ozone manager improvement have
certainly helped in the latest runs.
Thanks,
Mukul
On 24/02/20 8:44 am, Bharat Viswanadham wrote:
+1 for Beta given major performance improvement work went in Ozone Manager
and Datanode Pipeline.
I have been testing Teragen runs and now we have consistent runs and
performance is almost near to HDFS with disaggregated Storage and compute
cluster.
Thanks,
Bharat
On Sun, Feb 23, 2020 at 6:35 PM Sammi Chen <sammic...@apache.org> wrote:
+1, Impressive performance achievement on OzoneManager, let's move to
Beta.
Bests,
Sammi Chen
On Thu, Feb 20, 2020 at 4:17 AM Anu Engineer <aengin...@apache.org> wrote:
Hi All,
I would like to propose moving Ozone from 'Alpha' tags to 'Beta' tags
when
we do future releases. Here are a couple of reasons why I think we should
make this move.
1. Ozone Manager or the Namenode for Ozone scales to more than 1
billion
keys. We tested this in our labs in an organic fashion; that is, we
were
able to create more than 1 billion keys from external clients with no
loss
in performance.
2. The ozone Manager meets the performance and resource constraints
that
we set out to achieve. We were able to sustain the same throughput at
Ozone
manager for over three days that took us to get this 1 billion keys.
That
is, we did not have to shut down or resize memory for the namenode as
we
went through this exercise.
3. The most critical, we did this experiment with 64GB of memory
allocation in JVM and 64 GB of RAM off-heap allocation. That is, the
Ozone
Manager was able to achieve this scale with far less memory footprint
than
HDFS.
4. Ozone's performance is at par with HDFS when running workloads like
Hive (
https://blog.cloudera.com/benchmarking-ozone-clouderas-next-generation-storage-for-cdp/
)
5. We have been able to run long-running clusters with Ozone.
Having achieved these goals, I propose that we move from the planned
0.4.2-Alpha release to 0.5.0-Beta as our next release. If we hear no
concerns about this, we would like to move Ozone from Alpha to Beta
releases.
Thanks
Anu
P.S. I am CC-ing HDFS dev since many people who are interested in Ozone
still have not subscribed to Ozone dev lists. My apologies if it feels
like
spam, I promise that over time we will become less noisy in the HDFS
channel.
PPS. I know lots of you will want to know more specifics; Our blog
presses
are working overtime and I promise you that you will get to see all the
details pretty soon.
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
