+1, skimmed over the code and tried some basic stuff. Thanx Tsz-Wo for the driving this!!
-Ayush > On 06-Dec-2022, at 7:06 AM, Arpit Agarwal <a...@apache.org> wrote: > > +1 for merging this feature. This is an important performance improvement > for Ozone. > > Nicholas, what's the plan to have streaming writes enabled automatically for > large files? > > ----- Original message ----- > From: Tsz Wo Sze <szets...@gmail.com> > To: dev Ozone <dev@ozone.apache.org> > Subject: [DISCUSS] Proposal to merge HDDS-4454 Ozone Streaming Write Pipeline > into master > Date: Thursday, December 01, 2022 12:24 PM > > Dear Ozone Devs, > > We would like to start this discussion thread for the proposal to merge > https://issues.apache.org/jira/browse/HDDS-4454 Ozone Streaming Write > Pipeline to the master branch. > > Currently, Ozone write pattern is bursty and involves multiple buffers > copies as well multiple ratis log syncs in a block write. The idea of the > Jira (HDDS-4454) is to use a zero buffer copy based Ratis Streaming > (RATIS-979) in Ozone write path for better performance and resource > utilization. > > Note that HDDS-4454 is a new feature such that it is mainly adding new > code. Therefore, the risk is very low to the existing functionalities and > features in Ozone. > > For the sake of master branch feature stability, we provide a configuration > (dfs.container.ratis.datastream.enabled) to enable/disable this feature. By > default, this feature is disabled. > > For more information, please check out Ozone the Streaming Write Pipeline > (HDDS-4454) feature wiki page here: > > https://cwiki.apache.org/confluence/display/OZONE/Merge+Ozone+Streaming+Write+Pipeline+%28HDDS-4454%29+Checklist > > If there are no objections for the merge, we could start the official vote. > > Thanks, > > Tsz-Wo > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@ozone.apache.org > For additional commands, e-mail: dev-h...@ozone.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@ozone.apache.org For additional commands, e-mail: dev-h...@ozone.apache.org
