+1 for merging this feature. This is an important performance improvement for Ozone.
Nicholas, what's the plan to have streaming writes enabled automatically for large files? ----- Original message ----- From: Tsz Wo Sze <szets...@gmail.com> To: dev Ozone <dev@ozone.apache.org> Subject: [DISCUSS] Proposal to merge HDDS-4454 Ozone Streaming Write Pipeline into master Date: Thursday, December 01, 2022 12:24 PM Dear Ozone Devs, We would like to start this discussion thread for the proposal to merge https://issues.apache.org/jira/browse/HDDS-4454 Ozone Streaming Write Pipeline to the master branch. Currently, Ozone write pattern is bursty and involves multiple buffers copies as well multiple ratis log syncs in a block write. The idea of the Jira (HDDS-4454) is to use a zero buffer copy based Ratis Streaming (RATIS-979) in Ozone write path for better performance and resource utilization. Note that HDDS-4454 is a new feature such that it is mainly adding new code. Therefore, the risk is very low to the existing functionalities and features in Ozone. For the sake of master branch feature stability, we provide a configuration (dfs.container.ratis.datastream.enabled) to enable/disable this feature. By default, this feature is disabled. For more information, please check out Ozone the Streaming Write Pipeline (HDDS-4454) feature wiki page here: https://cwiki.apache.org/confluence/display/OZONE/Merge+Ozone+Streaming+Write+Pipeline+%28HDDS-4454%29+Checklist If there are no objections for the merge, we could start the official vote. Thanks, Tsz-Wo --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@ozone.apache.org For additional commands, e-mail: dev-h...@ozone.apache.org
