+1 to merge from me. Disclaimer: I have contributed design, code, and review work to this feature branch.
Stephen. On Tue, Sep 29, 2026 at 12:01 AM Ethan Rose <[email protected]> wrote: > Hi Ozone devs, > > This is a vote thread to merge the zero downtime upgrade feature branch ( > HDDS-14496-zdu) into master, which will enable zero downtime upgrades from > the first release containing this feature to all future releases. Please > familiarize yourself with the following documents when voting. All docs PRs > remain open so we can address questions and comments as they arise during > the voting process. > > - Design document <https://github.com/apache/ozone/pull/9664> > - Branch merge checklist <https://github.com/apache/ozone-site/pull/562 > > > - User Docs <https://github.com/apache/ozone-site/pull/545> > - Developer docs <https://github.com/apache/ozone-site/pull/560>: This > guide covers how to handle compatibility for zero downtime upgrade and > should be understood by all developers and reviewers. > > Summary of changes: New Requirements for Developers > > *Once this branch is merged, all further commits must be ZDU compatible as > outlined in the developer guide*. > > The new versioning framework created on this branch provides tools to > safely incorporate incompatible changes, but it does not automatically > resolve them. That remains the responsibility of developers and reviewers. > Improved Developer Experience > > Each component now uses a single ComponentVersion to track all incompatible > changes across disk and network as outlined in the design doc. Developers > no longer need to reason about whether their incompatible change requires a > LayoutFeature, ComponentVersion, or both. As part of this change, the > internal upgrade framework was rewritten and exposes a simpler API to > developers. This includes strongly typed component versions (with integer > conversion deferred until serialization), and an isSupportedBy method to > handle all version comparisons. > Improved Admin Experience > > To ensure finalization proceeds in the correct order as outlined in the > design document, admins no longer have to finalize OM and SCM separately. A > single ozone admin upgrade finalize command triggers asynchronous > finalization throughout the cluster in the defined order. A single status > endpoint can be queried by ozone admin upgrade status, and clients can > trigger and block on finalization with one ozone admin upgrade finalize > --wait command, which handles polling of the status endpoint by the client > automatically and is idempotent. > > Additionally, a Grafana dashboard has been added to provide a heads up view > of all components during an upgrade. It includes filtering to zoom in on a > particular area and aggregates across Datanodes to handle large clusters. > Removed Prepare For Upgrade > > The "prepare for upgrade" command which put OMs in a read-only mode before > an upgrade is no longer required. The CLI has been left as a no-op for > compatibility with older upgrade scripts. See the developer guide linked > above for instructions to handle incompatible changes to OM write requests. > Work In Progress > > There is some ongoing work we will continue in parallel with the merge vote > and finish before the branch is merged: > > - We are currently merging the latest master branch into the feature > branch, resolving conflicts, and running it through CI. The final hash > for > merge will be shared here when ready. > - The ZDU design doc will be updated based on the latest copilot review > and other minor deviations identified from the resulting implementation. > - A PR to add a missed admin check on the ozone admin upgrade status > command is in flight: https://github.com/apache/ozone/pull/11354 > > Future Work > > As mentioned in the merge checklist, the current OM request versioning > framework on master was left intact on the ZDU branch. However, the opt-in > annotation based approach does not suit the new ZDU requirements where > every new request needs to be versioned. Dev work has started on a new > framework, but the change is large and was deliberately saved for master > after the branch merge so that it can be reviewed by a wider audience. > > Additionally, we will be investigating static analysis and AI skills to > flag potentially incompatible changes during code reviews and the release > process. > ------------------------------ > > Thanks to Stephen, Zita, and Roland who also worked on the development of > this feature and to everyone who shared inputs on the design. > > We will leave the vote thread open for at least 7 days, although more time > may be required for developers to familiarize themselves with the new ZDU > requirements. > > - Ethan >
