Re: [DISCUSS] Next Ozone major release 1.4.0

Hi devs, I propose backporting https://issues.apache.org/jira/browse/HDDS-9807 from the master branch to the 1.4.0 branch. This issue has an impact on the use of erasure coding, so the proposed fix will be beneficial for the upcoming release. > On 16 Dec 2023, at 00:18, Tsz Wo Sze wrote: > > H

Re: [DISCUSS] Next Ozone major release 1.4.0

Hi Nicholas, @szetszwo, thank you for your reminder. I have updated the related tickets in HDDS-9929 . Please take a look. Warm Regards. -- Yiyang Zhou Tsz Wo Sze 于2023年12月16日周六 05:18写道： > Hi Yiyang, > > Thanks for creating the 1.4.0 branc

Re: [DISCUSS] Next Ozone major release 1.4.0

Hi Yiyang, Thanks for creating the 1.4.0 branch! We have 1.4.0 two blockers, HDDS-9342 and HDDS-9876, which are closely related. Tsz-Wo

Re: [DISCUSS] Next Ozone major release 1.4.0

Hello dev, I'm excited to share that the Ozone-1.4 branch is now set up and ready. Here’s what you need to know: - Ozone-1.4 Branch: All release-related tasks should now be conducted on the ozone-1.4 branch, tracked in HDDS-9928 <

Re: [DISCUSS] Next Ozone major release 1.4.0

Hello dev, Thanks @Wei-Chiu Chuang for the advice. LGTM to branch out now, if no other comments till tomorrow, I will do the branch out and version update then. For current blockers, they will be cherry-picked to branch-1.4 after being fixed. Warm Regards. -- Yiyang Zhou Wei-Chi

Re: [DISCUSS] Next Ozone major release 1.4.0

Both are fixed now. I see that master branch is still referring to 1.4.0-SNAPSHOT. I'd recommend to branch out to branch-1.4, update master branch to use 1.5.0-SNAPSHOT, so we can move into code freeze phase, allowing only blocker issues in the branch. And declare all fixed issues in master branch

Re: [DISCUSS] Next Ozone major release 1.4.0

Hi devs, @Maxim, thank you for the update. Since Datanode write performance degradation is fixed, only two tickets need to be fixed: - https://issues.apache.org/jira/browse/HDDS-9709 - https://issues.apache.org/jira/browse/HDDS-9762 Warm Regards. -- Yiyang Zhou Maxim Myskov

Re: [DISCUSS] Next Ozone major release 1.4.0

Hi devs, HDDS-9817 was an issue of our internal Ozone build. It is now closed (no code changes needed). > On 1 Dec 2023, at 13:05, Maxim Myskov wrote: > > Hi devs, > > We've identified a critical bug that we believe is a blocker for the upcomi

Re: [DISCUSS] Next Ozone major release 1.4.0

Hi devs, We've identified a critical bug that we believe is a blocker for the upcoming release. The issue is documented here: * https://issues.apache.org/jira/browse/HDDS-9817 S3 get secret endpoint returns 500 The issue is about S3 get secret endpoint returning a 500 error, causing our clien

Re: [DISCUSS] Next Ozone major release 1.4.0

Hi Janus, Thanks for working on the 1.4.0 release! For HDDS-9784, it probably is not a blocker since it is about the performance of setting various values of a conf. Users could change the conf as they please. There are trade-offs between system behaviors. It probably won't have a perfect conf

Re: [DISCUSS] Next Ozone major release 1.4.0

Hi Attila and devs, Thank you for the updates. We also found another issue related to Datanodes' write performance downgration, as mentioned in HDDS-9784 . Let's postpone the branch cut until these 3 tickets resolved. To fix tic

Re: [DISCUSS] Next Ozone major release 1.4.0

Hi Yiyang and devs, In today's US community sync two other bugs targeted at 1.4.0 were marked as blocker: * HDDS-9709. Intermittent NO_REPLICA_FOUND due to OM pipeline cache bug * HDDS-9762. Hadoop s3a protocol does not work with FSO buckets The first one should be quick: it already has a patch

Re: [DISCUSS] Next Ozone major release 1.4.0

Hi Ozone Dev, As https://issues.apache.org/jira/browse/HDDS-8876 is being fixed, there are no more blocker tickets for Ozone 1.4.0, we can start the branch cut for the release. Post the branch cut, this branch will be used for stabilizing the release, and only critical bug fixes or essent

Re: [DISCUSS] Next Ozone major release 1.4.0

Thank you, Tsz-Wo, for the summary. I'm working with pifta on these security JIRAs. For other issues, let's try to help them to move on. On Wed, 8 Nov 2023 at 02:12, Tsz Wo Sze wrote: > Hi Ozone Dev, > > Ratis just have rolled out the thirdparty 1.0.5 release. The Ratis 3.0.0 > release will co

Re: [DISCUSS] Next Ozone major release 1.4.0

Hi Ozone Dev, Ratis just have rolled out the thirdparty 1.0.5 release. The Ratis 3.0.0 release will come out soon. I will keep testing it with Ozone [1]. Currently, we have the following blockers targeted to Ozone 1.4.0: - HDDS-5230 Select best licenses for each dual licensed dependencies - HDD

Re: [DISCUSS] Next Ozone major release 1.4.0

Hello, For Ozone Jira tickets, most blocker and critical tickets have been confirmed not a blocker to the 1.4.0 release. For Ratis side, since Ratis 3.0.0 is coming to release, we'd like to include the new version of Ratis in 1.4.0, so we will cut the branch and start the release process

Re: [DISCUSS] Next Ozone major release 1.4.0

Thank you, Siyao for the update. On Sat, 26 Aug 2023 at 06:28, Siyao Meng wrote: > Hi Sammi / Ethan, > > Note the Docker arm64 image support is done in an independent repo [1], > not the main Ozone repo. > > arm64 Docker image can already be built and run by devs as we speak. And > the *on

Re: [DISCUSS] Next Ozone major release 1.4.0

Hi Sammi / Ethan, Note the Docker arm64 image support is done in an independent repo [1], not the main Ozone repo. arm64 Docker image can already be built and run by devs as we speak. And the *only* thing preventing me from pushing that arm64 Docker image right now is a (shared) Docker Hub a

Re: [DISCUSS] Next Ozone major release 1.4.0

@Ethan +1. It's good to have an arm docker image shipped with 1.4.0. Not sure how much the gap is. Bests, Sammi Chen On Thu, 24 Aug 2023 at 15:53, Slava Tutrinov wrote: > Hi everyone, > Seems we have a blocker ticket - > https://issues.apache.org/jira/browse/HDDS-9196. > Now I'm in progress o

Re: [DISCUSS] Next Ozone major release 1.4.0

Hi everyone, Seems we have a blocker ticket - https://issues.apache.org/jira/browse/HDDS-9196. Now I'm in progress of resolving the issue On Wed, Aug 23, 2023 at 11:08 PM Ethan Rose wrote: > Hi Yiyang, > > I think you will need to log in to Jira to view the Dashboard. It works for > me only afte

Re: [DISCUSS] Next Ozone major release 1.4.0

Hi Yiyang, I think you will need to log in to Jira to view the Dashboard. It works for me only after logging in. Changing the subject, do you think we would be able to have arm docker images for 1.4.0? Apple Silicon is becoming more popular for developer machines and the x86 images are too slow t

Re: [DISCUSS] Next Ozone major release 1.4.0

Hello Pifta, Seems I don't have access to the dashboard, could you help to grant the permissions? Warm Regards. -- Yiyang Zhou István Fajth 于2023年8月10日周四 19:57写道： > Hi Yiyang, > > Thank you for putting together the query, I have modified it a bit, and I > have created a Dashboard with th

Re: [DISCUSS] Next Ozone major release 1.4.0

Hi Yiyang, Thank you for putting together the query, I have modified it a bit, and I have created a Dashboard with the modified and one more query. The dashboard should be available for anyone in the ozone group, and editable for the ozone-pmc. The dashboard is here: https://issues.apache.org/jir

Re: [DISCUSS] Next Ozone major release 1.4.0

Hello, Thanks @Ethan, @Pifta for the update. I agree a dashboard is a better idea to manage the tickets. This dashboard URL, It filters the "BLOCKER" and "CRITICAL" tickets: https://issues.apache.org/jira/browse/HDDS-9046?jql=project%20%3D%20HDDS%20AND%20resolution%20%3D%20Unresolved%20a

Re: [DISCUSS] Next Ozone major release 1.4.0

Hi Janus, Thank you for summarizing and organizing these things to help us prepare for the release! HDDS-7335, and HDDS-7401 are umbrella JIRAs and they are not needed to be closed in 1.4.0. Their priority was changed for ordering reasons only. On the other hand critical items regarding rotation

Re: [DISCUSS] Next Ozone major release 1.4.0

Thanks for making this list Yiyang. For HDDS-3214 the issue is no longer valid and I have resolved it. How should that be updated in this sheet? Also, it might be easier to create a dashboard in Apache jira based on a filter for these issues, since that will automatically update as the issues are t

Re: [DISCUSS] Next Ozone major release 1.4.0

Hello, I have sorted out the open tickets with priority higher than "Critical" since 01/01/2023, hope the reporters and assignees can check if these tickets will block the release of the new version of Ozone. The tickets are in the sheet named "CriticalTickets", please have a look. ( htt

Re: [DISCUSS] Next Ozone major release 1.4.0

Hello, Thanks @Sammi, @Siddhant for the update. I will go through the open JIRA issues to figure out if there are any blockers to the release soon. Warm Regards. -- Yiyang Zhou Siddhant Sangwan 于2023年8月4日周五 14:33写道： > Hi, > > Replication Manager, EC offline recovery, and EC + Contai

Re: [DISCUSS] Next Ozone major release 1.4.0

Hi, Replication Manager, EC offline recovery, and EC + Container Balancer are ready for release. Thanks for working on this, Yiyang. Best, Siddhant On Fri, 4 Aug, 2023, 11:45 Sammi Chen, wrote: > Hi Yiyang, > > Feature owners confirmed that SCM decommission, Recon features and > stability of R

Re: [DISCUSS] Next Ozone major release 1.4.0

Hi Yiyang, Feature owners confirmed that SCM decommission, Recon features and stability of Recon improved, RM and EC are ready, the hsync feature is not ready but is off by default. Bests, Sammi On Thu, 3 Aug 2023 at 10:20, Janus Chow wrote: > Hello, > > Thanks @Pifta for the update. > >

Re: [DISCUSS] Next Ozone major release 1.4.0

Hello, Thanks @Pifta for the update. Take your time, hope to hear good news soon. Updated the Google sheet and set ETA to "17/08/2023" for now. Warm Regards. -- Yiyang Zhou István Fajth 于2023年8月2日周三 23:37写道： > Hello Yiyang, > > We have some troubles around CA certificate rotation th

Re: [DISCUSS] Next Ozone major release 1.4.0

Hello Yiyang, We have some troubles around CA certificate rotation that we are trying to solve in the coming 1-2 weeks. Regular service certificates are renewing fine until CA certificates are valid but CA certificate renewal is turned off by default at the moment, while we work on solving some in

Re: [DISCUSS] Next Ozone major release 1.4.0

Hello, Thanks @Ethan, @Prashant for the update. I have updated the "ready for release" information in the google sheet. (Ref: https://docs.google.com/spreadsheets/d/16ZiB5Aeg_bWCDuPJ1ie5BzZg2ih3dYUBObg94JOV2EM/edit?pli=1#gid=1194627601 ) Warm Regards. -- Yiyang Zhou Prashant Pogde

Re: [DISCUSS] Next Ozone major release 1.4.0

Thanks for working on the release Yiyang. Snapshot feature is also ready for release. Regards, Prashant > On Jul 26, 2023, at 12:10 AM, Janus Chow wrote: > > Hello, Dev, > >This is a addendum for last topic summary. > > Tag Count URL > Snapshot 127 https://issues.apache.org/jira/browse/H

Re: [DISCUSS] Next Ozone major release 1.4.0

Thanks for working on the release Yiyang. The container scanner and volume scanner improvements I have been working on are ready for release. Ethan On Mon, Jul 31, 2023 at 12:35 AM Janus Chow wrote: > Hello, Dev, > > For the above mentioned topics, can I ask if they are ready for the new >

Re: [DISCUSS] Next Ozone major release 1.4.0

Hello, Dev, For the above mentioned topics, can I ask if they are ready for the new release? Hope to hear from the PICs so that we can move forward with the release. Warm Regards. -- Yiyang Zhou Janus Chow 于2023年7月26日周三 15:10写道： > Hello, Dev, > > This is a addendum for last top

Re: [DISCUSS] Next Ozone major release 1.4.0

Hello, Dev, This is a addendum for last topic summary. Tag Count URL Snapshot 127 https://issues.apache.org/jira/browse/HDDS-6517 https://issues.apache.org/jira/browse/HDDS-7984 RM 74 https://issues.apache.org/jira/browse/HDDS-7759 EC 65 https

Re: [DISCUSS] Next Ozone major release 1.4.0

Hello, Dev, I have made a rough summary on the PRs since 1.3.0, there are over 900+ PRs since Jan 2023. Thank you all for the great contribution. Among the PRs, I have picked out some main topics as follows: Tag Count URL Snapshot 127 https://issues.apache.org/jira/browse/HDDS-6517 https

Re: [DISCUSS] Next Ozone major release 1.4.0

Thanks, Sammi, Siyao. I have created the ticket for the release： https://issues.apache.org/jira/browse/HDDS-8944 Feel free to add comments about the features or patches that need to be added to Ozone 1.4.0. I will wait until all comments are resolved, and then create the release branch. Thank yo

Re: [DISCUSS] Next Ozone major release 1.4.0

+1 for kicking off the 1.4.0 release process. Thanks Yiyang for volunteering to be the RM. On the snapshot feature (HDDS-6517 ) front, public APIs and proto messages should now be stable. So it wouldn't block a release with it atm. -Siyao On Ju

Re: [DISCUSS] Next Ozone major release 1.4.0

Thank you, Yiyang, for volunteering the 1.4.0 RM ! Bests, Sammi On Tue, 20 Jun 2023 at 14:50, Janus Chow wrote: > Hello, Everyone. > > I'm quite interested in this release work, if the RM is still > available, I would like to be the RM of Ozone 1.4.0. > > Warm Regards. > > -- > Yiyang Zho

Re: [DISCUSS] Next Ozone major release 1.4.0

Hello, Everyone. I'm quite interested in this release work, if the RM is still available, I would like to be the RM of Ozone 1.4.0. Warm Regards. -- Yiyang Zhou István Fajth 于2023年6月8日周四 16:08写道： > On the certificate rotation front, we are approaching a major milestone, > with > all serv

Re: [DISCUSS] Next Ozone major release 1.4.0

On the certificate rotation front, we are approaching a major milestone, with all services becoming capable of rotating their certificates without service disruption including the rootCA certificate as well. We also have resolved a few things around the need for the primordial node, we can not get

Re: [DISCUSS] Next Ozone major release 1.4.0

We can include the block token work that is in the process of being merged. That work considerably impacts performance, and delaying it to 1.5.0 will lead to a negative experience with a secure Ozone cluster for folks adopting Ozone in 1.4.0 Regards, Ritesh On Wed, Jun 7, 2023 at 9:49 AM Ethan Ros

Re: [DISCUSS] Next Ozone major release 1.4.0

+1 for a 1.4.0 release. We had hoped for a 1.3.1 release but it looks like that never gained the momentum it needed. It would be good to hear from devs working on larger features whether or not they would like to target this release. Things like snapshots, cert rotation, hsync, recon improvements e