Thanks for making this list Yiyang. For HDDS-3214 the issue is no longer valid and I have resolved it. How should that be updated in this sheet? Also, it might be easier to create a dashboard in Apache jira based on a filter for these issues, since that will automatically update as the issues are triaged.
Ethan On Mon, Aug 7, 2023 at 8:13 PM Janus Chow <yiyang0...@gmail.com> wrote: > Hello, > > I have sorted out the open tickets with priority higher than "Critical" > since 01/01/2023, hope the reporters and assignees can check if these > tickets will block the release of the new version of Ozone. > > The tickets are in the sheet named "CriticalTickets", please have a > look. ( > > https://docs.google.com/spreadsheets/d/16ZiB5Aeg_bWCDuPJ1ie5BzZg2ih3dYUBObg94JOV2EM/edit#gid=889472519 > ) > > > Warm Regards. > > -- > Yiyang Zhou > > > Janus Chow <yiyang0...@gmail.com> 于2023年8月4日周五 16:10写道： > > > Hello, > > > > Thanks @Sammi, @Siddhant for the update. > > > > I will go through the open JIRA issues to figure out if there are any > > blockers to the release soon. > > > > Warm Regards. > > > > -- > > Yiyang Zhou > > > > > > Siddhant Sangwan <siddhantsangwan...@gmail.com> 于2023年8月4日周五 14:33写道： > > > >> Hi, > >> > >> Replication Manager, EC offline recovery, and EC + Container Balancer > are > >> ready for release. Thanks for working on this, Yiyang. > >> > >> Best, > >> Siddhant > >> > >> On Fri, 4 Aug, 2023, 11:45 Sammi Chen, <sammic...@apache.org> wrote: > >> > >> > Hi Yiyang, > >> > > >> > Feature owners confirmed that SCM decommission, Recon features and > >> > stability of Recon improved, RM and EC are ready, the hsync feature > is > >> > not ready but is off by default. > >> > > >> > Bests, > >> > Sammi > >> > > >> > On Thu, 3 Aug 2023 at 10:20, Janus Chow <yiyang0...@gmail.com> wrote: > >> > > >> > > Hello, > >> > > > >> > > Thanks @Pifta for the update. > >> > > > >> > > Take your time, hope to hear good news soon. Updated the Google > >> sheet > >> > > and set ETA to "17/08/2023" for now. > >> > > > >> > > Warm Regards. > >> > > > >> > > -- > >> > > Yiyang Zhou > >> > > > >> > > > >> > > István Fajth <fapi...@gmail.com> 于2023年8月2日周三 23:37写道： > >> > > > >> > > > Hello Yiyang, > >> > > > > >> > > > We have some troubles around CA certificate rotation that we are > >> trying > >> > > to > >> > > > solve in the coming 1-2 weeks. Regular service certificates are > >> > renewing > >> > > > fine until CA certificates are valid but CA certificate renewal is > >> > turned > >> > > > off by default at the moment, while we work on solving some > >> interesting > >> > > > timing issues that arise with the current solution. I would love > to > >> > have > >> > > > this fixed and released in 1.4 as a feature that is full on by > >> default > >> > if > >> > > > security is enabled, but yes, we need some time to get there. > >> > > > > >> > > > Pifta > >> > > > > >> > > > Janus Chow <yiyang0...@gmail.com> ezt írta (időpont: 2023. aug. > 2., > >> > Sze, > >> > > > 3:33): > >> > > > > >> > > > > Hello, > >> > > > > > >> > > > > Thanks @Ethan, @Prashant for the update. > >> > > > > > >> > > > > I have updated the "ready for release" information in the > >> google > >> > > > > sheet. (Ref: > >> > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > https://docs.google.com/spreadsheets/d/16ZiB5Aeg_bWCDuPJ1ie5BzZg2ih3dYUBObg94JOV2EM/edit?pli=1#gid=1194627601 > >> > > > > ) > >> > > > > > >> > > > > > >> > > > > Warm Regards. > >> > > > > > >> > > > > -- > >> > > > > Yiyang Zhou > >> > > > > > >> > > > > > >> > > > > Prashant Pogde <ppo...@cloudera.com.invalid> 于2023年8月2日周三 > >> 07:18写道： > >> > > > > > >> > > > > > Thanks for working on the release Yiyang. Snapshot feature is > >> also > >> > > > ready > >> > > > > > for release. > >> > > > > > > >> > > > > > Regards, > >> > > > > > Prashant > >> > > > > > > >> > > > > > > On Jul 26, 2023, at 12:10 AM, Janus Chow < > >> yiyang0...@gmail.com> > >> > > > wrote: > >> > > > > > > > >> > > > > > > Hello, Dev, > >> > > > > > > > >> > > > > > > This is a addendum for last topic summary. > >> > > > > > > > >> > > > > > > Tag Count URL > >> > > > > > > Snapshot 127 > https://issues.apache.org/jira/browse/HDDS-6517 > >> > > > > > > https://issues.apache.org/jira/browse/HDDS-7984 > >> > > > > > > <https://issues.apache.org/jira/browse/HDDS-6517> > >> > > > > > > RM 74 https://issues.apache.org/jira/browse/HDDS-7759 > >> > > > > > > EC 65 https://issues.apache.org/jira/browse/HDDS-6462 > >> > > > > > > Recon 39 > >> > > > > > > Certificate 27 > >> https://issues.apache.org/jira/browse/HDDS-7391 > >> > > > > > > Direct copy 25 > >> https://issues.apache.org/jira/browse/HDDS-2274 > >> > > > > > > Hsync 23 https://issues.apache.org/jira/browse/HDDS-7593 > >> > > > > > > > >> > > > > > > > >> > > > > > > Warm Regards. > >> > > > > > > > >> > > > > > > -- > >> > > > > > > Yiyang Zhou > >> > > > > > > > >> > > > > > > > >> > > > > > > Janus Chow <yiyang0...@gmail.com> 于2023年7月26日周三 15:00写道： > >> > > > > > > > >> > > > > > >> Hello, Dev, > >> > > > > > >> > >> > > > > > >> I have made a rough summary on the PRs since 1.3.0, > there > >> are > >> > > > over > >> > > > > > >> 900+ PRs since Jan 2023. Thank you all for the great > >> > contribution. > >> > > > > > >> > >> > > > > > >> Among the PRs, I have picked out some main topics as > >> follows: > >> > > > > > >> > >> > > > > > >> Tag Count URL > >> > > > > > >> Snapshot 127 > https://issues.apache.org/jira/browse/HDDS-6517 > >> > > > > > >> https://issues.apache.org/jira/browse/HDDS-7984 > >> > > > > > >> <https://issues.apache.org/jira/browse/HDDS-6517> > >> > > > > > >> RM 74 https://issues.apache.org/jira/browse/HDDS-7759 > >> > > > > > >> EC 65 https://issues.apache.org/jira/browse/HDDS-6462 > >> > > > > > >> Recon 39 > >> > > > > > >> Certificate 27 > >> https://issues.apache.org/jira/browse/HDDS-7391 > >> > > > > > >> Direct copy 25 > >> https://issues.apache.org/jira/browse/HDDS-2274 > >> > > > > > >> I will try to check with the PICs of these topics to see > >> if > >> > > these > >> > > > > > >> projects are about to be suitable for a release cut. > >> > > > > > >> > >> > > > > > >> Welcome to add or supplement any missing information. > >> > > > > > >> > >> > > > > > >> Thank you again for the great work. > >> > > > > > >> > >> > > > > > >> > >> > > > > > >> > >> > > > > > >> Ref: > >> > > > > > >> > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > https://docs.google.com/spreadsheets/d/16ZiB5Aeg_bWCDuPJ1ie5BzZg2ih3dYUBObg94JOV2EM/edit?usp=sharing > >> > > > > > >> > >> > > > > > >> Warm Regards. > >> > > > > > >> > >> > > > > > >> -- > >> > > > > > >> Yiyang Zhou > >> > > > > > >> > >> > > > > > >> > >> > > > > > >> Janus Chow <yiyang0...@gmail.com> 于2023年6月28日周三 10:58写道： > >> > > > > > >> > >> > > > > > >>> Thanks, Sammi, Siyao. > >> > > > > > >>> > >> > > > > > >>> I have created the ticket for the release： > >> > > > > > >>> https://issues.apache.org/jira/browse/HDDS-8944 > >> > > > > > >>> > >> > > > > > >>> Feel free to add comments about the features or patches > that > >> > need > >> > > > to > >> > > > > be > >> > > > > > >>> added to Ozone 1.4.0. I will wait until all comments are > >> > > resolved, > >> > > > > and > >> > > > > > then > >> > > > > > >>> create the release branch. > >> > > > > > >>> > >> > > > > > >>> Thank you all. > >> > > > > > >>> > >> > > > > > >>> Warm Regards. > >> > > > > > >>> > >> > > > > > >>> -- > >> > > > > > >>> Yiyang Zhou > >> > > > > > >>> > >> > > > > > >>> > >> > > > > > >>> Siyao Meng <si...@apache.org> 于2023年6月28日周三 05:18写道： > >> > > > > > >>> > >> > > > > > >>>> +1 for kicking off the 1.4.0 release process. > >> > > > > > >>>> > >> > > > > > >>>> Thanks Yiyang for volunteering to be the RM. > >> > > > > > >>>> > >> > > > > > >>>> On the snapshot feature (HDDS-6517 > >> > > > > > >>>> <https://issues.apache.org/jira/browse/HDDS-6517>) > front, > >> > > public > >> > > > > APIs > >> > > > > > >>>> and > >> > > > > > >>>> proto messages should now be stable. So it wouldn't > block a > >> > > > release > >> > > > > > with > >> > > > > > >>>> it > >> > > > > > >>>> atm. > >> > > > > > >>>> > >> > > > > > >>>> > >> > > > > > >>>> -Siyao > >> > > > > > >>>> > >> > > > > > >>>> On Jun 26, 2023 at 9:59:15 PM, Sammi Chen < > >> > sammic...@apache.org > >> > > > > >> > > > > > wrote: > >> > > > > > >>>> > >> > > > > > >>>>> Thank you, Yiyang, for volunteering the 1.4.0 RM ! > >> > > > > > >>>>> > >> > > > > > >>>>> Bests, > >> > > > > > >>>>> Sammi > >> > > > > > >>>>> > >> > > > > > >>>>> On Tue, 20 Jun 2023 at 14:50, Janus Chow < > >> > yiyang0...@gmail.com > >> > > > > >> > > > > > wrote: > >> > > > > > >>>>> > >> > > > > > >>>>> Hello, Everyone. > >> > > > > > >>>>> > >> > > > > > >>>>> > >> > > > > > >>>>> I'm quite interested in this release work, if the RM > is > >> > > still > >> > > > > > >>>>> > >> > > > > > >>>>> available, I would like to be the RM of Ozone 1.4.0. > >> > > > > > >>>>> > >> > > > > > >>>>> > >> > > > > > >>>>> Warm Regards. > >> > > > > > >>>>> > >> > > > > > >>>>> > >> > > > > > >>>>> -- > >> > > > > > >>>>> > >> > > > > > >>>>> Yiyang Zhou > >> > > > > > >>>>> > >> > > > > > >>>>> > >> > > > > > >>>>> > >> > > > > > >>>>> István Fajth <fapi...@gmail.com> 于2023年6月8日周四 16:08写道： > >> > > > > > >>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>> On the certificate rotation front, we are approaching a > >> > major > >> > > > > > >>>> milestone, > >> > > > > > >>>>> > >> > > > > > >>>>>> with > >> > > > > > >>>>> > >> > > > > > >>>>>> all services becoming capable of rotating their > >> certificates > >> > > > > without > >> > > > > > >>>>> > >> > > > > > >>>>>> service disruption including the rootCA certificate as > >> well. > >> > > > > > >>>>> > >> > > > > > >>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>> We also have resolved a few things around the need for > >> the > >> > > > > > primordial > >> > > > > > >>>>> > >> > > > > > >>>>> node, > >> > > > > > >>>>> > >> > > > > > >>>>>> we can not get rid of the need of a special node during > >> the > >> > > > first > >> > > > > > >>>>> > >> > > > > > >>>>> bootstrap > >> > > > > > >>>>> > >> > > > > > >>>>>> of > >> > > > > > >>>>> > >> > > > > > >>>>>> the PKI system, but after that the special node is not > >> > needed > >> > > > > > >>>> anymore, > >> > > > > > >>>>> > >> > > > > > >>>>> and > >> > > > > > >>>>> > >> > > > > > >>>>>> the > >> > > > > > >>>>> > >> > > > > > >>>>>> leader SCM will be able to initiate the rotation of CA > >> > > > > certificates > >> > > > > > >>>> we > >> > > > > > >>>>> > >> > > > > > >>>>> have > >> > > > > > >>>>> > >> > > > > > >>>>>> in > >> > > > > > >>>>> > >> > > > > > >>>>>> the system. > >> > > > > > >>>>> > >> > > > > > >>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>> For us the next big thing is handling the certificate > >> > > > revocation, > >> > > > > > >>>> also to > >> > > > > > >>>>> > >> > > > > > >>>>>> do > >> > > > > > >>>>> > >> > > > > > >>>>>> some further code cleanup and simplification, it would > be > >> > nice > >> > > > to > >> > > > > > >>>> have it > >> > > > > > >>>>> > >> > > > > > >>>>>> released soon after it is ready, but as we do not have > it > >> > > right > >> > > > > now > >> > > > > > >>>>> > >> > > > > > >>>>> either > >> > > > > > >>>>> > >> > > > > > >>>>>> we can live through another release without it. It is a > >> bit > >> > > > > > >>>> unrealistic > >> > > > > > >>>>> > >> > > > > > >>>>> to > >> > > > > > >>>>> > >> > > > > > >>>>>> have it included in a 1.5.0 release if it comes out > >> within > >> > the > >> > > > > next > >> > > > > > >>>> 1-2 > >> > > > > > >>>>> > >> > > > > > >>>>>> month. > >> > > > > > >>>>> > >> > > > > > >>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>> But the finished certificate rotation feature is > somewhat > >> > > > > mandatory, > >> > > > > > >>>> as > >> > > > > > >>>>> > >> > > > > > >>>>>> there > >> > > > > > >>>>> > >> > > > > > >>>>>> are changes in how we store certificates, and even > though > >> > the > >> > > > > > startup > >> > > > > > >>>>> > >> > > > > > >>>>> needs > >> > > > > > >>>>> > >> > > > > > >>>>>> to > >> > > > > > >>>>> > >> > > > > > >>>>>> and will handle the old format, we would like to > >> introduce a > >> > > > > change > >> > > > > > >>>> and > >> > > > > > >>>>> > >> > > > > > >>>>>> transform the metadata directory structure during an > >> upgrade > >> > > > > > >>>>> > >> > > > > > >>>>> finalization. > >> > > > > > >>>>> > >> > > > > > >>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>> Pifta > >> > > > > > >>>>> > >> > > > > > >>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>> Ritesh Shukla <rit...@cloudera.com.invalid> ezt írta > >> > > (időpont: > >> > > > > > 2023. > >> > > > > > >>>>> > >> > > > > > >>>>> jún. > >> > > > > > >>>>> > >> > > > > > >>>>>> 7., Sze, 18:53): > >> > > > > > >>>>> > >> > > > > > >>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>>> We can include the block token work that is in the > >> process > >> > of > >> > > > > being > >> > > > > > >>>>> > >> > > > > > >>>>>> merged. > >> > > > > > >>>>> > >> > > > > > >>>>>>> That work considerably impacts performance, and > >> delaying it > >> > > to > >> > > > > > >>>> 1.5.0 > >> > > > > > >>>>> > >> > > > > > >>>>> will > >> > > > > > >>>>> > >> > > > > > >>>>>>> lead to a negative experience with a secure Ozone > >> cluster > >> > for > >> > > > > > >>>>> > >> > > > > > >>>>>>> folks adopting Ozone in 1.4.0 > >> > > > > > >>>>> > >> > > > > > >>>>>>> Regards, > >> > > > > > >>>>> > >> > > > > > >>>>>>> Ritesh > >> > > > > > >>>>> > >> > > > > > >>>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>>> On Wed, Jun 7, 2023 at 9:49 AM Ethan Rose > >> > > > > > >>>> <er...@cloudera.com.invalid> > >> > > > > > >>>>> > >> > > > > > >>>>>>> wrote: > >> > > > > > >>>>> > >> > > > > > >>>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>>>> +1 for a 1.4.0 release. We had hoped for a 1.3.1 > >> release > >> > but > >> > > > it > >> > > > > > >>>> looks > >> > > > > > >>>>> > >> > > > > > >>>>>>> like > >> > > > > > >>>>> > >> > > > > > >>>>>>>> that never gained the momentum it needed. It would be > >> good > >> > > to > >> > > > > > >>>> hear > >> > > > > > >>>>> > >> > > > > > >>>>> from > >> > > > > > >>>>> > >> > > > > > >>>>>>>> devs working on larger features whether or not they > >> would > >> > > like > >> > > > > to > >> > > > > > >>>>> > >> > > > > > >>>>>> target > >> > > > > > >>>>> > >> > > > > > >>>>>>>> this release. Things like snapshots, cert rotation, > >> hsync, > >> > > > recon > >> > > > > > >>>>> > >> > > > > > >>>>>>>> improvements etc. From my end I'm planning to have > v1of > >> > the > >> > > > > > >>>> container > >> > > > > > >>>>> > >> > > > > > >>>>>>>> scanner done by the end of this month. It would be > >> nice to > >> > > > have > >> > > > > > >>>> in > >> > > > > > >>>>> > >> > > > > > >>>>> the > >> > > > > > >>>>> > >> > > > > > >>>>>>> next > >> > > > > > >>>>> > >> > > > > > >>>>>>>> release but not essential. > >> > > > > > >>>>> > >> > > > > > >>>>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>>>> Also worth noting that doing a major release will > lock > >> in > >> > > > protos > >> > > > > > >>>> and > >> > > > > > >>>>> > >> > > > > > >>>>>> disk > >> > > > > > >>>>> > >> > > > > > >>>>>>>> structures as they are on master for compatibility > >> > reasons. > >> > > We > >> > > > > > >>>> have > >> > > > > > >>>>> > >> > > > > > >>>>>> tried > >> > > > > > >>>>> > >> > > > > > >>>>>>>> to keep the master branch always releasable, but just > >> an > >> > FYI > >> > > > to > >> > > > > > >>>> devs > >> > > > > > >>>>> > >> > > > > > >>>>>>>> working on larger tasks on master right now. > >> > > > > > >>>>> > >> > > > > > >>>>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>>>> Ethan > >> > > > > > >>>>> > >> > > > > > >>>>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>>>> On Wed, Jun 7, 2023 at 1:44 AM Sammi Chen < > >> > > > sammic...@apache.org > >> > > > > > > >> > > > > > >>>>> > >> > > > > > >>>>>> wrote: > >> > > > > > >>>>> > >> > > > > > >>>>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>>>>> Dear Ozone Devs, > >> > > > > > >>>>> > >> > > > > > >>>>>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>>>>> It's been almost 5 month since the last major 1.3.0 > >> > release > >> > > > in > >> > > > > > >>>> Dec > >> > > > > > >>>>> > >> > > > > > >>>>>>> 2022. > >> > > > > > >>>>> > >> > > > > > >>>>>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>>>>> In the past 5 month, there have been a lot of issues > >> > fixed > >> > > > and > >> > > > > > >>>>> > >> > > > > > >>>>>>>> improvements > >> > > > > > >>>>> > >> > > > > > >>>>>>>>> made, together with new features, like httpFs, scm > >> > > > > > >>>> decommission, > >> > > > > > >>>>> > >> > > > > > >>>>>> Recon > >> > > > > > >>>>> > >> > > > > > >>>>>>>> new > >> > > > > > >>>>> > >> > > > > > >>>>>>>>> functions, snapshot, EC balancer support, the the > >> coming > >> > > > > > >>>> symmetric > >> > > > > > >>>>> > >> > > > > > >>>>>>> block > >> > > > > > >>>>> > >> > > > > > >>>>>>>>> token and certificate rotation, etc. > >> > > > > > >>>>> > >> > > > > > >>>>>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>>>>> So far, there are already 733 JIRAs resolved on > 1.4.0. > >> > > > > > >>>>> > >> > > > > > >>>>>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>> > >> > > > > > >>>>> > >> > > > > > >>>> > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > https://issues.apache.org/jira/issues/?jql=project+%3D+HDDS+and+fixVersion+%3D+1.4.0 > >> > > > > > >>>>> > >> > > > > > >>>>>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>>>>> Usually it will take months to do a major release. > So > >> I > >> > > > > > >>>> propose to > >> > > > > > >>>>> > >> > > > > > >>>>>>> start > >> > > > > > >>>>> > >> > > > > > >>>>>>>>> the discussion of 1.4.0 now, things like which > >> features > >> > > > should > >> > > > > > >>>> be > >> > > > > > >>>>> > >> > > > > > >>>>>>>> involved > >> > > > > > >>>>> > >> > > > > > >>>>>>>>> in this release. > >> > > > > > >>>>> > >> > > > > > >>>>>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>>>>> And this release doesn't have a Release Manager yet. > >> > > Welcome > >> > > > > > >>>>> > >> > > > > > >>>>>>>>> anyone(Comitters or PMCs, for it requires some > >> privileges > >> > > on > >> > > > > > >>>> Apache > >> > > > > > >>>>> > >> > > > > > >>>>>>>>> facilities to do the release) to volunteer to be the > >> RM > >> > of > >> > > > > > >>>> 1.4.0. > >> > > > > > >>>>> > >> > > > > > >>>>>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>>>>> Regards, > >> > > > > > >>>>> > >> > > > > > >>>>>>>>> Sammi > >> > > > > > >>>>> > >> > > > > > >>>>>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>>> -- > >> > > > > > >>>>> > >> > > > > > >>>>>> Pifta > >> > > > > > >>>>> > >> > > > > > >>>>>> > >> > > > > > >>>>> > >> > > > > > >>>>> > >> > > > > > >>>>> > >> > > > > > >>>> > >> > > > > > >>> > >> > > > > > > >> > > > > > > >> > > > > > > >> > --------------------------------------------------------------------- > >> > > > > > To unsubscribe, e-mail: dev-unsubscr...@ozone.apache.org > >> > > > > > For additional commands, e-mail: dev-h...@ozone.apache.org > >> > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > >> > > > -- > >> > > > Pifta > >> > > > > >> > > > >> > > >> > > >
