Back at this. The Hadoop thirdparty 1.1.0 is released. We have one real blocker for the 3.3.1 release (HDFS-15790 Make ProtobufRpcEngineProtos and ProtobufRpcEngineProtos2 Co-Exist <https://issues.apache.org/jira/browse/HDFS-15790>) I tried to rebase the PR but it didn't work like I wanted. Some help is needed there. If either Vinay comes back to rebase the PR, or if someone else can take over the PR, that would be great.
Meanwhile, I'll cut a branch for 3.3.1 later. If everything works out fine, I should be able to make a RC0 by tonight (which doesn't contain HDFS-15790) so you guys can quickly check out. The first RC will probably not have the aarch64 binary. I don't have much experience with that. Cheers, Wei-Chiu On Thu, May 13, 2021 at 9:04 AM Wei-Chiu Chuang <weic...@cloudera.com> wrote: > Hello it's me again. > > While working on the hadoop-thirdparty release, I would like to make > progress on the Hadoop 3.3.1 release in parallel as well. > Looking at the jira dashboard, I think the only real blocker is HDFS-15790 > https://github.com/apache/hadoop/pull/2767 > > I'll try to review the remaining jiras and push out those that don't have > much progress today. > > Thanks all! > Weichiu > > > On Tue, Apr 20, 2021 at 12:57 PM Wei-Chiu Chuang <weic...@cloudera.com> > wrote: > >> Billie and Viraj, >> >> As of today we are at 657 resolved jiras in 3.3.1 >> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HADOOP%2C%20HDFS%2C%20YARN%2C%20MAPREDUCE)%20AND%20status%20in%20(Resolved)%20AND%20fixVersion%20in%20(3.3.1) >> There are so many new features ( >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3.3+Release) I >> counted 10, that it is barely a dot release. >> >> Please make every effort to backport the said jiras to branch-3.3. I'd >> like to cut the branch sooner than later. I'd also call for more frequent >> releases. At the current development pace, a dot release every quarter >> would make sense IMO. >> >> Given the number of release blockers, I propose a tentative code freeze >> date: end of next Friday. After that I'd like to cut the branch and make an >> RC. >> >> Thoughts? I can't do this all alone. If you find things that are worth >> stopping the release, please tag the jira with the label 'release-blocker'. >> >> >> >> On Tue, Apr 20, 2021 at 2:44 AM Billie Rinaldi <bil...@apache.org> wrote: >> >>> I was thinking of backporting HADOOP-16948 for 3.3.1. >>> >>> Billie >>> >>> On Mon, Apr 19, 2021 at 1:33 AM Wei-Chiu Chuang >>> <weic...@cloudera.com.invalid> wrote: >>> >>> > Hello, reviving this thread. >>> > >>> > I created a dashboard for Hadoop 3.3.1 release. >>> > >>> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12336122 >>> > Also a jira to track the release work: HADOOP-17647 >>> > <https://issues.apache.org/jira/browse/HADOOP-17647> >>> > >>> > We are currently at 5 release blockers and 3 critical issues for Hadoop >>> > 3.3.1. I'll go through each of them and push out the ones that aren't >>> > really blocking us. >>> > >>> > If you believe there are more features/bug fixes we should include in >>> 3.3.1 >>> > (I spent the past few weeks backporting jiras but I'm sure I missed >>> some) >>> > please shout out. >>> > >>> > Meanwhile, I believe we need to release hadoop-thirdparty 1.1.0 too. >>> There >>> > are a number of tasks to be done there too. Let's start another thread >>> for >>> > hadoop-thirdparty 1.1.0 release. >>> > >>> > On Mon, Mar 15, 2021 at 7:04 PM hemanth boyina < >>> hemanthboyina...@gmail.com >>> > > >>> > wrote: >>> > >>> > > Hi Steve and Wei-Chiu >>> > > >>> > > Regarding the IPV6,Few years back we have rebased the HADOOP-11890 to >>> > trunk >>> > > and tried to work out with IPV6, we have faced some issues and have >>> made >>> > > the required changes for ipv6 to work.After the changes were made we >>> have >>> > > tested the IPV6 changes on top of Ipv4 and Ipv6 machines and tested >>> > > rigorously.Its been quite a some time these changes were deployed in >>> > > production cluster and have been in use for extensive purpose. >>> > > >>> > > I think it's good time to add this feature. >>> > > >>> > > Thanks >>> > > Hemanth Boyina >>> > > >>> > > >>> > > >>> > > On Thu, 11 Mar 2021, 10:22 Vinayakumar B, <vinayakum...@apache.org> >>> > wrote: >>> > > >>> > > > Hi David, >>> > > > >>> > > > >> Still hoping for help here: >>> > > > >>> > > > >> https://issues.apache.org/jira/browse/HDFS-15790 >>> > > > >>> > > > I will raise a PR for the said solution soon (in a day or two). >>> > > > >>> > > > -Vinay >>> > > > >>> > > > On Thu, 11 Mar 2021 at 5:39 AM, David <dam6...@gmail.com> wrote: >>> > > > >>> > > > > Hello, >>> > > > > >>> > > > > Still hoping for help here: >>> > > > > >>> > > > > https://issues.apache.org/jira/browse/HDFS-15790 >>> > > > > >>> > > > > Looks like it has been worked on, not sure how to best move it >>> > forward. >>> > > > > >>> > > > > On Wed, Mar 10, 2021, 12:21 PM Steve Loughran >>> > > > <ste...@cloudera.com.invalid >>> > > > > > >>> > > > > wrote: >>> > > > > >>> > > > > > I'm going to argue its too late to do IPv6 support close to a >>> > > release, >>> > > > as >>> > > > > > it's best if its on developer machines for some time to let all >>> > > quirks >>> > > > > > surface. It's not so much IPv6 itself, but do we cause any >>> > > regressions >>> > > > on >>> > > > > > IPv4? >>> > > > > > >>> > > > > > But: it can/should go into trunk and stabilize there >>> > > > > > >>> > > > > > On Thu, 4 Mar 2021 at 03:52, Muralikrishna Dmmkr < >>> > > > > > muralikrishna.dm...@gmail.com> wrote: >>> > > > > > >>> > > > > > > Hi Brahma, >>> > > > > > > >>> > > > > > > I have missed out mentioning about the IPV6 feature in the >>> last >>> > > mail, >>> > > > > > > Support for IPV6 has been in development since 2015 and We >>> have >>> > > done >>> > > > a >>> > > > > > good >>> > > > > > > amount of testing at our organisation, the feature is stable >>> and >>> > > used >>> > > > > by >>> > > > > > > our customers extensively in the last one year. I think it >>> is a >>> > > good >>> > > > > time >>> > > > > > > to add the IPV6 support to 3.3.1. >>> > > > > > > >>> > > > > > > https://issues.apache.org/jira/browse/HADOOP-11890 >>> > > > > > > >>> > > > > > > Thanks >>> > > > > > > D M Murali Krishna Reddy >>> > > > > > > >>> > > > > > > On Wed, Feb 24, 2021 at 9:13 AM Muralikrishna Dmmkr < >>> > > > > > > muralikrishna.dm...@gmail.com> wrote: >>> > > > > > > >>> > > > > > > > Hi Brahma, >>> > > > > > > > >>> > > > > > > > Can we have this new feature "YARN Registry based AM >>> discovery >>> > > with >>> > > > > > retry >>> > > > > > > > and in-flight task persistent via JHS" in the upcoming >>> 3.1.1 >>> > > > > release. I >>> > > > > > > > have also attached a test-report in the below jira. >>> > > > > > > > >>> > > > > > > > https://issues.apache.org/jira/browse/MAPREDUCE-6726 >>> > > > > > > > >>> > > > > > > > >>> > > > > > > > Thanks, >>> > > > > > > > D M Murali Krishna Reddy >>> > > > > > > > >>> > > > > > > > On Tue, Feb 23, 2021 at 10:11 AM Brahma Reddy Battula < >>> > > > > > bra...@apache.org >>> > > > > > > > >>> > > > > > > > wrote: >>> > > > > > > > >>> > > > > > > >> Hi Bilwa, >>> > > > > > > >> >>> > > > > > > >> I have commented on the jira's you mentioned. Based on the >>> > > > stability >>> > > > > > we >>> > > > > > > >> can >>> > > > > > > >> plan this.But needs to be merged ASAP. >>> > > > > > > >> >>> > > > > > > >> >>> > > > > > > >> >>> > > > > > > >> On Fri, Feb 19, 2021 at 5:20 PM bilwa st < >>> stbi...@gmail.com> >>> > > > wrote: >>> > > > > > > >> >>> > > > > > > >> > Hi Brahma, >>> > > > > > > >> > >>> > > > > > > >> > Can we have below features in 3.3.1 release? We have >>> been >>> > > using >>> > > > > > these >>> > > > > > > >> > features for a long time. They are stable and tested in >>> > bigger >>> > > > > > > clusters. >>> > > > > > > >> > >>> > > > > > > >> > 1. Container reuse - >>> > > > > > > >> https://issues.apache.org/jira/browse/MAPREDUCE-6749 >>> > > > > > > >> > 2. Speculative attempts should not run on the same node >>> - >>> > > > > > > >> > https://issues.apache.org/jira/browse/MAPREDUCE-7169 >>> > > > > > > >> > >>> > > > > > > >> > Thanks, >>> > > > > > > >> > Bilwa >>> > > > > > > >> > >>> > > > > > > >> > On Thu, Feb 18, 2021, 1:49 PM Brahma Reddy Battula < >>> > > > > > bra...@apache.org >>> > > > > > > > >>> > > > > > > >> > wrote: >>> > > > > > > >> > >>> > > > > > > >> >> Sorry for the late reply.. >>> > > > > > > >> >> >>> > > > > > > >> >> I will come up with a plan.. Please let me know if >>> anybody >>> > > has >>> > > > > some >>> > > > > > > >> >> features/improvements/bugs that need to be included. >>> > > > > > > >> >> >>> > > > > > > >> >> On Mon, Feb 15, 2021 at 9:39 PM Sunil Govindan < >>> > > > > sun...@apache.org> >>> > > > > > > >> wrote: >>> > > > > > > >> >> >>> > > > > > > >> >> > Hi Wei-Chiu, >>> > > > > > > >> >> > >>> > > > > > > >> >> > What will be the next steps here for 3.3.1 planning? >>> > > > > > > >> >> > >>> > > > > > > >> >> > Thanks >>> > > > > > > >> >> > Sunil >>> > > > > > > >> >> > >>> > > > > > > >> >> > On Mon, Feb 8, 2021 at 11:56 PM Stack < >>> st...@duboce.net> >>> > > > > wrote: >>> > > > > > > >> >> > >>> > > > > > > >> >> > > On Wed, Feb 3, 2021 at 6:41 AM Steve Loughran >>> > > > > > > >> >> > <ste...@cloudera.com.invalid >>> > > > > > > >> >> > > > >>> > > > > > > >> >> > > wrote: >>> > > > > > > >> >> > > >>> > > > > > > >> >> > > > >>> > > > > > > >> >> > > > Regarding blockers &c: how about we have a little >>> > > > hackathon >>> > > > > > > >> where we >>> > > > > > > >> >> > try >>> > > > > > > >> >> > > > and get things in. This means a promise of review >>> > time >>> > > > from >>> > > > > > the >>> > > > > > > >> >> people >>> > > > > > > >> >> > > with >>> > > > > > > >> >> > > > commit rights and other people who understand the >>> > code >>> > > > > > (Stack?) >>> > > > > > > >> >> > > > >>> > > > > > > >> >> > > > >>> > > > > > > >> >> > > >>> > > > > > > >> >> > > I'm up for helping get 3.3.1 out (reviewing, >>> hackathon, >>> > > > > > testing). >>> > > > > > > >> >> > > Thanks, >>> > > > > > > >> >> > > S >>> > > > > > > >> >> > > >>> > > > > > > >> >> > > >>> > > > > > > >> >> > > >>> > > > > > > >> >> > > >>> > > > > > > >> >> > > > -steve >>> > > > > > > >> >> > > > >>> > > > > > > >> >> > > > On Thu, 28 Jan 2021 at 06:48, Ayush Saxena < >>> > > > > > ayush...@gmail.com >>> > > > > > > > >>> > > > > > > >> >> wrote: >>> > > > > > > >> >> > > > >>> > > > > > > >> >> > > > > +1 >>> > > > > > > >> >> > > > > Just to mention we would need to release >>> > > > > hadoop-thirdparty >>> > > > > > > too >>> > > > > > > >> >> > before. >>> > > > > > > >> >> > > > > Presently we are using the snapshot version of >>> it. >>> > > > > > > >> >> > > > > >>> > > > > > > >> >> > > > > -Ayush >>> > > > > > > >> >> > > > > >>> > > > > > > >> >> > > > > > On 28-Jan-2021, at 6:59 AM, Wei-Chiu Chuang < >>> > > > > > > >> weic...@apache.org >>> > > > > > > >> >> > >>> > > > > > > >> >> > > > wrote: >>> > > > > > > >> >> > > > > > >>> > > > > > > >> >> > > > > > Hi all, >>> > > > > > > >> >> > > > > > >>> > > > > > > >> >> > > > > > Hadoop 3.3.0 was released half a year ago, >>> and as >>> > > of >>> > > > > now >>> > > > > > > >> we've >>> > > > > > > >> >> > > > > accumulated >>> > > > > > > >> >> > > > > > more than 400 changes in the branch-3.3. A >>> number >>> > > of >>> > > > > > > >> >> downstreamers >>> > > > > > > >> >> > > are >>> > > > > > > >> >> > > > > > eagerly waiting for 3.3.1 which addresses the >>> > guava >>> > > > > > version >>> > > > > > > >> >> > conflict >>> > > > > > > >> >> > > > > issue. >>> > > > > > > >> >> > > > > > >>> > > > > > > >> >> > > > > > >>> > > > > > > >> >> > > > > >>> > > > > > > >> >> > > > >>> > > > > > > >> >> > > >>> > > > > > > >> >> > >>> > > > > > > >> >> >>> > > > > > > >> >>> > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > > >>> > >>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20and%20fixVersion%20in%20(3.3.1)%20and%20status%20%3D%20Resolved%20 >>> > > > > > > >> >> > > > > > >>> > > > > > > >> >> > > > > > We should start the release work for 3.3.1 >>> before >>> > > the >>> > > > > > diff >>> > > > > > > >> >> becomes >>> > > > > > > >> >> > > even >>> > > > > > > >> >> > > > > > larger. >>> > > > > > > >> >> > > > > > >>> > > > > > > >> >> > > > > > I believe there are currently only two real >>> > > blockers >>> > > > > > for a >>> > > > > > > >> >> 3.3.1 >>> > > > > > > >> >> > > > (using >>> > > > > > > >> >> > > > > > this filter >>> > > > > > > >> >> > > > > > >>> > > > > > > >> >> > > > > >>> > > > > > > >> >> > > > >>> > > > > > > >> >> > > >>> > > > > > > >> >> > >>> > > > > > > >> >> >>> > > > > > > >> >>> > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > > >>> > >>> https://issues.apache.org/jira/issues/?filter=-1&jql=project%20in%20(HDFS%2C%20HADOOP%2C%20YARN%2C%20MAPREDUCE)%20AND%20cf%5B12310320%5D%20in%20(3.3.1)%20AND%20status%20not%20in%20(Resolved)%20ORDER%20BY%20priority%20DESC >>> > > > > > > >> >> > > > > > ) >>> > > > > > > >> >> > > > > > >>> > > > > > > >> >> > > > > > >>> > > > > > > >> >> > > > > > 1. HDFS-15566 < >>> > > > > > > >> >> https://issues.apache.org/jira/browse/HDFS-15566> >>> > > > > > > >> >> > > > > > 2. >>> > > > > > > >> >> > > > > > 1. HADOOP-17112 < >>> > > > > > > >> >> > > > >>> https://issues.apache.org/jira/browse/HADOOP-17112 >>> > > > > > > >> >> > > > > > >>> > > > > > > >> >> > > > > > 2. >>> > > > > > > >> >> > > > > > >>> > > > > > > >> >> > > > > > >>> > > > > > > >> >> > > > > > >>> > > > > > > >> >> > > > > > Is there anyone who would volunteer to be the >>> > 3.3.1 >>> > > > RM? >>> > > > > > > >> >> > > > > > >>> > > > > > > >> >> > > > > > Also, the HowToRelease wiki does not >>> describe the >>> > > ARM >>> > > > > > build >>> > > > > > > >> >> > process. >>> > > > > > > >> >> > > > > That's >>> > > > > > > >> >> > > > > > going to be important for future releases. >>> > > > > > > >> >> > > > > >>> > > > > > > >> >> > > > > >>> > > > > > > >> >> >>> > > > > > >>> > --------------------------------------------------------------------- >>> > > > > > > >> >> > > > > To unsubscribe, e-mail: >>> > > > > > > hdfs-dev-unsubscr...@hadoop.apache.org >>> > > > > > > >> >> > > > > For additional commands, e-mail: >>> > > > > > > >> hdfs-dev-h...@hadoop.apache.org >>> > > > > > > >> >> > > > > >>> > > > > > > >> >> > > > > >>> > > > > > > >> >> > > > >>> > > > > > > >> >> > > >>> > > > > > > >> >> > >>> > > > > > > >> >> >>> > > > > > > >> >> >>> > > > > > > >> >> -- >>> > > > > > > >> >> >>> > > > > > > >> >> >>> > > > > > > >> >> >>> > > > > > > >> >> --Brahma Reddy Battula >>> > > > > > > >> >> >>> > > > > > > >> > >>> > > > > > > >> >>> > > > > > > >> -- >>> > > > > > > >> >>> > > > > > > >> >>> > > > > > > >> >>> > > > > > > >> --Brahma Reddy Battula >>> > > > > > > >> >>> > > > > > > > >>> > > > > > > >>> > > > > > >>> > > > > >>> > > > -- >>> > > > -Vinay >>> > > > >>> > > >>> > >>> >>
