Hi Arun, Can you assign one of the issues to me. Il be glad to help out.
Thanks, Nick. On Tue, Oct 14, 2014 at 11:25 AM, Arun C Murthy <a...@hortonworks.com> wrote: > 2.6.0 is close now. > > Here are the remaining blockers, I'm hoping cut an RC in the next week or > so: > http://s.apache.org/hadoop-2.6.0-blockers > > thanks, > Arun > > On Sep 30, 2014, at 10:42 AM, Arun C Murthy <a...@hortonworks.com> wrote: > > > Folks, > > > > I've created branch-2.6 to stabilize the release. > > > > Committers, please exercise caution henceforth on commits other than > the ones we've discussed on this thread already. > > > > By default new features should now be targeted to the version "2.7" > henceforth - I've ensure all the projects have that version on jira. > > > > thanks, > > Arun > > > > On Sep 26, 2014, at 1:08 AM, Arun Murthy <a...@hortonworks.com> wrote: > > > >> Sounds good. I'll branch this weekend and we can merge the jiras we > >> discussed in this thread as they they get wrapped next week. > >> > >> Thanks everyone. > >> > >> Arun > >> > >> > >>> On Sep 24, 2014, at 7:39 PM, Vinod Kumar Vavilapalli < > vino...@apache.org> wrote: > >>> > >>> We can branch off in a week or two so that work on branch-2 itself can > go > >>> ahead with other features that can't fit in 2.6. Independent of that, > we > >>> can then decide on the timeline of the release candidates once > branch-2.6 > >>> is close to being done w.r.t the planned features. > >>> > >>> Branching it off can let us focus on specific features that we want in > for > >>> 2.6 and then eventually blockers for the release, nothing else. There > is a > >>> trivial pain of committing to one more branch, but it's worth it in > this > >>> case IMO. > >>> > >>> A lot of efforts are happening in parallel from the YARN side from > where I > >>> see. 2.6 is a little bulky if only on the YARN side and I'm afraid if > we > >>> don't branch off and selectively try to get stuff in, it is likely to > be in > >>> a perpetual delay. > >>> > >>> My 2 cents. > >>> > >>> +Vinod > >>> > >>> On Wed, Sep 24, 2014 at 3:28 PM, Suresh Srinivas < > sur...@hortonworks.com> > >>> wrote: > >>> > >>>> Given some of the features are in final stages of stabilization, > >>>> Arun, we should hold off creating 2.6 branch or building an RC by a > week? > >>>> All the features in flux are important ones and worth delaying the > release > >>>> by a week. > >>>> > >>>> On Wed, Sep 24, 2014 at 11:36 AM, Andrew Wang < > andrew.w...@cloudera.com> > >>>> wrote: > >>>> > >>>>> Hey Nicholas, > >>>>> > >>>>> My concern about Archival Storage isn't related to the code quality > or > >>>> the > >>>>> size of the feature. I think that you and Jing did good work. My > concern > >>>> is > >>>>> that once we ship, we're locked into that set of archival storage > APIs, > >>>> and > >>>>> these APIs are not yet finalized. Simply being able to turn off the > >>>> feature > >>>>> does not change the compatibility story. > >>>>> > >>>>> I'm willing to devote time to help review these JIRAs and kick the > tires > >>>> on > >>>>> the APIs, but my point above was that I'm not sure it'd all be done > by > >>>> the > >>>>> end of the week. Testing might also reveal additional changes that > need > >>>> to > >>>>> be made, which also might not happen by end-of-week. > >>>>> > >>>>> I guess the question before us is if we're comfortable putting > something > >>>> in > >>>>> branch-2.6 and then potentially adding API changes after. I'm okay > with > >>>>> that as long as we're all aware that this might happen. > >>>>> > >>>>> Arun, as RM is this cool with you? Again, I like this feature and I'm > >>>> fine > >>>>> with it's inclusion, just a heads up that we might need some extra > time > >>>> to > >>>>> finalize things before an RC can be cut. > >>>>> > >>>>> Thanks, > >>>>> Andrew > >>>>> > >>>>> On Tue, Sep 23, 2014 at 7:30 PM, Tsz Wo (Nicholas), Sze < > >>>>> s29752-hadoop...@yahoo.com.invalid> wrote: > >>>>> > >>>>>> Hi, > >>>>>> > >>>>>> I am worry about KMS and transparent encryption since there are > quite > >>>>> many > >>>>>> bugs discovered after it got merged to branch-2. It gives us an > >>>>> impression > >>>>>> that the feature is not yet well tested. Indeed, transparent > >>>> encryption > >>>>> is > >>>>>> a complicated feature which changes the core part of HDFS. It is > not > >>>>> easy > >>>>>> to get everything right. > >>>>>> > >>>>>> > >>>>>> For HDFS-6584: Archival Storage, it is a relatively simple and low > risk > >>>>>> feature. It introduces a new storage type ARCHIVE and the concept > of > >>>>> block > >>>>>> storage policy to HDFS. When a cluster is configured with ARCHIVE > >>>>> storage, > >>>>>> the blocks will be stored using the appropriate storage types > specified > >>>>> by > >>>>>> storage policies assigned to the files/directories. Cluster admin > >>>> could > >>>>>> disable the feature by simply not configuring any storage type and > not > >>>>>> setting any storage policy as before. As Suresh mentioned, > HDFS-6584 > >>>> is > >>>>>> in the final stages to be merged to branch-2. > >>>>>> > >>>>>> Regards, > >>>>>> Tsz-Wo > >>>>>> > >>>>>> > >>>>>> > >>>>>> On Wednesday, September 24, 2014 7:00 AM, Suresh Srinivas < > >>>>>> sur...@hortonworks.com> wrote: > >>>>>> > >>>>>> > >>>>>>> > >>>>>>> > >>>>>>> I actually would like to see both archival storage and single > replica > >>>>>>> memory writes to be in 2.6 release. Archival storage is in the > final > >>>>>> stages > >>>>>>> of getting ready for branch-2 merge as Nicholas has already > indicated > >>>> on > >>>>>>> the dev mailing list. Hopefully HDFS-6581 gets ready sooner. Both > of > >>>>> these > >>>>>>> features are being in development for sometime. > >>>>>>> > >>>>>>> On Tue, Sep 23, 2014 at 3:27 PM, Andrew Wang < > >>>> andrew.w...@cloudera.com> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> Hey Arun, > >>>>>>>> > >>>>>>>> Maybe we could do a quick run through of the Roadmap wiki and > >>>>>> add/retarget > >>>>>>>> things accordingly? > >>>>>>>> > >>>>>>>> I think the KMS and transparent encryption are ready to go. We've > >>>> got > >>>>> a > >>>>>>>> very few further bug fixes pending, but that's it. > >>>>>>>> > >>>>>>>> Two HDFS things that I think probably won't make the end of the > week > >>>>> are > >>>>>>>> archival storage (HDFS-6584) and single replica memory writes > >>>>>> (HDFS-6581), > >>>>>>>> which I believe are under the HSM banner. HDFS-6484 was just > merged > >>>> to > >>>>>>>> trunk and I think needs a little more work before it goes into > >>>>> branch-2. > >>>>>>>> HDFS-6581 hasn't even been merged to trunk yet, so seems a bit > >>>> further > >>>>>> off > >>>>>>>> yet. > >>>>>>>> > >>>>>>>> Just my 2c as I did not work directly on these features. I just > >>>>>> generally > >>>>>>>> shy away from shipping bits quite this fresh. > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> Andrew > >>>>>>>> > >>>>>>>> On Tue, Sep 23, 2014 at 3:03 PM, Arun Murthy <a...@hortonworks.com > > > >>>>>> wrote: > >>>>>>>> > >>>>>>>>> Looks like most of the content is in and hadoop-2.6 is shaping up > >>>>>> nicely. > >>>>>>>>> > >>>>>>>>> I'll create branch-2.6 by end of the week and we can go from > there > >>>>> to > >>>>>>>>> stabilize it - hopefully in the next few weeks. > >>>>>>>>> > >>>>>>>>> Thoughts? > >>>>>>>>> > >>>>>>>>> thanks, > >>>>>>>>> Arun > >>>>>>>>> > >>>>>>>>> On Tue, Aug 12, 2014 at 1:34 PM, Arun C Murthy < > >>>> a...@hortonworks.com > >>>>>> > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> Folks, > >>>>>>>>>> > >>>>>>>>>> With hadoop-2.5 nearly done, it's time to start thinking ahead > >>>> to > >>>>>>>>>> hadoop-2.6. > >>>>>>>>>> > >>>>>>>>>> Currently, here is the Roadmap per the wiki: > >>>>>>>>>> > >>>>>>>>>> • HADOOP > >>>>>>>>>> • Credential provider HADOOP-10607 > >>>>>>>>>> • HDFS > >>>>>>>>>> • Heterogeneous storage (Phase 2) - Support APIs > >>>>> for > >>>>>>>>> using > >>>>>>>>>> storage tiers by the applications HDFS-5682 > >>>>>>>>>> • Memory as storage tier HDFS-5851 > >>>>>>>>>> • YARN > >>>>>>>>>> • Dynamic Resource Configuration YARN-291 > >>>>>>>>>> • NodeManager Restart YARN-1336 > >>>>>>>>>> • ResourceManager HA Phase 2 YARN-556 > >>>>>>>>>> • Support for admin-specified labels in YARN > >>>>>> YARN-796 > >>>>>>>>>> • Support for automatic, shared cache for YARN > >>>>>>>>> application > >>>>>>>>>> artifacts YARN-1492 > >>>>>>>>>> • Support NodeGroup layer topology on YARN > >>>> YARN-18 > >>>>>>>>>> • Support for Docker containers in YARN > >>>> YARN-1964 > >>>>>>>>>> • YARN service registry YARN-913 > >>>>>>>>>> > >>>>>>>>>> My suspicion is, as is normal, some will make the cut and some > >>>>>> won't. > >>>>>>>>>> Please do add/subtract from the list as appropriate. Ideally, it > >>>>>> would > >>>>>>>> be > >>>>>>>>>> good to ship hadoop-2.6 in a 6-8 weeks (say, October) to keep > >>>> up a > >>>>>>>>> cadence. > >>>>>>>>>> > >>>>>>>>>> More importantly, as we discussed previously, we'd like > >>>>> hadoop-2.6 > >>>>>> to > >>>>>>>> be > >>>>>>>>>> the *last* Apache Hadoop 2.x release which support JDK6. I'll > >>>>> start > >>>>>> a > >>>>>>>>>> discussion with other communities (HBase, Pig, Hive, Oozie etc.) > >>>>> and > >>>>>>>> see > >>>>>>>>>> how they feel about this. > >>>>>>>>>> > >>>>>>>>>> thanks, > >>>>>>>>>> Arun > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> Arun C. Murthy > >>>>>>>>> Hortonworks Inc. > >>>>>>>>> http://hortonworks.com/ > >>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> CONFIDENTIALITY NOTICE > >>>>>>>>> NOTICE: This message is intended for the use of the individual or > >>>>>> entity > >>>>>>>> to > >>>>>>>>> which it is addressed and may contain information that is > >>>>>> confidential, > >>>>>>>>> privileged and exempt from disclosure under applicable law. If > the > >>>>>> reader > >>>>>>>>> of this message is not the intended recipient, you are hereby > >>>>> notified > >>>>>>>> that > >>>>>>>>> any printing, copying, dissemination, distribution, disclosure or > >>>>>>>>> forwarding of this communication is strictly prohibited. If you > >>>> have > >>>>>>>>> received this communication in error, please contact the sender > >>>>>>>> immediately > >>>>>>>>> and delete it from your system. Thank You. > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> -- > >>>>>>> http://hortonworks.com/download/ > >>>>>>> > >>>>>>> -- > >>>>>>> CONFIDENTIALITY NOTICE > >>>>>>> NOTICE: This message is intended for the use of the individual or > >>>> entity > >>>>>> to > >>>>>>> which it is addressed and may contain information that is > >>>> confidential, > >>>>>>> privileged and exempt from disclosure under applicable law. If the > >>>>> reader > >>>>>>> of this message is not the intended recipient, you are hereby > notified > >>>>>> that > >>>>>>> any printing, copying, dissemination, distribution, disclosure or > >>>>>>> forwarding of this communication is strictly prohibited. If you > have > >>>>>>> received this communication in error, please contact the sender > >>>>>> immediately > >>>>>>> and delete it from your system. Thank You. > >>>> > >>>> > >>>> > >>>> -- > >>>> http://hortonworks.com/download/ > >>>> > >>>> -- > >>>> CONFIDENTIALITY NOTICE > >>>> NOTICE: This message is intended for the use of the individual or > entity to > >>>> which it is addressed and may contain information that is > confidential, > >>>> privileged and exempt from disclosure under applicable law. If the > reader > >>>> of this message is not the intended recipient, you are hereby > notified that > >>>> any printing, copying, dissemination, distribution, disclosure or > >>>> forwarding of this communication is strictly prohibited. If you have > >>>> received this communication in error, please contact the sender > immediately > >>>> and delete it from your system. Thank You. > >>> > >>> -- > >>> CONFIDENTIALITY NOTICE > >>> NOTICE: This message is intended for the use of the individual or > entity to > >>> which it is addressed and may contain information that is confidential, > >>> privileged and exempt from disclosure under applicable law. If the > reader > >>> of this message is not the intended recipient, you are hereby notified > that > >>> any printing, copying, dissemination, distribution, disclosure or > >>> forwarding of this communication is strictly prohibited. If you have > >>> received this communication in error, please contact the sender > immediately > >>> and delete it from your system. Thank You. > > > > -- > > Arun C. Murthy > > Hortonworks Inc. > > http://hortonworks.com/hdp/ > > > > > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/hdp/ > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. >
