Re: [VOTE] Merge HDFS-7285 (erasure coding) branch to trunk

; > >design doc on HDFS-7285. > > > > >HADOOP-11264 focuses on providing flexible and high-performance > > > > >codec calculation support. > > > > > > > > > >The nightly Jenkins job of the branch has reported several > > > > &g

Re: Questions on HDFS-8880

If it's duplicate we should probably back it out, but taking a step back, is the issue that there isn't good documentation about configuring Metrics2 / FileSync? I see the API docs, but a user-focused guide on how to configure Metrics2 would probably be a welcome addition. HBase has a blog at http

Re: [DISCUSS] About the details of JDK-8 support

We've been supporting JDK8 as a runtime for CDH5 for a while now (meaning the full stack including HBase), so I agree that we're good there. I'm against dropping JDK7 support though in branch-2. Even bumping dependency versions scares me, since it often leads to downstream pain. Any comment about

Re: [DISCUSS] About the details of JDK-8 support

> > > On 7 Oct 2015, at 17:23, Andrew Wang wrote: > > > > We've been supporting JDK8 as a runtime for CDH5 for a while now (meaning > > the full stack including HBase), so I agree that we're good there. > > > > > with Kerberos on? > &g

Re: DFSClient got deadlock when close file and failed to renew lease

Hi daniedeng, Please file a JIRA at https://issues.apache.org/jira/browse/HDFS with details about your issue, and someone will take a look. Best, Andrew On Sun, Oct 18, 2015 at 6:43 PM, daniedeng(邓飞) wrote: > > > -- > daniedeng(邓飞) > > > *发件人：* daniedeng(邓飞) > *发送时

Re: [VOTE] Merge HDFS-7285 (erasure coding) branch to trunk

; > > > > I would merge them as part of HDFS-9182. > > > > > > Thanks, > > > Uma > > > > > > > > > > > > On 10/5/15, 12:48 AM, "Vinayakumar B" wrote: > > > > > > >Hi Andrew, > > > > I see CHANGE

Re: [VOTE] Release Apache Hadoop 2.6.2

during which a lot of people (atleast myself > and team) didn’t pay attention to this vote. > > Thanks > +Vinod > > > On Oct 26, 2015, at 1:50 PM, Andrew Wang andrew.w...@cloudera.com>> > wrote: > > Hey Sangjin, did you add your release signing keys to the K

Re: Erasure coding in branch-2 [Was Re: [VOTE] Merge HDFS-7285 (erasure coding) branch to trunk]

ty, integration testing and bake-in, and a realistic chance of > having it end up on users’ clusters soonish. > > +Vinod > > > On Oct 19, 2015, at 1:44 PM, Andrew Wang > wrote: > > > > I think our plan thus far has been to target this for 3.0. I'm okay with > >

Re: Erasure coding in branch-2 [Was Re: [VOTE] Merge HDFS-7285 (erasure coding) branch to trunk]

started on a 2.9. OTOH, 3.x is long ways off, > > > >and given all the incompatibilities there, it would be a while > > > >before users can get their hands on EC if it were to be only on > > > >3.x. At best, this may force sites that want EC to backport the &

Re: Erasure coding in branch-2 [Was Re: [VOTE] Merge HDFS-7285 (erasure coding) branch to trunk]

We used to get help from Bigtop when it comes to integration testing. Do we think that's possible for 2.8? On Wed, Nov 4, 2015 at 10:08 AM, Steve Loughran wrote: > > > On 2 Nov 2015, at 23:11, Vinod Vavilapalli > wrote: > > > > Yes, I’ve already started looking at 2.8.0, that is exactly how I e

Re: [DISCUSS] Looking to a 2.8.0 release

Hey Vinod, I'm fine with the idea of alpha/beta marking in the abstract, but had a question: do we define these terms in our compatibility policy or elsewhere? I think it's commonly understood among us developers (alpha means not fully tested and API unstable, beta means it's not fully tested but

Re: [DISCUSS] Looking to a 2.8.0 release

vise all newly added configuration properties to make sure they > follow our general naming patterns. New contributors sometimes create > non-standard properties that we come to regret supporting. > - Generate a list of newly added public entry-points and validate that > they are all ind

Re: [DISCUSS] Looking to a 2.8.0 release

> > > maybe discuss having a list @ release time. As an example, s3 and > encryption at rest shipped in beta stage... what's in 2.8 that "we don't > yet trust ourselves?". Me, I'd put erasure coding in there just because > I've no familiarity with it > > Quick clarification, EC isn't scheduled for

Re: TestDirectoryScanner.testThrottle() Failures

Done On Wed, Dec 16, 2015 at 4:17 PM, Daniel Templeton wrote: > Would someone please review and commit HDFS-9300 so that the > testThrottle() test will stop failing. It's a 2-line patch. > > Thanks, > Daniel >

Re: [VOTE] Release Apache Hadoop 2.7.2 RC1

My 2c is that we should have monotonicity in releases. That way no "upgrade" is a regression. On Wed, Dec 23, 2015 at 10:00 PM, Tsuyoshi Ozawa wrote: > Hi Vinod, > > thank you for the clarification. > > > - Pull these 16 tickets into 2.7.2 and roll a new RC > > > What do people think? Do folks

Re: [VOTE] Release Apache Hadoop 2.7.2 RC1

I like monotonic releases since it's simple for users to understand. Is it difficult to backport to 2.7.x if you're already backporting to 2.6.x? I don't follow why special casing some class of fixes is desirable. Also for maintenance releases, aren't all included fixes supposed to be for serious

Re: [VOTE] Release Apache Hadoop 2.7.2 RC1

On Mon, Jan 11, 2016 at 7:22 AM, Junping Du wrote: > bq. Is it difficult to backport to 2.7.x if you're already backporting to > 2.6.x? I don't follow why special casing some class of fixes is desirable. > It is not difficult to backport the commits between 2.6.x and 2.7.x. > However, it do *dif

Re: [VOTE] Release hadoop-2.0.3-alpha

Verified the tarball checksums. Ran a couple example jobs on a 3 node cluster successfully, with the same WARN caveat as Bobby. +1 (non-binding). On Thu, Feb 7, 2013 at 7:33 AM, Robert Evans wrote: > I downloaded the binary package and ran a few example jobs on a 3 node > cluster. Everything se

are the HDFS javadocs published on the website?

Hi all, I think something changed recently regarding the online HDFS javadocs. I'm fairly sure they used to be available online, since it's indexed by google: https://www.google.com/?q=inurl:distributedfilesystem++site%3Ahadoop.apache.org However, all of those results 404 now. Going to the curr

Re: are the HDFS javadocs published on the website?

ut a redirect in to the HDFS user guide to > fix those two broken links. If folks prefer that approach I'd be > happy to implement it. > > Doug > > On Thu, Feb 14, 2013 at 3:48 PM, Andrew Wang > wrote: > > Hi all, > > > > I think something changed r

Re: Unable to delete symlinks in HDFS via FileContext

Hi Dia, That's definitely a weird one, let's try to figure out what's going on. Can you possibly share the complete FileContext snippet you're using to do this test? You could also try using fully-qualified URIs everywhere (including for symlink creation), to remove any possible ambiguity. Also,

Re: transfer -> CreateSocketForPipeline : hardcoded length of 2?

Hi Jay, Actually, my question on seeing that code is wondering why it's hardcoded to 2, rather than targets.length. The pipeline length is supposed to be the number of datanodes in the pipeline. This might be a bug. Regarding the timeout, it makes sense to boost the timeout based on the length of

Re: Why is FileSystem.createNonRecursive deprecated?

Hi Ravi, I wasn't around for HADOOP-6840, but I'm guessing it's deprecated for the same reasons as primitiveCreate: FileSystem is supposed to eventually to be supplanted by FileContext. FileContext#create also has a more manageable number of method signatures through the use of flags, and in fact

Re: Feature request to provide DFSInputStream subclassing mechanism

I don't think exposing DFSClient and DistributedFileSystem members is necessary to achieve what you're trying to do. We've got wrapper FileSystems like FilterFileSystem and ViewFileSystem which you might be able to use for inspiration, and the HCFS wiki lists some third-party FileSystems that might

Re: Feature request to provide DFSInputStream subclassing mechanism

any changes to Hadoop > needed to make it work. In other words we will be willing to maintain any > implementation changes that may become necessary as Hadoop internals change > in future releases. > > Thanks, > Jeff > > > On 8/7/13 11:30 AM, Andrew Wang wrote:

Re: Secure deletion of blocks

Hi Matt, Here are some code pointers: - When doing a file deletion, the NameNode turns the file into a set of blocks that need to be deleted. - When datanodes heartbeat in to the NN (see BPServiceActor#offerService), the NN replies with blocks to be invalidated (see BlockCommand and DatanodeProto

Re: hsync is too slower than hflush

50ms is believable. hsync makes each DN call fsync and wait for acks, so you'd expect at least a disk seek time (~10ms) with some extra time depending on how much unsync'd data is being written. So, just as some back of the envelope math, assuming a disk that can write at 100MB/s: 50ms - 10ms see

Re: hsync is too slower than hflush

g > Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > > > On Monday, August 26, 2013 at 7:07 AM, Andrew Wang wrote: > > > 50ms is believable. hsync makes each DN call fsync and wait for acks, so > > you'd expect at least a disk seek time (~10ms) with some extra time &

Re: hsync is too slower than hflush

h Sparrow (http://www.sparrowmailapp.com/?sig) > > > > > > On Monday, August 26, 2013 at 11:18 AM, Andrew Wang wrote: > > > > > Ah, I forgot the checksum fsync, so two seeks. Even with 4k writes, > 50ms > > > still feels in the right ballpark. Best case

symlink support in Hadoop 2 GA

Hi all, I wanted to broadcast plans for putting the FileSystem symlinks work (HADOOP-8040) into branch-2.1 for the pending Hadoop 2 GA release. I think it's pretty important we get it in since it's not a compatible change; if it misses the GA train, we're not going to have symlinks until the next

Re: [VOTE] Release Apache Hadoop 2.1.1-beta

Hey all, Sorry to hijack the vote thread, but it'd be good to get some input on my email from yesterday re: symlink support in branch-2.1. I think it really should be in GA one way or the other. http://mail-archives.apache.org/mod_mbox/hadoop-common-dev/201309.mbox/%3CCAGB5D2ZDjqt69oFfv_HOsWEH18T

Re: symlink support in Hadoop 2 GA

by adding possibly newer APIs and > leaving > > existing APIs as is. If this can be done, my vote is to enable this > feature > > in 2.3. Even if it cannot be done, I am concerned that this is coming > quite > > late and we should see if could allow some incompatible cha

Re: symlink support in Hadoop 2 GA

It's an incompatible change. Existing APIs like listStatus and globStatus need to be symlink aware now, which can break assumptions of user code. We've had FileStatus#isSymlink() since the early days, but lots of user code hasn't been updated to use it. I think Eli's earlier email did a good job a

Re: [VOTE] Release Apache Hadoop 2.1.1-beta

We still need to resolve some symlink issues; are we planning to spin a new RC? Leaving it as-is is not a good option. On Sun, Sep 22, 2013 at 11:23 PM, Roman Shaposhnik wrote: > On Mon, Sep 16, 2013 at 11:38 PM, Arun C Murthy > wrote: > > Folks, > > > > I've created a release candidate (rc0)

Re: [VOTE] Release Apache Hadoop 2.1.1-beta

Hey Arun, That plan sounds good to me, thanks for being on top of things. What's the new fix version we should be using (2.1.2 or 2.2.0)? Would be good to get the same clarification regarding which branches should be receiving commits. I think a 2.1.2 would be nice to get the symlinks changes in a

Re: 2.1.2 (Was: Re: [VOTE] Release Apache Hadoop 2.1.1-beta)

HADOOP-9984 is going to break interface compatibility for out-of-tree FileSystems. It'd also be good to let downstream components do some testing before GA. Thanks, Andrew On Tue, Oct 1, 2013 at 5:18 PM, Jagane Sundar wrote: > +1 > Makes good sense. > > Jagane > > -Original Message- >

Re: 2.1.2 (Was: Re: [VOTE] Release Apache Hadoop 2.1.1-beta)

If we're serious about not breaking compatibility after GA, then we need to slow down and make sure we get these new APIs right, or can add them in a compatible fashion. HADOOP-9984 ended up being a bigger change than initially expected, and we need to break compatibility with out-of-tree FileSyst

Re: symlink support in Hadoop 2 GA

Colin posted a summary of our phone call yesterday (attendees: myself, Colin, Daryn, Nathan, Jason, Chris, Suresh, Sanjay) on HADOOP-9984: https://issues.apache.org/jira/browse/HADOOP-9984?focusedCommentId=13785701&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13785

Re: About Block related classes in hdfs package

Hey Yoonmin, Unfortunately I agree it's a bit complex, especially because "Block" is sometimes used where "Replica" might be more accurate. If you find any ambiguities like this, I think we'd happily take patches with clarifying comments / javadoc. The best way to learn is to read the code, but m

[VOTE] Merge HDFS-4949 to trunk

Hello all, I'd like to call a vote to merge the HDFS-4949 branch (in-memory caching) to trunk. Colin McCabe and I have been hard at work the last 3.5 months implementing this feature, and feel that it's reached a level of stability and utility where it's ready for broader testing and integration.

Re: Managing docs with hadoop-1 & hadoop-2

Hey folks, I've been seeing some reports about search results for Hadoop being broken because stable now points to the v2 docs, where a lot of stuff has moved around. e.g. http://hadoop.apache.org/docs/stable/fair_scheduler.html (404, first result on google for "hadoop fair scheduler") http://ha

Re: [VOTE] Merge HDFS-4949 to trunk

shot test > >> plan > >> > that was posted to HDFS-2802. For my own part, I've run the new unit > >> > tests, and I've tested end-to-end in a pseudo-distributed deployment. > >> It's > >> > unlikely that I'll get a chance to test fully di

Re: Replacing the JSP web UIs to HTML 5 applications

I'm also not convinced that a Javascript-based approach is the way to go. We shouldn't switch the default UI until (at a minimum) we have the command-line tools that Colin requested, and even then I'd still want to retain support for text-based browsers like elinks unless there are compelling techn

Re: HDFS read/write data throttling

Hey Lohit, This is an interesting topic, and something I actually worked on in grad school before coming to Cloudera. It'd help if you could outline some of your usecases and how per-FileSystem throttling would help. For what I was doing, it made more sense to throttle on the DN side since you hav

Re: HDFS read/write data throttling

t; > If someone were to do IOP throttling in the 3.x+ timeline, > >1. It needs clear use cases, YARN containers being #1 for me >2. We'd have to look at all the research done on this in the past to see >what works, doesn't > > Andrew, what citations of relevance

Re: HDFS read/write data throttling

n Wed, Nov 13, 2013 at 2:54 AM, Steve Loughran wrote: > this is interesting -I've moved my comments over to the JIRA and it would > be good for yours to go there too. > > is there a URL for your paper? > > > On 13 November 2013 06:27, Andrew Wang wrote: > > > Hey Ste

Re: HDFS read/write data throttling

https://issues.apache.org/jira/browse/HDFS-5499 On Mon, Nov 18, 2013 at 10:46 AM, Jay Vyas wrote: > Where is the jira for this? > > Sent from my iPhone > > > On Nov 18, 2013, at 1:25 PM, Andrew Wang > wrote: > > > > Thanks for asking, here's a link:

Re: when datanode will delete these invalidate blocks?

Try looking in the heartbeat code on the NN and DN, it should clear things up. The namenode sends these block invalidations to the DN on the DN heartbeat response. The DN then deletes the blocks and on the next heartbeat reports to the NN that it invalidated the blocks. The NN then removes the inva

Re: issue about rpc activity metrics

The metrics system generates a number of different entries per in-code metrics object. For instance, the "SendHeartbeat" MutableRate will generate both "NumOps" and "AvgTime". Look in NameNodeMetrics.java for where these are updated. Best, Andrew On Tue, Nov 19, 2013 at 10:52 PM, ch huang wrot

Re: Metrics2 code

Hey LiuLei, Gauges can go up and down, counters only go up. Snapshot doesn't actually reset anything, it's just a way for the metrics system to get an updated value. There aren't any time-based rolling metrics to my knowledge besides MutableQuantiles. Best, Andrew On Wed, Nov 20, 2013 at 7:34 P

Re: [VOTE] Merge HDFS-2832 Heterogeneous Storage Phase 1 to trunk

to trunk, we will continue to test and fix any bugs that may be > found on trunk as well as add further tests as outlined in the test plan. > > The bulk of the design and implementation was done by Suresh Srinivas, > Sanjay Radia, Nicholas Sze, Junping Du and me. Also, thanks to Eric

Re: [VOTE] Merge HDFS-2832 Heterogeneous Storage Phase 1 to trunk

port which can > > align with the 2.5 time frame, with the second merge potentially in > > March/April. > > > > Arpit > > > > > > On Fri, Dec 6, 2013 at 3:15 PM, Andrew Wang > >wrote: > > > > > Hi everyone, > > > > > >

Re: persistent under-replicated blocks

Hi Chris, BCC'ing hdfs-dev@ since you're using CDH, moving us to cdh-user@. You should be able to manually copy the under-replicated blocks and md5 files to a different datanode and restart it. I'm curious that you're having this issue though, I haven't encountered it before. Can you send your NN

Re: Re-swizzle 2.3

I just finished tuning up branch-2.3 and fixing up the HDFS and Common CHANGES.txt in trunk, branch-2, and branch-2.3. I had to merge back a few JIRAs committed between the swizzle and now where the fix version was 2.3 but weren't in branch-2.3. I think the only two HDFS and Common JIRAs that are

Re: Re-swizzle 2.3

APREDUCE's CHANGES.txt on trunk, branch-2 and branch-2.3. Let me know > if you find some inaccuracies. > > > > Thanks, > > +Vinod > > > > On Jan 29, 2014, at 10:49 PM, Vinod Kumar Vavilapalli < > vino...@apache.org> wrote: > > > >> > &g

Re: DISCUSS: Hadoop Compatability Guidelines

There's also the DataNode data directory layout. FS edit logs should also be included if we're including the fsimage. Historically we've bumped these in minor and major releases, though I'm not sure whether precedent supports the practice. It means you can't downgrade, and features that need metad

Re: [VOTE] Merge yarn-native-services branch into trunk

Hi folks, This vote closes today. I see a -1 from Allen on inclusion in beta1. I see there's active fixing going on, but given that we're one week out from RC0, I think we should drop this from beta1. Allen, Jian, others, is this reasonable? What release should we retarget this for? I don't have

2017-09-07 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-09-07 Slightly early update since I'll be out tomorrow. We're one week out, and focus is on blocker burndown. Highlights: - 3.1.0 release planning is underway, led by Wangda. Target release date is in

Re: [VOTE] Merge yarn-native-services branch into trunk

is too close to the release, after talking with other team > members, we are fine to drop this from beta, > > And we want to target this for GA. > I’m withdrawing this vote and will start afresh vote later for GA. > Thanks all who voted this effort ! > > Thanks, > Jian >

2017-09-19 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-09-19 Sorry for the late update. We're down to one blocker and one EC must do! Made great progress over the last week and a bit. We will likely cut RC0 this week. Highlights: - Down to just two blocker i

Re: [DISCUSS] moving to Apache Yetus Audience Annotations

Is this itself an incompatible change? I imagine the bytecode will be different. I think we're too late to do this for beta1 given that I want to cut an RC0 today. On Fri, Sep 22, 2017 at 7:03 AM, Sean Busbey wrote: > When Apache Yetus formed, it started with several key pieces of Hadoop that >

Re: [DISCUSS] moving to Apache Yetus Audience Annotations

get in for 3.0, I can make a jira and put it > on the back burner for when trunk goes to 4.0? > > On Fri, Sep 22, 2017 at 12:49 PM, Andrew Wang > wrote: > >> Is this itself an incompatible change? I imagine the bytecode will be >> different. >> >> I think we&#

2017-09-22 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-09-22 We've had some late breaking blockers related to Docker support that are delaying the release. We're on a day-by-day slip at this point. Highlights: - I did a successful test create-release earlie

Heads up: branching branch-3.0.0-beta1 off of branch-3.0

Hi folks, We've driven the blocker count down to 0, and I went through and made sure the fix versions and release notes and so on are all lined up. I'm going to cut branch-3.0.0-beta1 off branch-3.0 and try and get RC0 out today. Cheers, Andrew

Re: Heads up: branching branch-3.0.0-beta1 off of branch-3.0

Branch has been cut, branch-3.0 is now open for commits for 3.0.0 GA. HEAD of branch-3.0.0-beta1 is 2223393ad1d5ffdd62da79e1546de79c6259dc12. On Thu, Sep 28, 2017 at 10:52 AM, Andrew Wang wrote: > Hi folks, > > We've driven the blocker count down to 0, and I went through and m

[VOTE] Release Apache Hadoop 3.0.0-beta1 RC0

Hi all, Let me start, as always, by thanking the many, many contributors who helped with this release! I've prepared an RC0 for 3.0.0-beta1: http://home.apache.org/~wang/3.0.0-beta1-RC0/ This vote will run five days, ending on Nov 3rd at 5PM Pacific. beta1 contains 576 fixed JIRA issues compris

Re: [DISCUSS] Merging API-based scheduler configuration to trunk/branch-2

Hi Jonathan, I'm okay with putting this into branch-3.0 for GA if it can be merged within the next two weeks. Even though beta1 has slipped by a month, I want to stick to the targeted GA data of Nov 1st as much as possible. Of course, let's not sacrifice quality or stability for speed; if somethin

2017-09-20 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-09-29 After about a month of slip, RC0 has been sent out for a VOTE. Focus now turns to GA, where we will attempt to keep the original beta1 target date (early November). Highlights: - RC0 vote was sent o

Re: [VOTE] Release Apache Hadoop 3.0.0-beta1 RC0

2017 at 9:45 AM, Brahma Reddy Battula wrote: > > Thanks Andrew. > > +1 (non binding) > > --Built from source > --installed 3 node HA cluster > --Verified shell commands and UI > --Ran wordcount/pic jobs > > > > > On Fri, 29 Sep 2017 at 5:34 AM, Andrew Wang

Re: [VOTE] Release Apache Hadoop 3.0.0-beta1 RC0

mode for KMS and HttpFs >> default HTTP >> > servlet /conf, /stacks, and /logLevel == >> > >> > One example below: >> > >> > >Connecting to >> > > https://localhost:14000/logLevel?log=org.apache.hadoop.fs. >

Re: [VOTE] Release Apache Hadoop 3.0.0-beta1 RC0

M HA/work-preservring-restart/ NM-work-preserving restart > for ATSv2 entities. > - verified all ATSv2 REST end points to retrieve the entities > - ran sample MR jobs and distributed jobs > > Thanks & Regards > Rohith Sharma K S > > On 4 October 2017 at 05:31, Andrew Wan

2017-10-06 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-10-06 The beta1 RC0 vote passed, and beta1 is out! Now tracking GA features. Highlights: - 3.0.0-beta1 has been released! - Router-based federation merge vote should be about to pass - API-based sch

Re: 2017-10-06 Hadoop 3 release status update

for beta1, and we've still got plenty of other blockers/criticals to work on for GA. If you feel differently, please reach out, I can make myself very available next week for a call. Best, Andrew On Fri, Oct 6, 2017 at 3:12 PM, Allen Wittenauer wrote: > > > On Oct 6, 2017,

2017-10-20 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-10-20 Apologies for skipping the update last week. Here's how we're tracking for GA. Highlights: - Merge of HDFS router-based federation and API-based scheduler configuration with no reported problems.

Re: Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

FWIW we've been running branch-3.0 unit tests successfully internally, though we have separate jobs for Common, HDFS, YARN, and MR. The failures here are probably a property of running everything in the same JVM, which I've found problematic in the past due to OOMs. On Tue, Oct 24, 2017 at 4:04 PM

2017-10-31 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+ updates 2017-10-31 Lots of progress towards GA, we look on track for cutting RC0 this week. I ran the versions script to check the branch matches up with JIRA and fixed things up, and also checked that the changelog and r

Re: [DISCUSS] A final minor release off branch-2?

What are the known gaps that need bridging between 2.x and 3.x? >From an HDFS perspective, we've tested wire compat, rolling upgrade, and rollback. >From a YARN perspective, we've tested wire compat and rolling upgrade. Arun just mentioned an NM rollback issue that I'm not familiar with. Anythin

Heads up: branching branch-3.0.0 for GA

Hi folks, We've resolved all the blockers for 3.0.0 and the release notes and changelog look good, so I'm going to cut the branch and get started on the RC. * branch-3.0 will advance to 3.0.1-SNAPSHOT * branch-3.0.0 will go to 3.0.0 Please keep this in mind when committing. Cheers, Andrew

Re: Heads up: branching branch-3.0.0 for GA

Branching is complete. Please use the 3.0.1 fix version for further commits to branch-3.0. Ping me if you want something in branch-3.0.0 since I'm rolling RC0 now. On Tue, Nov 14, 2017 at 11:08 AM, Andrew Wang wrote: > Hi folks, > > We've resolved all the blockers for 3.

[VOTE] Release Apache Hadoop 3.0.0 RC0

Hi folks, Thanks as always to the many, many contributors who helped with this release. I've created RC0 for Apache Hadoop 3.0.0. The artifacts are available here: http://people.apache.org/~wang/3.0.0-RC0/ This vote will run 5 days, ending on Nov 19th at 1:30pm Pacific. 3.0.0 GA contains 291 fi

Re: [DISCUSS] A final minor release off branch-2?

To follow up on my earlier email, I don't think there's need for a bridge release given that we've successfully tested rolling upgrade from 2.x to 3.0.0. I expect we'll keep making improvements to smooth over any additional incompatibilities found, but there isn't a requirement that a user upgrade

Re: [DISCUSS] A final minor release off branch-2?

Hi Junping, On Wed, Nov 15, 2017 at 1:37 AM, Junping Du wrote: > Thanks Vinod to bring up this discussion, which is just in time. > > I agree with most responses that option C is not a good choice as our > community bandwidth is precious and we should focus on very limited > mainstream branches

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

gt; > On 2017/11/15 6:34, Andrew Wang wrote: > >> Hi folks, >> >> Thanks as always to the many, many contributors who helped with this >> release. I've created RC0 for Apache Hadoop 3.0.0. The artifacts are >> available here: >> >> http://people.apache.

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

to our broader > community and I see no harm in giving everyone a few days to evaluate it > more thoroughly. > > On a lighter note, your deadline is also 4 minutes short of the required 5 > days. :) > > Regards, > Arpit > > > > On 11/14/17, 1:34 PM, "Andrew Wa

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

ind 'resource-types.xml'. >>> >>> Side notes >>> >>> - When did we stop putting CHANGES files into the source artifacts? >>> - Even after "mvn install"ing once, shading is repeated again and again >>> for every new 'mv

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

Thanks for the thorough review Vinod, some inline responses: *Issues found during testing* > > Major > - The previously supported way of being able to use different tar-balls > for different sub-modules is completely broken - common and HDFS tar.gz are > completely empty. > Is this something peo

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

On Mon, Nov 20, 2017 at 9:59 PM, Sangjin Lee wrote: > > On Mon, Nov 20, 2017 at 9:46 PM, Andrew Wang > wrote: > >> Thanks for the spot Sangjin. I think this bug introduced in >> create-release by HADOOP-14835. The multi-pass maven build generates these >> dummy cli

Re: Apache Hadoop 2.8.3 Release Plan

I'm against including new features in maintenance releases, since they're meant to be bug-fix only. If we're struggling with being able to deliver new features in a safe and timely fashion, let's try to address that, not overload the meaning of "maintenance release". Best, Andrew On Mon, Nov 20,

Re: Apache Hadoop 2.8.3 Release Plan

> > > >> If we're struggling with being able to deliver new features in a safe > and timely fashion, let's try to address that... > > This is interesting. Do you aware any means to do that? Thanks! > > I've mentioned this a few times on the lists before, but our biggest gap in keeping branches rele

Re: Apache Hadoop 2.8.3 Release Plan

hough I saw some discussions are going on > at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes > this improvement, then users consuming this feature/improvement have no 2.9 > release to upgrade or forcefully upgrade with regression. We may need a > better upgrade

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

On Mon, Nov 20, 2017 at 11:33 PM, Allen Wittenauer wrote: > > The original release script and instructions broke the build up > into three or so steps. When I rewrote it, I kept that same model. It’s > probably time to re-think that. In particular, it should probably be one > big step th

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

Hi folks, Thanks again for the testing help with the RC. Here's our dashboard for the 3.0.0 release: https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12329849 Right now we're tracking three blockers: * HADOOP-15058 is the create-release fix, I just put up a patch which needs re

2017-12-01 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-12-01 Haven't written one of these in a month. I had high hopes for RC0, but it failed due to HADOOP-15058 - create-release site build output

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

FYI that we got our last blocker in today, so I'm currently rolling RC1. Stay tuned! On Thu, Nov 30, 2017 at 8:32 AM, Allen Wittenauer wrote: > > > On Nov 30, 2017, at 1:07 AM, Rohith Sharma K S < > rohithsharm...@apache.org> wrote: > > > > > > >. If ATSv1 isn’t replaced by ATSv2, then why is it

[VOTE] Release Apache Hadoop 3.0.0 RC1

Hi all, Let me start, as always, by thanking the efforts of all the contributors who contributed to this release, especially those who jumped on the issues found in RC0. I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302 fixed JIRAs since the previous 3.0.0-beta1 release. Y

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Sangjin Lee wrote: > > > > Hi Andrew, > > > > Thanks much for your effort! Just to be clear, could you please state the > > git commit id of the RC1 we're voting for? > > > > Sangjin > > > > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang >

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

y of tags. > > On Mon, Dec 11, 2017 at 10:31 AM, Andrew Wang > wrote: > >> Sorry, forgot to push the tag. It's up there now. >> >> On Sun, Dec 10, 2017 at 8:31 PM, Vinod Kumar Vavilapalli < >> vino...@apache.org> wrote: >> >>> I couldn'

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

ant to double check so we don't > ship anything private to you in public :) > > > > On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar > wrote: > >> +1 (non-binding) >> Thanks for driving this, Andrew Wang!! >> >> - downloaded the src tarball and verified m

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

t;/" with XOR-2-1 policy. > > Thanks Andrew for this great effort! > > Best, > > > On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang > wrote: > > Hi Wei-Chiu, > > > > The patchprocess directory is left over from the create-release process, > > and

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

ng it a stable release. > > > > > > Thanks, > > > > Junping > > > > > > From: Robert Kanter > > Sent: Tuesday, December 12, 2017 3:10 PM > > To: Arun Suresh > > Cc: Andrew Wan

  1   2   3   4   5   6   7   8   >