Re: HADOOP-15124 review

I think it's fine to ask for review help on the dev list, sometimes JIRAs are missed or get stuck. It also helps sometimes to git blame the files you're touching and see who is a likely reviewer, and then pinging them on JIRA to ask for a review. Igor, hopefully a committer takes interest in your

[jira] [Created] (HDFS-13616) Batch listing of multiple directories

Andrew Wang created HDFS-13616: -- Summary: Batch listing of multiple directories Key: HDFS-13616 URL: https://issues.apache.org/jira/browse/HDFS-13616 Project: Hadoop HDFS Issue Type: New

[jira] [Created] (HDFS-13611) Unsafe use of Text as a ConcurrentHashMap key in PBHelperClient

Andrew Wang created HDFS-13611: -- Summary: Unsafe use of Text as a ConcurrentHashMap key in PBHelperClient Key: HDFS-13611 URL: https://issues.apache.org/jira/browse/HDFS-13611 Project: Hadoop HDFS

[jira] [Created] (HDFS-13601) Optimize ByteString conversions in PBHelper

Andrew Wang created HDFS-13601: -- Summary: Optimize ByteString conversions in PBHelper Key: HDFS-13601 URL: https://issues.apache.org/jira/browse/HDFS-13601 Project: Hadoop HDFS Issue Type: Bug

Re: [VOTE] Adopt HDSL as a new Hadoop subproject

+1 If this VOTE is to gather consensus about establishing a new subproject, let's definitely proceed with that. It sounds like we're already discussing changes to the details of how the project will be run, and releasing from the branch vs. maven profile is not a blocker for me. I raised it since

Re: [VOTE] Adopt HDSL as a new Hadoop subproject

?), I feel like we should settle these questions first. Best, Andrew On Mar 22, 2018 9:51 AM, "Chris Douglas" wrote: +1 (binding) This compromise seems to address most of the concerns raised during the discussion. Thanks for proposing and driving this, Owen. On Thu, Mar 22, 2018 at 9:

Re: [VOTE] Adopt HDSL as a new Hadoop subproject

elease process. Thanks Anu On 3/21/18, 2:18 PM, "Andrew Wang" wrote: Hi folks, Sorry for not replying earlier, I've been on vacation for the last week and am writing this on my phone from the airport now. Please excuse me if this is terser than my normal emails.

Re: [VOTE] Adopt HDSL as a new Hadoop subproject

Hi folks, Sorry for not replying earlier, I've been on vacation for the last week and am writing this on my phone from the airport now. Please excuse me if this is terser than my normal emails. I really like the direction of Owen's proposal, thanks Owen for driving this toward a conclusion accept

Re: [VOTE] Merging branch HDFS-7240 to trunk

Hi Sanjay, thanks for the response, replying inline: - NN on top HDSL where the NN uses the new block layer (Both Daryn and Owen > acknowledge the benefit of the new block layer). We have two choices here > ** a) Evolve NN so that it can interact with both old and new block layer, > ** b) Fork

Re: [VOTE] Merging branch HDFS-7240 to trunk

Thanks, > Wangda > > > On Fri, Mar 2, 2018 at 4:24 PM, Owen O'Malley > wrote: > >> On Thu, Mar 1, 2018 at 11:03 PM, Andrew Wang >> wrote: >> >> Owen mentioned making a Hadoop subproject; we'd have to >> > hash out what exactly this means (

Re: [VOTE] Merging branch HDFS-7240 to trunk

Hi Sanjay, I have different opinions about what's important and how to eventually integrate this code, and that's not because I'm "conveniently ignoring" your responses. I'm also not making some of the arguments you claim I am making. Attacking arguments I'm not making is not going to change my mi

Re: [VOTE] Merging branch HDFS-7240 to trunk

actoring required to separate the FSN and BM and make it pluggable. At that point (likely in the Hadoop 4 timeframe), we'll be ready to pursue HDFS-on-HDSL integration. Best, Andrew On Tue, Feb 27, 2018 at 11:23 PM, Andrew Wang wrote: > > > > > > > > > > *Hi J

Re: [VOTE] Merging branch HDFS-7240 to trunk

*Hi Jitendra and all,Thanks for putting this together. I caught up on the discussion on JIRA and document at HDFS-10419, and still have the same concerns raised earlier

Re: [DISCUSS] 2.9+ stabilization branch

3. > GPU support would be a serious catalyst for us to move forward, which I > also heard from other organizations interested in ML. > > Thanks, > --Konstantin > > On Tue, Feb 27, 2018 at 1:28 PM, Andrew Wang > wrote: > >> Hi Arun/Subru, >> >> Bumping the mi

Re: [DISCUSS] 2.9+ stabilization branch

Hi Arun/Subru, Bumping the minimum Java version is a major change, and incompatible for users who are unable to upgrade their JVM version. We're beyond the EOL for Java 7, but as we know from our experience with Java 6, there are plenty of users who stick on old Java versions. Bumping the Java ver

Re: Apache Hadoop 3.0.1 Release plan

Hi Eddy, thanks for taking this on, Historically we've waited for the first RC to cut the release branch since it keeps things simpler for committers. Also, could you check the permissions on your JIRA filter? It shows as private for me. Best, Andrew On Tue, Jan 9, 2018 at 11:17 AM, Lei Xu wro

Re: [ANNOUNCE] Apache Hadoop 3.0.0 GA is released

for prior releases. > > Thanks, > Jonathan Kelly > > On Thu, Dec 14, 2017 at 10:45 AM Andrew Wang > wrote: > >> Hi all, >> >> I'm pleased to announce that Apache Hadoop 3.0.0 is generally available >> (GA). >> >> 3.0.0 GA consis

Re: [ANNOUNCE] Apache Hadoop 3.0.0 GA is released

pit Agarwal wrote: > That makes sense for Beta users but most of our users will be upgrading > from a previous GA release and the changelog will mislead them. The webpage > does not mention this is a delta from the beta release. > > > > > > *From: *Andrew Wang > *Da

Re: [ANNOUNCE] Apache Hadoop 3.0.0 GA is released

sume you only counted 3.0.0 changes in this log excluding > alphas/betas. However, users shouldn’t have to manually compile > incompatibilities by summing up a/b release notes. Can we fix the changelog > after the fact? > > > > > On 12/14/17, 10:45 AM, "Andrew Wang

[ANNOUNCE] Apache Hadoop 3.0.0 GA is released

Hi all, I'm pleased to announce that Apache Hadoop 3.0.0 is generally available (GA). 3.0.0 GA consists of 302 bug fixes, improvements, and other enhancements since 3.0.0-beta1. This release marks a point of quality and stability for the 3.0.0 release line, and users of earlier 3.0.0-alpha and -b

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

ng it a stable release. > > > > > > Thanks, > > > > Junping > > > > > > From: Robert Kanter > > Sent: Tuesday, December 12, 2017 3:10 PM > > To: Arun Suresh > > Cc: Andrew Wan

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

t;/" with XOR-2-1 policy. > > Thanks Andrew for this great effort! > > Best, > > > On Tue, Dec 12, 2017 at 9:55 AM, Andrew Wang > wrote: > > Hi Wei-Chiu, > > > > The patchprocess directory is left over from the create-release process, > > and

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

ant to double check so we don't > ship anything private to you in public :) > > > > On Tue, Dec 12, 2017 at 7:48 AM, Ajay Kumar > wrote: > >> +1 (non-binding) >> Thanks for driving this, Andrew Wang!! >> >> - downloaded the src tarball and verified m

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

y of tags. > > On Mon, Dec 11, 2017 at 10:31 AM, Andrew Wang > wrote: > >> Sorry, forgot to push the tag. It's up there now. >> >> On Sun, Dec 10, 2017 at 8:31 PM, Vinod Kumar Vavilapalli < >> vino...@apache.org> wrote: >> >>> I couldn'

Re: [VOTE] Release Apache Hadoop 3.0.0 RC1

Sangjin Lee wrote: > > > > Hi Andrew, > > > > Thanks much for your effort! Just to be clear, could you please state the > > git commit id of the RC1 we're voting for? > > > > Sangjin > > > > On Fri, Dec 8, 2017 at 12:31 PM, Andrew Wang >

[VOTE] Release Apache Hadoop 3.0.0 RC1

Hi all, Let me start, as always, by thanking the efforts of all the contributors who contributed to this release, especially those who jumped on the issues found in RC0. I've prepared RC1 for Apache Hadoop 3.0.0. This release incorporates 302 fixed JIRAs since the previous 3.0.0-beta1 release. Y

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

FYI that we got our last blocker in today, so I'm currently rolling RC1. Stay tuned! On Thu, Nov 30, 2017 at 8:32 AM, Allen Wittenauer wrote: > > > On Nov 30, 2017, at 1:07 AM, Rohith Sharma K S < > rohithsharm...@apache.org> wrote: > > > > > > >. If ATSv1 isn’t replaced by ATSv2, then why is it

2017-12-01 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-12-01 Haven't written one of these in a month. I had high hopes for RC0, but it failed due to HADOOP-15058 - create-release site build output

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

Hi folks, Thanks again for the testing help with the RC. Here's our dashboard for the 3.0.0 release: https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12329849 Right now we're tracking three blockers: * HADOOP-15058 is the create-release fix, I just put up a patch which needs re

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

On Mon, Nov 20, 2017 at 11:33 PM, Allen Wittenauer wrote: > > The original release script and instructions broke the build up > into three or so steps. When I rewrote it, I kept that same model. It’s > probably time to re-think that. In particular, it should probably be one > big step th

Re: Apache Hadoop 2.8.3 Release Plan

hough I saw some discussions are going on > at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes > this improvement, then users consuming this feature/improvement have no 2.9 > release to upgrade or forcefully upgrade with regression. We may need a > better upgrade

Re: Apache Hadoop 2.8.3 Release Plan

> > > >> If we're struggling with being able to deliver new features in a safe > and timely fashion, let's try to address that... > > This is interesting. Do you aware any means to do that? Thanks! > > I've mentioned this a few times on the lists before, but our biggest gap in keeping branches rele

Re: Apache Hadoop 2.8.3 Release Plan

I'm against including new features in maintenance releases, since they're meant to be bug-fix only. If we're struggling with being able to deliver new features in a safe and timely fashion, let's try to address that, not overload the meaning of "maintenance release". Best, Andrew On Mon, Nov 20,

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

On Mon, Nov 20, 2017 at 9:59 PM, Sangjin Lee wrote: > > On Mon, Nov 20, 2017 at 9:46 PM, Andrew Wang > wrote: > >> Thanks for the spot Sangjin. I think this bug introduced in >> create-release by HADOOP-14835. The multi-pass maven build generates these >> dummy cli

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

Thanks for the thorough review Vinod, some inline responses: *Issues found during testing* > > Major > - The previously supported way of being able to use different tar-balls > for different sub-modules is completely broken - common and HDFS tar.gz are > completely empty. > Is this something peo

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

ind 'resource-types.xml'. >>> >>> Side notes >>> >>> - When did we stop putting CHANGES files into the source artifacts? >>> - Even after "mvn install"ing once, shading is repeated again and again >>> for every new 'mv

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

to our broader > community and I see no harm in giving everyone a few days to evaluate it > more thoroughly. > > On a lighter note, your deadline is also 4 minutes short of the required 5 > days. :) > > Regards, > Arpit > > > > On 11/14/17, 1:34 PM, "Andrew Wa

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

gt; > On 2017/11/15 6:34, Andrew Wang wrote: > >> Hi folks, >> >> Thanks as always to the many, many contributors who helped with this >> release. I've created RC0 for Apache Hadoop 3.0.0. The artifacts are >> available here: >> >> http://people.apache.

Re: [DISCUSS] A final minor release off branch-2?

Hi Junping, On Wed, Nov 15, 2017 at 1:37 AM, Junping Du wrote: > Thanks Vinod to bring up this discussion, which is just in time. > > I agree with most responses that option C is not a good choice as our > community bandwidth is precious and we should focus on very limited > mainstream branches

Re: [DISCUSS] A final minor release off branch-2?

To follow up on my earlier email, I don't think there's need for a bridge release given that we've successfully tested rolling upgrade from 2.x to 3.0.0. I expect we'll keep making improvements to smooth over any additional incompatibilities found, but there isn't a requirement that a user upgrade

[VOTE] Release Apache Hadoop 3.0.0 RC0

Hi folks, Thanks as always to the many, many contributors who helped with this release. I've created RC0 for Apache Hadoop 3.0.0. The artifacts are available here: http://people.apache.org/~wang/3.0.0-RC0/ This vote will run 5 days, ending on Nov 19th at 1:30pm Pacific. 3.0.0 GA contains 291 fi

Re: Heads up: branching branch-3.0.0 for GA

Branching is complete. Please use the 3.0.1 fix version for further commits to branch-3.0. Ping me if you want something in branch-3.0.0 since I'm rolling RC0 now. On Tue, Nov 14, 2017 at 11:08 AM, Andrew Wang wrote: > Hi folks, > > We've resolved all the blockers for 3.

Heads up: branching branch-3.0.0 for GA

Hi folks, We've resolved all the blockers for 3.0.0 and the release notes and changelog look good, so I'm going to cut the branch and get started on the RC. * branch-3.0 will advance to 3.0.1-SNAPSHOT * branch-3.0.0 will go to 3.0.0 Please keep this in mind when committing. Cheers, Andrew

Re: [DISCUSS] A final minor release off branch-2?

What are the known gaps that need bridging between 2.x and 3.x? >From an HDFS perspective, we've tested wire compat, rolling upgrade, and rollback. >From a YARN perspective, we've tested wire compat and rolling upgrade. Arun just mentioned an NM rollback issue that I'm not familiar with. Anythin

2017-10-31 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+ updates 2017-10-31 Lots of progress towards GA, we look on track for cutting RC0 this week. I ran the versions script to check the branch matches up with JIRA and fixed things up, and also checked that the changelog and r

[jira] [Reopened] (HDFS-12499) dfs.namenode.shared.edits.dir property is currently namenode specific key

[ https://issues.apache.org/jira/browse/HDFS-12499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened HDFS-12499: > dfs.namenode.shared.edits.dir property is currently namenode specific

Re: Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

FWIW we've been running branch-3.0 unit tests successfully internally, though we have separate jobs for Common, HDFS, YARN, and MR. The failures here are probably a property of running everything in the same JVM, which I've found problematic in the past due to OOMs. On Tue, Oct 24, 2017 at 4:04 PM

2017-10-20 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-10-20 Apologies for skipping the update last week. Here's how we're tracking for GA. Highlights: - Merge of HDFS router-based federation and API-based scheduler configuration with no reported problems.

[jira] [Created] (HDFS-12660) Enable async edit logging test cases in TestFailureToReadEdits

Andrew Wang created HDFS-12660: -- Summary: Enable async edit logging test cases in TestFailureToReadEdits Key: HDFS-12660 URL: https://issues.apache.org/jira/browse/HDFS-12660 Project: Hadoop HDFS

[jira] [Reopened] (HDFS-12603) Enable async edit logging by default

[ https://issues.apache.org/jira/browse/HDFS-12603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened HDFS-12603: Reopening since the branch-2 patch broke the unit test. > Enable async edit logging by defa

Re: 2017-10-06 Hadoop 3 release status update

for beta1, and we've still got plenty of other blockers/criticals to work on for GA. If you feel differently, please reach out, I can make myself very available next week for a call. Best, Andrew On Fri, Oct 6, 2017 at 3:12 PM, Allen Wittenauer wrote: > > > On Oct 6, 2017,

[jira] [Created] (HDFS-12612) DFSStripedOutputStream#close will throw if called a second time with a failed streamer

Andrew Wang created HDFS-12612: -- Summary: DFSStripedOutputStream#close will throw if called a second time with a failed streamer Key: HDFS-12612 URL: https://issues.apache.org/jira/browse/HDFS-12612

2017-10-06 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-10-06 The beta1 RC0 vote passed, and beta1 is out! Now tracking GA features. Highlights: - 3.0.0-beta1 has been released! - Router-based federation merge vote should be about to pass - API-based sch

[jira] [Created] (HDFS-12603) Enable async edit logging by default

Andrew Wang created HDFS-12603: -- Summary: Enable async edit logging by default Key: HDFS-12603 URL: https://issues.apache.org/jira/browse/HDFS-12603 Project: Hadoop HDFS Issue Type: Improvement

Re: [VOTE] Release Apache Hadoop 3.0.0-beta1 RC0

M HA/work-preservring-restart/ NM-work-preserving restart > for ATSv2 entities. > - verified all ATSv2 REST end points to retrieve the entities > - ran sample MR jobs and distributed jobs > > Thanks & Regards > Rohith Sharma K S > > On 4 October 2017 at 05:31, Andrew Wan

Re: [VOTE] Release Apache Hadoop 3.0.0-beta1 RC0

mode for KMS and HttpFs >> default HTTP >> > servlet /conf, /stacks, and /logLevel == >> > >> > One example below: >> > >> > >Connecting to >> > > https://localhost:14000/logLevel?log=org.apache.hadoop.fs. >

[jira] [Resolved] (HDFS-12442) WebHdfsFileSystem#getFileBlockLocations will always return BlockLocation#corrupt as false

[ https://issues.apache.org/jira/browse/HDFS-12442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HDFS-12442. Resolution: Invalid > WebHdfsFileSystem#getFileBlockLocations will always return > BlockLo

[jira] [Reopened] (HDFS-12442) WebHdfsFileSystem#getFileBlockLocations will always return BlockLocation#corrupt as false

[ https://issues.apache.org/jira/browse/HDFS-12442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened HDFS-12442: > WebHdfsFileSystem#getFileBlockLocations will always return > BlockLocation#corrupt as

Re: [VOTE] Release Apache Hadoop 3.0.0-beta1 RC0

2017 at 9:45 AM, Brahma Reddy Battula wrote: > > Thanks Andrew. > > +1 (non binding) > > --Built from source > --installed 3 node HA cluster > --Verified shell commands and UI > --Ran wordcount/pic jobs > > > > > On Fri, 29 Sep 2017 at 5:34 AM, Andrew Wang

2017-09-20 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-09-29 After about a month of slip, RC0 has been sent out for a VOTE. Focus now turns to GA, where we will attempt to keep the original beta1 target date (early November). Highlights: - RC0 vote was sent o

[jira] [Created] (HDFS-12567) BlockPlacementPolicyRackFaultTolerant fails with racks with very few nodes

Andrew Wang created HDFS-12567: -- Summary: BlockPlacementPolicyRackFaultTolerant fails with racks with very few nodes Key: HDFS-12567 URL: https://issues.apache.org/jira/browse/HDFS-12567 Project: Hadoop

Re: [DISCUSS] Merging API-based scheduler configuration to trunk/branch-2

Hi Jonathan, I'm okay with putting this into branch-3.0 for GA if it can be merged within the next two weeks. Even though beta1 has slipped by a month, I want to stick to the targeted GA data of Nov 1st as much as possible. Of course, let's not sacrifice quality or stability for speed; if somethin

[VOTE] Release Apache Hadoop 3.0.0-beta1 RC0

Hi all, Let me start, as always, by thanking the many, many contributors who helped with this release! I've prepared an RC0 for 3.0.0-beta1: http://home.apache.org/~wang/3.0.0-beta1-RC0/ This vote will run five days, ending on Nov 3rd at 5PM Pacific. beta1 contains 576 fixed JIRA issues compris

Re: Heads up: branching branch-3.0.0-beta1 off of branch-3.0

Branch has been cut, branch-3.0 is now open for commits for 3.0.0 GA. HEAD of branch-3.0.0-beta1 is 2223393ad1d5ffdd62da79e1546de79c6259dc12. On Thu, Sep 28, 2017 at 10:52 AM, Andrew Wang wrote: > Hi folks, > > We've driven the blocker count down to 0, and I went through and m

Heads up: branching branch-3.0.0-beta1 off of branch-3.0

Hi folks, We've driven the blocker count down to 0, and I went through and made sure the fix versions and release notes and so on are all lined up. I'm going to cut branch-3.0.0-beta1 off branch-3.0 and try and get RC0 out today. Cheers, Andrew

[jira] [Created] (HDFS-12534) Provide logical BlockLocations for EC files for better split calculation

Andrew Wang created HDFS-12534: -- Summary: Provide logical BlockLocations for EC files for better split calculation Key: HDFS-12534 URL: https://issues.apache.org/jira/browse/HDFS-12534 Project: Hadoop

2017-09-22 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-09-22 We've had some late breaking blockers related to Docker support that are delaying the release. We're on a day-by-day slip at this point. Highlights: - I did a successful test create-release earlie

Re: [DISCUSS] moving to Apache Yetus Audience Annotations

get in for 3.0, I can make a jira and put it > on the back burner for when trunk goes to 4.0? > > On Fri, Sep 22, 2017 at 12:49 PM, Andrew Wang > wrote: > >> Is this itself an incompatible change? I imagine the bytecode will be >> different. >> >> I think we&#

Re: [DISCUSS] moving to Apache Yetus Audience Annotations

Is this itself an incompatible change? I imagine the bytecode will be different. I think we're too late to do this for beta1 given that I want to cut an RC0 today. On Fri, Sep 22, 2017 at 7:03 AM, Sean Busbey wrote: > When Apache Yetus formed, it started with several key pieces of Hadoop that >

[jira] [Created] (HDFS-12497) Re-enable TestDFSStripedOutputStreamWithFailure tests

Andrew Wang created HDFS-12497: -- Summary: Re-enable TestDFSStripedOutputStreamWithFailure tests Key: HDFS-12497 URL: https://issues.apache.org/jira/browse/HDFS-12497 Project: Hadoop HDFS Issue

2017-09-19 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-09-19 Sorry for the late update. We're down to one blocker and one EC must do! Made great progress over the last week and a bit. We will likely cut RC0 this week. Highlights: - Down to just two blocker i

[jira] [Resolved] (HDFS-11156) Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API

[ https://issues.apache.org/jira/browse/HDFS-11156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HDFS-11156. Resolution: Fixed Target Version/s: 3.0.0-alpha2, 2.8.0 (was: 2.8.0, 3.0.0-alpha2) Hi

[jira] [Resolved] (HDFS-12457) Revert HDFS-11156 Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API

[ https://issues.apache.org/jira/browse/HDFS-12457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HDFS-12457. Resolution: Fixed Assignee: Andrew Wang Fix Version/s: 3.0.0-beta1 Reverted per

[jira] [Created] (HDFS-12457) Revert HDFS-11156 Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API

Andrew Wang created HDFS-12457: -- Summary: Revert HDFS-11156 Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API Key: HDFS-12457 URL: https://issues.apache.org/jira/browse/HDFS-12457 Project: Hadoop

[jira] [Reopened] (HDFS-6874) Add GETFILEBLOCKLOCATIONS operation to HttpFS

[ https://issues.apache.org/jira/browse/HDFS-6874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened HDFS-6874: --- Reverted this per discussion on HDFS-11156. > Add GETFILEBLOCKLOCATIONS operation to Htt

[jira] [Created] (HDFS-12444) Reduce runtime of TestWriteReadStripedFile

Andrew Wang created HDFS-12444: -- Summary: Reduce runtime of TestWriteReadStripedFile Key: HDFS-12444 URL: https://issues.apache.org/jira/browse/HDFS-12444 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-12438) Rename dfs.datanode.ec.reconstruction.stripedblock.threads.size to dfs.datanode.ec.reconstruction.stripedblock.threads

Andrew Wang created HDFS-12438: -- Summary: Rename dfs.datanode.ec.reconstruction.stripedblock.threads.size to dfs.datanode.ec.reconstruction.stripedblock.threads Key: HDFS-12438 URL: https://issues.apache.org/jira

Re: [VOTE] Merge yarn-native-services branch into trunk

is too close to the release, after talking with other team > members, we are fine to drop this from beta, > > And we want to target this for GA. > I’m withdrawing this vote and will start afresh vote later for GA. > Thanks all who voted this effort ! > > Thanks, > Jian >

2017-09-07 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-09-07 Slightly early update since I'll be out tomorrow. We're one week out, and focus is on blocker burndown. Highlights: - 3.1.0 release planning is underway, led by Wangda. Target release date is in

Re: [VOTE] Merge yarn-native-services branch into trunk

Hi folks, This vote closes today. I see a -1 from Allen on inclusion in beta1. I see there's active fixing going on, but given that we're one week out from RC0, I think we should drop this from beta1. Allen, Jian, others, is this reasonable? What release should we retarget this for? I don't have

Re: DISCUSS: Hadoop Compatability Guidelines

There's also the DataNode data directory layout. FS edit logs should also be included if we're including the fsimage. Historically we've bumped these in minor and major releases, though I'm not sure whether precedent supports the practice. It means you can't downgrade, and features that need metad

2017-09-01 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-09-01 We're two weeks out from beta1, focus is on blocker burndown. Highlights: - S3Guard merged! - TSv2 alpha2 merged! - branch-3.0 has been cut after discussion on dev lists. Red flags: - 10

Heads up: branch-3.0 has been cut, commit here for 3.0.0-beta1

Hi folks, I've proceeded with the plan from our earlier thread and cut branch-3.0. The branches and maven versions are now set as follows: trunk: 3.1.0-SNAPSHOT branch-3.0: 3.0.0-beta1-SNAPSHOT branch-2's are still the same. This means if you want to commit something for beta1, commit it to bra

Re: [DISCUSS] Branches and versions for Hadoop 3

ahead with the branching. > > Regards, > Varun Saxena. > > On Tue, Aug 29, 2017 at 11:35 PM, Andrew Wang > wrote: > >> Sure. Ping me when the TSv2 goes in, and I can take care of branching. >> >> We're still waiting on the native services and S3Guard merg

[jira] [Resolved] (HDFS-11515) -du throws ConcurrentModificationException

[ https://issues.apache.org/jira/browse/HDFS-11515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HDFS-11515. Resolution: Invalid Target Version/s: 2.8.1, 3.0.0-beta1 (was: 3.0.0-beta1, 2.8.1) Agree

[jira] [Resolved] (HDFS-8295) Add MODIFY and REMOVE ECSchema editlog operations

[ https://issues.apache.org/jira/browse/HDFS-8295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HDFS-8295. --- Resolution: Invalid I'm resolving this since it's likely stale, we've substantiall

[jira] [Resolved] (HDFS-9604) Move ErasureCodingPolicyManager to FSDirectory

[ https://issues.apache.org/jira/browse/HDFS-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HDFS-9604. --- Resolution: Invalid This code has been refactored a few times now, no longer valid. > M

[jira] [Resolved] (HDFS-9386) Erasure coding: updateBlockForPipeline sometimes returns non-striped block for striped file

[ https://issues.apache.org/jira/browse/HDFS-9386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HDFS-9386. --- Resolution: Cannot Reproduce Target Version/s: (was: ) Haven't seen this recently an

[jira] [Resolved] (HDFS-8140) ECSchema supports for offline EditsVisitor over an OEV XML file

[ https://issues.apache.org/jira/browse/HDFS-8140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HDFS-8140. --- Resolution: Duplicate This is a dupe of HDFS-11467. > ECSchema supports for offline EditsVisitor o

Re: [DISCUSS] Merge HDFS-10467 to (Router-based federation) trunk

> > v) mvn install (and package) is failing with following error > > [INFO] Adding ignore: * > [WARNING] Rule 1: org.apache.maven.plugins.enforcer.BanDuplicateClasses > failed with message: > Duplicate classes found: > > Found in: > org.apache.hadoop:hadoop-client-minicluster:jar:3.0.0- > b

[jira] [Created] (HDFS-12377) Refactor TestReadStripedFileWithDecoding to avoid test timeouts

Andrew Wang created HDFS-12377: -- Summary: Refactor TestReadStripedFileWithDecoding to avoid test timeouts Key: HDFS-12377 URL: https://issues.apache.org/jira/browse/HDFS-12377 Project: Hadoop HDFS

Re: Branch merges and 3.0.0-beta1 scope

Hi Vinod, On Fri, Aug 25, 2017 at 2:42 PM, Vinod Kumar Vavilapalli wrote: > > From a release management perspective, it's *extremely* reasonable to > block the inclusion of new features a month from the planned release date. > A typical software development lifecycle includes weeks of feature fr

Re: [DISCUSS] Branches and versions for Hadoop 3

the two branches >> and TSv2 get committed. >> >> Thanks, >> Wangda >> >> On Tue, Aug 29, 2017 at 11:25 AM, Andrew Wang >> wrote: >> >> > Hi Wangda, >> > >> > I'll cut two branches: branch-3.0 (3.0.0-SNAPSHOT) and >> b

Re: [DISCUSS] Branches and versions for Hadoop 3

ile, YARN-5734) are > targeted to merge to 3.0-GA but not 3.0-beta1, which branch we should > commit to, and when we can commit? Also, similar to 3.0.0-alpha1 to 4, you > will cut branch-3.0.0-beta1, correct? > > Thanks, > Wangda > > > On Tue, Aug 29, 2017 at 11:05 AM, Andr

Re: [DISCUSS] Branches and versions for Hadoop 3

your resource profile vote ends tomorrow, could we please > coordinate our merges? > > thanks > Vrushali > > > On Mon, Aug 28, 2017 at 10:45 PM, Rohith Sharma K S < > rohithsharm...@apache.org> wrote: > >> On 29 August 2017 at 06:24, Andrew Wang wrote: >> &

Re: [DISCUSS] Branches and versions for Hadoop 3

So far I've seen no -1's to the branching proposal, so I plan to execute this tomorrow unless there's further feedback. Regarding the above discussion, I think Jason and I have essentially the same opinion. I hope that keeping trunk a release branch means a higher bar for merges and code review i

2017-08-25 Hadoop 3 release status update

Hi all, I've written up a status report for the current state of Hadoop 3 on the wiki. I've also pasted it below for your convenience. Cheers, Andrew https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-08-25 Another month flew by without an update. This is a

Re: Branch merges and 3.0.0-beta1 scope

t; changes are safe. >> >> Discussed with Andrew offline, we decided to not put this to beta1 since >> beta1 is not far away. But we want to put it before GA if sufficient tests >> are done. >> >> Thanks, >> Wangda >> >> >> >> On Fri,

Re: Branch merges and 3.0.0-beta1 scope

t; On Fri, Aug 25, 2017 at 10:54 AM, Rohith Sharma K S < > rohithsharm...@apache.org> wrote: > >> On 25 August 2017 at 22:39, Andrew Wang wrote: >> >> > Hi Rohith, >> > >> > Given that we're advertising TSv2 as an alpha feature, I think we

[DISCUSS] Branches and versions for Hadoop 3

Hi folks, With 3.0.0-beta1 fast approaching, I wanted to go over the proposed branching strategy. In the early 2.x days, moving trunk immediately to 3.0.0 was a mistake. branch-2 and trunk were virtually identical, and increased backport complexity. Until we need to make incompatible changes, the

  1   2   3   4   5   6   7   8   >