Re: Release numbering for branch-2 releases

+1 for a release with all the changes that are committed. That way it carries all the important bug fixes. So, rather than debate more, I had a brief chat with Suresh and Todd. Todd > suggested calling the next release as hadoop-2.1.0-alpha to indicate the > incompatibility a little better. Thi

Re: Release numbering for branch-2 releases

On Mon, Feb 4, 2013 at 10:46 AM, Arun C Murthy wrote: > > On Feb 1, 2013, at 2:34 AM, Tom White wrote: > > Whereas Arun is proposing > > > > 2.0.0-alpha, 2.0.1-alpha, 2.0.2-alpha, 2.1.0-alpha, 2.2.0-beta, 2.3.0 > > > > and the casual observer might expect there to be a stable 2.0.1 (say) > > on

Re: Release numbering for branch-2 releases

On Mon, Feb 4, 2013 at 1:07 PM, Owen O'Malley wrote: > I think that using "-(alpha,beta)" tags on the release versions is a really > bad idea. Why? Can you please share some reasons? I actually think alpha and beta and stable/GA are much better way to set the expectation of the quality of a re

Heads up - merge branch-trunk-win to trunk

The support for Hadoop on Windows was proposed in HADOOP-8079 almost a year ago. The goal was to make Hadoop natively integrated, full-featured, and performance and scalability tuned on Windows Server or Windows Azure. We are happy to announce that

Re: VOTE: HDFS-347 merge

Todd, Some of us have been trying to help test and review the code. However you might have missed the following, which has resulted in the review not completing: 02/06/13 - After intent for merge was sent, I posted comment saying consolidate patch has extraneous changes. That was non trivial amou

Re: VOTE: HDFS-347 merge

> > > This was just an error with the "consolidate merge patch". Like I said > in the previous email, these patches are just for Jenkins QA to run > on, and I assume that any HDFS committer is able to look at the branch > itself to understand the changes in it. It's easy to accidentally end > up wi

Re: VOTE: HDFS-347 merge

On Wed, Feb 20, 2013 at 3:13 PM, Todd Lipcon wrote: > On Wed, Feb 20, 2013 at 3:08 PM, Tsz Wo Sze wrote: > > The reason to keep it around is that the HDFS-347 only support Unix but > not > > other OS. > > Given that this is an optimization, and we have a ton of optimizations > which don't yet ru

Re: VOTE: HDFS-347 merge

> > > > > I have to disagree. No where in the jira or the design it is explicitly > > stated that > > the old short circuit functionality is being removed. My assumption has > been > > that it will not be removed. > > I've tried this avenue in the past on other insecurities which were > fixed. Sorr

Re: VOTE: HDFS-347 merge

> > The patches even going back as far as last September have all removed > the old code path. I sort of assumed that, if you are taking time to > review the patches, you would have noticed this... additionally, > Colin's comments on the JIRA said as much... eg: > Todd, we have different ways of r

Re: VOTE: HDFS-347 merge

t; removing this? Unless there was a lot of follow-on work, HDFS-2246 > doesn't look like a lot of code... -C > > On Wed, Feb 20, 2013 at 3:40 PM, Todd Lipcon wrote: > > On Wed, Feb 20, 2013 at 3:31 PM, Suresh Srinivas > wrote: > >>> Given that this

Re: VOTE: HDFS-347 merge

On Wed, Feb 20, 2013 at 5:12 PM, Aaron T. Myers wrote: > On Wed, Feb 20, 2013 at 4:29 PM, Chris Douglas > wrote: > > > Given that HDFS-347 is a strictly better approach, once committed, > > there will be ample motivation to add support for other OSes and > > remove HDFS-2246 entirely. Nobody is

Re: VOTE: HDFS-347 merge

ATM's suggestion of removing HDFS-2246 in trunk, but not branch-2, is > a rational compromise: it allows some period for others to adapt, but > not an indefinite one. It's not clear what you're proposing, if > anything. > I am not sure why a release that supports both these is such a bad idea. A

Re: VOTE: HDFS-347 merge

> > > There's no reason to maintain multiple implementations of the same > feature, that's why per the 2246 jira it was proposed as a "good short > term solution till HDFS-347 is completed". Why is ATM's compromise > unacceptable? > We have already discussed this. Here is the recap: HDFS-347 do

Re: VOTE: HDFS-347 merge

> > > I assume you mean in trunk? Given that ATM's proposal is to only > remove HDFS-2246 from branch-2 once (a) we're confident in HDFS-347 > and (b) adds Windows support, and we won't be releasing from trunk any > time soon - from a user perspective - HDFS-2246 will only be replaced > with HDFS

Re: VOTE: HDFS-347 merge

> > Suresh, if you're willing to "support and maintain" HDFS-2246, do you > have cycles to propose a patch to the HDFS-347 branch reintegrating > HDFS-2246 with the simplifications you outlined? In your review, did > you find anything else you'd like to address prior to the merge, or is > this the

Re: [Vote] Merge branch-trunk-win to trunk

e able to run nightly tests to > > validate everything still compiles/runs. This is not a blocker for me > > because we often rely on individuals and groups to test Hadoop, but I do > > think we need to have this discussion before we put it in. > > > > --Bobby > >

Re: [Vote] Merge branch-trunk-win to trunk

> > I'm concerned about the above. Personally, I don't have access to any > Windows boxes with development tools, and I know nothing about developing > on Windows. The only Windows I run is an 8GB VM with 1 GB RAM allocated, > for powerpoint :) > If I submit a patch and it gets -1 "tests failed"

Re: VOTE: HDFS-347 merge

Thanks Colin. Will check it out as soon as I can. On Tue, Mar 5, 2013 at 12:24 PM, Colin McCabe wrote: > On Tue, Feb 26, 2013 at 5:09 PM, Suresh Srinivas > wrote: > >> > >> Suresh, if you're willing to "support and maintain" HDFS-2246, do you > >>

Re: [Vote] Merge branch-trunk-win to trunk

Thank you all for voting and participating in the discussions. With 11 +1s from committers (more than the required 3 +1s from active committers per the Hadoop bylaws), 1 +0, 8 +1s from other contributors, and no -1s the merge vote passes. I have committed the consolidated patch from branch-trunk-

Re: Heartbeat interval and timeout: why 3 secs and 10 min?

You are right, in heartbeat response namenode sends commands to the datanode. Commands sent this way include deletion of blocks, replication, block recovery secret key updates etc. Increasing the heartbeat interval results in namenode not being able to quickly act on the events in the cluster and s

Re: [Vote] Merge branch-trunk-win to trunk

Adding other mailing lists I missed earlier. Cos, There is progress being made on that ticket. Also it has nothing to do with that. Please follow the discussion here and why this happened due to an invalid commit that was reverted - https://issues.apache.org/jira/browse/HDFS-4615?focusedComment

Re: Exception with QJM HDFS HA

This does seem like inode id change related. I will follow up on HDFS-4654. Sent from a mobile device On Mar 31, 2013, at 10:12 PM, Harsh J wrote: > A JIRA was posted by Azuryy for this at > https://issues.apache.org/jira/browse/HDFS-4654. > > On Mon, Apr 1, 2013 at 10:40 AM, Todd Lipcon wro

Re: VOTE: HDFS-347 merge

Colin, For the record, the last email in the previous thread in ended with the following comment from Nicholas: > It is great to hear that you agree to keep HDFS-2246. Please as well address my comments posted on HDFS-347 and let me know once you have posted a new patch on HDFS-347. I did not se

Re: VOTE: HDFS-347 merge

Mon, Apr 1, 2013 at 6:58 PM, Colin McCabe > wrote: > > > On Mon, Apr 1, 2013 at 5:04 PM, Suresh Srinivas >wrote: > > > >> Colin, > >> > >> For the record, the last email in the previous thread in ended with the > >> following comment from Nich

Re: VOTE: HDFS-347 merge

> We usually conclude the last VOTE before starting a new one. Otherwise, > people may be confused between the VOTEs. (In case you don't know our > convention. Please check with someone before starting a VOTE. Thanks.) > > > -1 > * The previous VOTE started by Colin has not been concluded. > N

Heads up - Snapshots feature merge into trunk

Support for snapshots feature is being worked on in the jira https://issues.apache.org/jira/browse/HDFS-2802. This is an important and a large feature in HDFS. Please see a brief presentation that describes the feature at a highlevel from the Snapshot discussion meetup we had a while back - https:/

Re: Heads up - Snapshots feature merge into trunk

I think we should take this on the jira than the merge heads up thread. Nicholas, please suggest a jira where we can continue the Some comments inline: On Wed, Apr 24, 2013 at 1:25 PM, Todd Lipcon wrote: > On Fri, Apr 19, 2013 at 3:36 AM, Aaron T. Myers wrote: > > > On Fri, Apr 19, 2013 at 6:

Re: Heads up - 2.0.5-beta

Thanks for starting this discussion. I volunteer to do a final review of protocol changes, so we can avoid incompatible changes to API and wire protocol post 2.0.5 in Common and HDFS. We have been working really hard on the following features. I would like to get into 2.x and see it reach HDFS use

Re: Heads up - 2.0.5-beta

Eli, I will post a more detailed reply soon. But one small correction: I'm also not sure there's currently consensus on what an incompatible > change is. For example, I think HADOOP-9151 is incompatible because it > broke client/server wire compatibility with previous releases and any > change th

[VOTE] Merge HDFS-2802 snapshot feature to trunk

This is a follow up to my earlier heads up about merging Snapshot feature to trunk - http://markmail.org/message/ixkyku2cebkewnzy. I am happy to announce that we have completed the development of the feature. It is ready to be merged into trunk. Development of snapshot feature is tracked in the ji

Re: DistributedFileSystem.listStatus() - Why does it do partial listings then assemble?

Additional reason, HDFS does not have limit on number of files in a directory. Some clusters had millions of files in a single directory. Listing such a directory resulted in very large responses, requiring large contiguous memory allocation in JVM (for the array) and unpredictable GC failures. O

Fwd: Heads up - 2.0.5-beta

Konstantin, I am arguing against invasive and destructive features proposed for the > release. > Your choice of words is deplorable, to say the least. Can you explain what do you mean by *destructive*? Please substantiate your claim on technical grounds. So far you have been quiet while we have

Re: [VOTE] Merge HDFS-2802 snapshot feature to trunk

IRA, but I don't think > that JIRA ever got filed. Mind doing that? > > > -- > Aaron T. Myers > Software Engineer, Cloudera > > > On Wed, May 1, 2013 at 11:54 AM, Suresh Srinivas >wrote: > > > This is a follow up to my earlier heads up about merging Snaps

Re: [VOTE] Merge HDFS-2802 snapshot feature to trunk

, Suresh Srinivas wrote: > Aaron, I have created HDFS-4808 to discuss the API issues you brought up. > I agree that this needs to be resolved before merging the code to branch-2. > Thanks for voting. > > > On Tue, May 7, 2013 at 5:29 PM, Aaron T. Myers wrote: > >> I'm +

Re: [VOTE] Merge HDFS-2802 snapshot feature to trunk

HDFS-2802 branch has been merged into trunk. Thanks Nicholas for maintaining the merge patch and merging the branch into trunk. Regards, Suresh On Wed, May 8, 2013 at 1:16 PM, Suresh Srinivas wrote: > With 3 +1 binding votes, the vote to merge Snapshot feature from HDFS-2802 > branch to

Re: Heads up: branch-2.1-beta

Did you change the java version? Can you please provide the JVM configuration from hadoop-env.sh? On Wed, Jun 19, 2013 at 10:02 PM, Roman Shaposhnik wrote: > On Wed, Jun 19, 2013 at 5:21 PM, Roman Shaposhnik wrote: > > On Wed, Jun 19, 2013 at 4:29 PM, Roman Shaposhnik > wrote: > >> On Tue, J

Re: Heads up: branch-2.1-beta

I still need to look at hadoop-env.sh. I suspect this is due to not passing Xms (starting java heap size) set to same value as Xmx (max java heap size). The recent inode id changes create hash apps based on max heap size and I have seen this problem without Xms. Sent from a mobile device On

Re: Heads up: branch-2.1-beta

Can you please run the tests with -Xms2G added? Sent from a mobile device On Jun 20, 2013, at 6:02 PM, Roman Shaposhnik wrote: > On Thu, Jun 20, 2013 at 5:59 PM, Suresh Srinivas > wrote: >> I still need look at hadoop-env.sh > > Bigtopt deployments don't utilize ha

Fwd: DesignLounge @ HadoopSummit

FYI HDFS committers and developers... This will be participant-driven unconference style session. So no formal presentations are planned. Some of the things that we could discuss are: - New features recently added to HDFS such as HA, Snapshots, NFS etc. - Other features that we as a community coul

Re: data loss after cluster wide power loss

ld still not be synced and would still be stored by the datanode >> in "blocksBeingWritten" rather than in "current"? If that is case, would >> it be better for the NameNode not to reject replicas that are in >> blocksBeingWritten, especially if it doesn&

Re: data loss after cluster wide power loss

On Wed, Jul 3, 2013 at 8:12 AM, Colin McCabe wrote: > On Mon, Jul 1, 2013 at 8:48 PM, Suresh Srinivas > wrote: > > Dave, > > > > Thanks for the detailed email. Sorry I did not read all the details you > had > > sent earlier completely (on my phone). As y

Re: Willing to work on Issue - HDFS-4901 - Where to start?

Vivek, For information on how to contribute see - http://wiki.apache.org/hadoop/HowToContribute Please register an iCLA with ASF - see http://www.apache.org/licenses/icla.txt Once this is done, I will add you as a contributor and assign the jira to you. Regards, Suresh On Mon, Jul 8, 2013 at 3

Re: Willing to work on Issue - HDFS-4901 - Where to start?

oop patches before, without > registering the iCLA with ASF. > > > Thanks, > > Niranjan Singh > > > > On Mon, Jul 8, 2013 at 8:05 PM, Suresh Srinivas >wrote: > > > Vivek, > > > > For information on how to contribute see - > > http://wiki.ap

Re: Willing to work on Issue - HDFS-4901 - Where to start?

you please assign the JIRA to me? > > Thank you. > > Regards, > Vivek > > > On Mon, Jul 8, 2013 at 8:05 PM, Suresh Srinivas >wrote: > > > Vivek, > > > > For information on how to contribute see - > > http://wiki.apache.org/hadoop/HowToContribu

Re: Suggestions required for HDFS-4926 (namenode webserver's page has a tooltip that is inconsistent with the datanode HTML link)

Jira has these comments right. I do not see necessity for posting that again to hdfs-dev mailing list. Please continue the discussion in Jira. Sent from phone On Jul 15, 2013, at 7:04 AM, Vivek Ganesan wrote: > Hi, > > We have 3 options (actually view points) laid out for resolution of HDFS-

Re: I'm interested in working with HDFS-4680. Can somebody be a mentor?

Please look at some of the work happening in HADOOP-9688, which is adding a unique UUID (16 bytes) for each RPC request. This is common to all Hadoop RPC, will be available in HDFS, YARN and MAPREDUCE. Please see the jira for more details. Reach out to me if you have any questions. On Wed, Jul 17

Re: Feature request to provide DFSInputStream subclassing mechanism

This is being targeted for release 2.3. 2.1.x release stream is for stabilizing. When it reaches stability, 2.2 GA will be released. The current features in development will make it to 2.3, including HDFS-2832. On Thu, Aug 8, 2013 at 2:04 PM, Matevz Tadel wrote: > Thanks Colin, I subscribed to

Re: Which SVN location should i checkout for creating patch to branch-1 (HDFS-2933) ?

Vivek, The current branch where features/bug fixes go to for release 1.x is https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1 1.x releases are created from the above branch. A new branch is created and the release is made from that branch. For example, you can see branch correspon

Re: symlink support in Hadoop 2 GA

I agree that this is an important change. However, 2.2.0 GA is getting ready to rollout in weeks. I am concerned that these changes will add not only incompatible changes late in the game, but also possibly instability. Java API incompatibility is some thing we have avoided for the most part and I

Re: [VOTE] Release Apache Hadoop 2.1.1-beta

+1 (binding) Verified the signatures and hashes for both src and binary tars. Built from the source, the binary distribution and the documentation. Started a single node cluster and tested the following: # Started HDFS cluster, verified the hdfs CLI commands such ls, copying data back and forth,

Re: 2.1.2 (Was: Re: [VOTE] Release Apache Hadoop 2.1.1-beta)

(This time copying all the lists) I am +1 for naming the new branch 2.2.0. On Tue, Oct 1, 2013 at 4:15 PM, Arun C Murthy wrote: > Guys, > > I took a look at the content in 2.1.2-beta so far, other than the > critical fixes such as HADOOP-9984 (symlinks) and few others in YARN/MR, > there is f

Re: SDP support for Hadoop RPC

Milind, please stop this. The topic here is not what Steve's employers wants to sell or recommend. Please stick to the technical issue. This is the second time this week where a thread unnecessarily goes beyond technical issues. If you have an axe to grind, please keep it off this forum. It is gett

Re: [VOTE] Release Apache Hadoop 2.2.0

+1 (binding) On Mon, Oct 7, 2013 at 12:00 AM, Arun C Murthy wrote: > Folks, > > I've created a release candidate (rc0) for hadoop-2.2.0 that I would like > to get released - this release fixes a small number of bugs and some > protocol/api issues which should ensure they are now stable and will

Re: [VOTE] Merge HDFS-4949 to trunk

I posted a comment in the other thread about feature branch merges. My preference is to make sure the requirements we have for regular patches be applied to feature branch patch as well (3 +1s is the only exception). Also adding details about what functionality is missing (I posted a comment on HD

Re: Decommission DataNode

No this is not a bug. This is the new behavior. Please see for details - https://issues.apache.org/jira/browse/HDFS-1547 On Tue, Oct 29, 2013 at 1:48 AM, lei liu wrote: > Should the datanode be shutdown when it is Decommissioned? > > I think if this is bug, I can fix it. > > > 2013/10/29 lei l

Re: Question regarding access to different hadoop 2.0 cluster

Lohit, The option you have enumerated at the end is the current way to set up multi cluster environment. That is, all the client side configurations will include the following: - Logical service names (either for federation or HA) - The corresponding physical namenode addresses information For si

Re: Question regarding access to different hadoop 2.0 cluster

f you have many > different clusters that talk to each other. > > --Bobby > > On 11/4/13 4:15 PM, "lohit" wrote: > > >Thanks Suresh! > > > > > >2013/11/4 Suresh Srinivas > > > >> Lohit, > >> > >> The option you have

Re: Deprecate hftp / hsftp

Thanks Haohui for all your hard work in this area. I am +1 on this proposal. On Tue, Nov 26, 2013 at 12:50 PM, Haohui Mai wrote: > Hi, > > Recently I've been focusing on fixing hftp / hsftp / webhdfs / swebhdfs in > various set ups. Now we have reached the state that all the above file > system

Re: [VOTE] Merge HDFS-2832 Heterogeneous Storage Phase 1 to trunk

n.pdffor > the details. Once the feature is > merged into trunk, we will continue to test and fix any bugs that may be > found on trunk as well as add further tests as outlined in the test plan. > > The bulk of the design and implementation was done by Suresh Srinivas, > Sanjay

Deprecate BackupNode

It is almost an year a jira proposed deprecating backup node - https://issues.apache.org/jira/browse/HDFS-4114. Maintaining it adds unnecessary work. As an example, when I added support for retry cache there were bunch of code paths related to backup node that added unnecessary work. I do not know

Re: Deprecate BackupNode

Konstantin, On Sun, Dec 8, 2013 at 1:06 PM, Konstantin Shvachko wrote: > I explained my reasoning in the jira > > https://issues.apache.org/jira/browse/HDFS-4114?focusedCommentId=13841326&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13841326 > Please see reply -

Re: Is it me or is the new bootstrap HDFS UI HWX green? (See tip of hadoop-2.3 branch)

Stack, This seems to me like coloring the good work someone has done with an unneeded controversy. Color is a matter of choice. The person who did the fine work had all the rights to choose what he sees as fit. I also think that while you might think "green" as vendor color, most people probably w

Re: Is it me or is the new bootstrap HDFS UI HWX green? (See tip of hadoop-2.3 branch)

Stack, 4. Bigtop/Oozie, other Apache projects altogether, have Cloudera references > in their UI (?) so the take away is HWX can do it too, or, because I raise > an issue here -- it is only legit if I do it too in all projects under the > Apache rainbow? > My point was different; these type of is

Re: [VOTE] Merge HDFS-5698 Use protobuf to serialize / deserialize FSImage to trunk

clude both > unit tests and end-to-end tests. They have already covered this > feature. > > Once the feature is merged into trunk, we will continue to test and > fix any bugs that may be found on trunk. > > The bulk of the design and implementation was done by Jing Zhao, > Sur

Re: [VOTE] Merging branch HDFS-7240 to trunk

Anu, Jing, Nicholas, Sanjay, Jitendra and many others, thank you for staying focused on this effort. It has been almost 3.5 years since HDFS-7240 was created and all of the work has happened over the years in the open in the feature branch. Storage layer improvements from HDFS-7240 is important no

Re: [VOTE] Adopt HDSL as a new Hadoop subproject

+1 (Binding) Regards, Suresh On Tue, Mar 20, 2018 at 11:20 AM, Owen O'Malley wrote: > All, > > Following our discussions on the previous thread (Merging branch HDFS-7240 > to trunk), I'd like to propose the following: > > * HDSL become a subproject of Hadoop. > * HDSL will release separately fr

Re: Review Request: HDFS-2301 Start/stop appropriate namenode internal services during transition to active and standby

> On 2011-10-06 23:37:24, Todd Lipcon wrote: > > branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/HAUtil.java, > > line 28 > > > > > > perhaps should be abstract since it won't ever

Merge changes from trunk to 0.23 release

Arun, Following issues need to be ported from trunk to 0.23. HADOOP-2298 TestDFSOverAvroRpc fails Justification: This makes a protocol method name change. Required for compatibility of 0.23 with future releases. Risk: low HDFS-2355 Enable using same configuration file in federation Justificati

Re: how to compile proto files?

Under hadoop-hdfs/src I run: Protoc -I=proto proto/.proto --java_out main/src On Thursday, December 8, 2011, Alejandro Abdelnur wrote: > I'm trying to change the build to compile the proto files (instead checking > in the generated java files) > > However, I'm not able to run protoc successfull

Re: how to compile proto files?

Abdelnur wrote: > Suresh, > > One proto file at the time works, but doing *.proto fails complaining about > duplicate definitions. > > Thxs. > > Alejandro > > On Thu, Dec 8, 2011 at 11:46 PM, Suresh Srinivas >wrote: > > > Under hadoop-hdfs/src I run: > &

Re: test failures on trunk

I will fix this bug today. On Thursday, December 15, 2011, Uma Maheswara Rao G wrote: > Yes Eli, Agree with you. > > I think we must take a look immediately now. > > I just debugged the failure related to this class cast exception. > > Following is the initial analysis: > > Previous behaviour

Re: Review request: trunk->HDFS-1623 merge

Todd, can you please hold off for the merge till Tuesday, until some of the other folks working on HA could catch up with some of the recent changes. On Fri, Feb 3, 2012 at 2:10 PM, Todd Lipcon wrote: > I've got a merge pending of trunk into HDFS-1623 -- it was a bit > complicated so wanted to a

Re: Review request: trunk->HDFS-1623 merge

I looked at the merge. It looks good. +1. On Wed, Feb 8, 2012 at 9:08 PM, Todd Lipcon wrote: > The branch developed some new conflicts due to recent changes in trunk > affecting the RPC between the DN and the NN (the "StorageReport" > stuff). I've done a new merge to address these conflicts here

Re: Merging the HA branch to trunk - Wednesday, February 29th

I am not sure any of these issues are serious show stoppers for merging into trunk. Why not merge into trunk and fix some of these issues? The reason is, merging is non trivial with two branches changing independently. Given that Jitendra has posted a merge patch, why not do it earlier? Do we need

Re: Merging the HA branch to trunk - Wednesday, February 29th

I am +1 on merging this to trunk. On Feb 29, 2012, at 4:03 PM, Todd Lipcon wrote: > +1 as well. > > My latest tests after applying the performance fixes indicate that > there is no statistically significant performance regression between > trunk and HA, even in tests designed to stress test the

Re: [VOTE] Merge the HA branch to trunk tomorrow - March 2nd

As I indicated in the earlier thread, I am +1 on merging HDFS-1623 branch to trunk. On Mar 1, 2012, at 9:05 PM, "Aaron T. Myers" wrote: > Hello HDFS devs, > > As mentioned in a thread started last week, I'd like to merge the HA branch > to trunk. I had originally intended to do this yesterday,

Merge Namenode HA feature to 0.23

Namenode HA (HDFS-1623) feature is now in trunk (yay!). It adds support for namenode high availability with active and standby namenodes, with manual failover. I propose merging this feature to 0.23 branch. Recently Nicholas merged protocol buffers/wire compatibility changes to 0.23. Merging Nameno

Re: Merge Namenode HA feature to 0.23

hanks for volunteering! > > You'll probably want to merge HDFS-1580, HDFS-1765, HDFS-2158, > HDFS-2188, HDFS-2334, HDFS-2476, HDFS-2477, and HDFS-2495 to branch-23 > first as these conflict and the patch will contain a bunch of non-HA > stuff. > > Thanks, > Eli >

Re: Merge Namenode HA feature to 0.23

I have merged the change required for merging Namenode HA. I have also attached a release 23 patch in the jira HDFS-1623. Please take a look the attached patch and let me know if that looks good. Regards, Suresh

Re: Proposal: abandon protocol translator layer for cluster-internal RPCs

I am sure Jitendra understands what Todd meant, given he was quite involved in the work. As Jitendra said, I would like to keep the wire type from the implementation type. Even for internal protocols. Rolling upgrades is important. I understand where Todd is coming from. We did this work for 10 pr

Re: Hadoop WebUI

Clement, To get the details related to how to contribute - see http://wiki.apache.org/hadoop/HowToContribute. UI is simple because it serves the purpose. More sophisticated UI for management and monitoring is being done in Ambari, see - http://incubator.apache.org/ambari/. The core hadoop UIs co

Re: Heads up: merge for QJM branch soon

I need a week or so to go over the design and review the code changes. I will post my comments to the jira directly. Meanwhile any updates made to the design document would help. Regards, Suresh On Wed, Sep 19, 2012 at 12:53 PM, Todd Lipcon wrote: > Hi all, > > Work has been progressing steadil

Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk

I am in favor of keeping QJM in HDFS. QJM is very specific to HDFS and is tightly coupled with HDFS code, essentially extending the current editlog functionality that writes to local disk to writing to a separate set of daemons. Clearly there is a need for this in HDFS. Konstantin, I see your poin

Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk

Todd, As I indicated in my comments on the jira, I think some of the design discussions and further simplification of design should happen before the merge. See - https://issues.apache.org/jira/browse/HDFS-3077?focusedCommentId=13470680&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-

Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk

On Mon, Oct 8, 2012 at 6:20 PM, Todd Lipcon wrote: > On Mon, Oct 8, 2012 at 6:01 PM, Suresh Srinivas >wrote: > > > Todd, > > > > As I indicated in my comments on the jira, I think some of the design > > discussions and further simplification of design should

Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk

On Mon, Oct 8, 2012 at 8:03 PM, Andrew Purtell wrote: > Our position on the QJM is we've already "taken delivery" from the feature > branch and will maintain a private HDFS fork of branch-2 if necessary, i.e. > we don't have a significant stake in this discussion except at a meta level > as poten

Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk

Given the completion of discussions and the resolution towards next steps I am +1 for the merge. On Wed, Oct 10, 2012 at 7:17 PM, sanjay Radia wrote: > > On Oct 9, 2012, at 1:44 PM, sanjay Radia wrote: > > > I have posted a comment on 3077 listing two areas that i would like to > fix in the QJM.

Re: profiling hdfs write path

Thank you Todd! I have been seeing similar attitude in many jiras, that I have tried hard to ignore and was wondering how to respond to this email. I could not have said it better. On Tue, Dec 4, 2012 at 9:27 AM, Todd Lipcon wrote: > f you're just going to insult us, please stay away. We don't

Re: How patch for renamed file handled while committing?

It is handled using svn rename command. There are times when committers do not notice this and end up committing the patch as is. This ends up deleting the contents of renamed files, leaving behind empty files. We had to cleanup such files sometime ago. On Tue, Feb 11, 2014 at 11:18 PM, Vinayakum

Re: [VOTE] Merge HDFS-4685 HDFS ACLs to trunk

this point. Code contributors are Vinayakumar B, > Sachin Jose, Renil Joseph, Haohui Mai, Chris Nauroth and Jing Zhao. Yesha > Vora contributed the test plan. The design document incorporates feedback > from many community members: Dilli Arumugam, Brandon Li, Haohui Mai, Kevin > Minder

Re: Thinking ahead to 2.4

Arun, Some of the previously 2.4 targeted features were made available in 2.3: - Heterogeneous storage support - Datanode cache The following are being targeted for 2.4: - Use protobuf for fsimge (already in) - ACLs (in trunk. In a week or so, this will be merged to branch-2.4) - Rolling upgrades

Re: [VOTE] Merge HDFS-5535 Rolling Upgrade Improvement

committed soon. All other issues are further > improvements which can be done after merge. > > The other remaining works are: > - Revise the design doc > - Post a test plan (Haohui is working on it.) > - Execute the manual tests (Haohui and Fengdong will work on it.) > > The w

Re: [VOTE] Release Apache Hadoop 2.4.0

+1 (binding) Verified the signatures and hashes for both src and binary tars. Built from the source, the binary distribution and the documentation. Started a single node cluster and tested the following: # Started HDFS cluster, verified the hdfs CLI commands such ls, copying data back and forth, v

Re: [Vote] Merge The HDFS XAttrs Feature Branch (HDFS-2006) to Trunk

I have not looked at the development closely. With rolling upgrades feature support in, are there any incompatible changes with this feature? Sent from phone > On May 16, 2014, at 10:30 AM, Chris Nauroth wrote: > > +1 for the merge. > > I've participated in ongoing design discussions and code

Re: [Vote] Merge The HDFS XAttrs Feature Branch (HDFS-2006) to Trunk

rstanding, I think the above means we should still be able to > rolling upgrade between 2.4 and whatever version eventually includes xattr > support. > > RPC-wise, HDFS-2006 only adds new RPCs, so I don't think there are any > concerns there. > > Thanks, > Andrew >

Re: [Vote] Merge The HDFS XAttrs Feature Branch (HDFS-2006) to Trunk

> I think you're alluding to the recent WebHDFS+ACLs compat JIRA here, No. My comments was unrelated. > but I > don't think xattrs will hit the same issues. The xattr functionality is > wholly contained within new RPC calls, and AFAIK didn't modify any existing > client calls. You have to expli

Re: hadoop-2.5 - June end?

We should also include extended attributes feature for HDFS from HDFS-2006 for release 2.5. On Mon, Jun 9, 2014 at 9:39 AM, Arun C Murthy wrote: > Folks, > > As you can see from the Roadmap wiki, it looks like several items are > still a bit away from being ready. > > I think rather than wait

Re: [VOTE] Merge fs-encryption branch to trunk

+1 (binding) This is a very important feature. I have gone through the design in detail. I have also discussed the details of design and needed as follow up with the team (I will post the details in the jira). The follow up items should not hold up merging this feature to trunk and can be done in

Re: Updates on migration to git

Karthik, I would like to see detailed information on how this migration will be done, how it will affect the existing project and commit process. This should be done in a document that can be reviewed instead of in an email thread on an ad-hoc basis. Was there any voting on this in PMC and should

Re: Updates on migration to git

, Suresh Srinivas wrote: > Karthik, > > I would like to see detailed information on how this migration will be > done, how it will affect the existing project and commit process. This > should be done in a document that can be reviewed instead of in an email > thread on an ad-ho

  1   2   3   4   5   6   >