hadoop 23.0 contents

Hey Arun, Was looking at the 23.0 release bits and noticed that the contents of branch-23.0 and jira don't jive with the rc2 tarball that was released and put on the site. Jira and branch-23.0 indicate the 7 changes (below) are part of 23.0, but the tarball doesn't contain them, it stops at hash d

Re: hadoop 23.0 contents

release... > > I'm traveling right now, let get back in a bit. > > thanks, > Arun > > Sent from my iPhone > > On Jan 10, 2012, at 9:55 PM, Eli Collins wrote: > >> Hey Arun, >> >> Was looking at the 23.0 release bits and noticed that the con

Re: Some s3n improvements to the 1.0 branch? Also, is there a 1.1.0 planned?

Hey Jagane, You can target your patches against branch-1.0, and then mark them with fixVersion 1.0.2 if you'd like the RM to pull them in for the next dot release. Thanks, Eli On Mon, Mar 5, 2012 at 9:47 AM, Jagane Sundar wrote: > Hello All, > > I would like to make some enhancements to the s3

Release with PB and HA

Hey gang, Now that branch-23 has PB and (almost) HA checked in, it would be good to get these out into the wild so more people can start kicking the tires. How about we cut a 23.3 release soon from branch-23? Or perhaps re-branch 23.2 if that's easier. Thanks, Eli

Re: Release with PB and HA

pend a lot of effort trying to make them > compatible. > Do all these changes have jiras with target version 0.23.3? Thanks, Eli > This work should be wrapped up in a few weeks. > > Regards, > Suresh > > On Fri, Mar 9, 2012 at 10:11 AM, Eli Collins wrote: > >&g

Re: [PLAN] Freeze date, Hadoop 1.0.2 RC

On Wed, Mar 14, 2012 at 4:41 PM, Suresh Srinivas wrote: > Matt, > > HADOOP-4855/HDFS-3075 adds capability to restore failed storage directories > on the namenode. This helps > in not having to restart the Namenode for flaky NFS storage directories and > improves operability. It has already been co

Re: Release with PB and HA

ot of effort trying to make them > compatible. > > This work should be wrapped up in a few weeks. > Hey Suresh, How's this working going? What jiras track it? Thanks, Eli > Regards, > Suresh > > On Fri, Mar 9, 2012 at 10:11 AM, Eli Collins wrote: > >> Hey gan

Re: GSoC benefits

Hey Doron, These links are a good starting point: http://wiki.apache.org/hadoop/HowToContribute http://wiki.apache.org/hadoop/ProjectSuggestions http://wiki.apache.org/hadoop/HadoopResearchProjects I'd also checkout unresolved new features in jira. This query is a starting point. You can edit th

Re: GSoC benefits

> even required? You could get a big win even on a multi-core system (the tests themselves don't effectively use multiple cores, so running multiple tests in parallel would give you a speedup even on a laptop (if you have sufficient memory). Thanks, Eli > > Good Day, > qballer &

Heads up wrt last upgrade-able version on 0.23

Hey gang, I filed HDFS-3137 so that the 0.23 releases would only support upgrades from Hadoop version 0.18 or later. People on versions before 0.18 (don't think there are many, if any, people running these) would have to upgrade to a version like 1.0 and then upgrade again from there if they want

Re: Proto files

Good idea, no reason we shouldn't, the build probably wasn't updated to include the who we added then. File a jira? On Saturday, March 24, 2012, Ralph Castain wrote: > Hi folks > > I notice that the .proto files are not present in the built tarball. This presents a problem to those of us working

Re: Regarding HDFS-234 (Integration with BookKeeper logging system)

Hi Uma, I don't think anyone is planning on merging it. If you want to try it out and merge it, go for it. Thanks, Eli On Thu, Mar 29, 2012 at 2:16 AM, Uma Maheswara Rao G wrote: > Hi Todd, Aaron, Suresh, > >  Are we going to merge this JIRA into 23 version? > > > Regards, > Uma > > > _

Re: [RESULT] - [VOTE] Rename hadoop branches post hadoop-1.x

On Thu, Mar 29, 2012 at 8:00 AM, Owen O'Malley wrote: > On Wed, Mar 28, 2012 at 5:11 PM, Doug Cutting wrote: > >> On 03/28/2012 12:39 PM, Owen O'Malley wrote: >> > [ ... ] So the RM of the 2 branch needs to make the call of what >> > should be 2.1 vs 3.0. >> >> I thought these were community deci

Re: Build failed in Jenkins: Hadoop-Common-trunk #364

2 at 2:21 AM, Apache Jenkins Server wrote: > See <https://builds.apache.org/job/Hadoop-Common-trunk/364/changes> > > Changes: > > [eli] HDFS-3171. The DatanodeID "name" field is overloaded. Contributed by > Eli Collins > > [eli] HADOOP-8211. Update common

Re: Requirements for patch review

On Wed, Apr 4, 2012 at 2:12 PM, Todd Lipcon wrote: > Hi folks, > > Some discussion between Nicholas, Aaron, and me started in the > comments of HDFS-3168 which I think is better exposed on the mailing > list instead of trailing an already-committed JIRA. > > The question at hand is what the policy

Re: Hadoop-1.0.3 and 1.1 release / code freeze proposals

On Fri, Apr 20, 2012 at 6:02 PM, Matt Foley wrote: > Hi all, > a couple major bugs have shown up in Hadoop-1.0.2.  I propose to make a > 1.0.3 release containing fixes for: >  - MAPREDUCE-4154 > streaming > MR job succeeds even if the streamin

Re: Java 7 and Hadoop

Hey Scott, Nice. Please update this page with your experience when you get a chance: http://wiki.apache.org/hadoop/HadoopJavaVersions Thanks, Eli On Thu, Apr 26, 2012 at 2:03 PM, Scott Carey wrote: > Java 7 update 4 has been released.  It is even available for MacOS X from > Oracle: > http:/

Re: Java 7 and Hadoop

On Thu, Apr 26, 2012 at 2:03 PM, Scott Carey wrote: > Java 7 update 4 has been released.  It is even available for MacOS X from > Oracle: > http://www.oracle.com/technetwork/java/javase/downloads/jdk-7u4-downloads-1591156.html > > Java 6 will reach end of life in about 6 months.   After that poin

Re: Test failures in TestFileSystemCanonicalization

Do you get the same failure with JDK6? On Wednesday, May 9, 2012, Trevor Robinson wrote: > Does anyone know anything about these test failures in 0.23.1 > (+cdh4b2) and trunk? > > Failed tests: > testShortAuthority(org.apache.hadoop.fs.TestFileSystemCanonicalization): > expected: but was: > tes

Re: Test failures in TestFileSystemCanonicalization

f course, look > into the cause/fix as time allows. > > On Wed, May 9, 2012 at 9:42 PM, Eli Collins wrote: >> Do you get the same failure with JDK6? >> >> On Wednesday, May 9, 2012, Trevor Robinson wrote: >> >>> Does anyone know anything about these test fa

Common/Hdfs trunk Jenkins jobs should be blue now

Hey gang, Just a heads up.. all the "Result = FAILURE" messages that jenkins comments on jira when patches get merged should be fixed now. These were due to the Hadoop-Common-trunk and Hadoop-Hdfs-trunk jobs failing due to being misconfigured. I configured them to just build, so only the precomm

Re: [VOTE - RELEASE PLAN] create branch-1.1 and make release candidate 1.1.0

+1 Thanks Matt On Thu, May 17, 2012 at 6:52 PM, Matt Foley wrote: > Hi all, > I would like to branch "branch-1.1" from the current HEAD of branch-1, > and proceed to make a release candidate for hadoop-1.1.0 from it. > I'm also hereby offering to be Release Manager for branch-1.1, as I have > b

Re: cmake

+1 having a build tool that supports multiple platforms is worth the dependency. I've also had good experiences with cmake. On Mon, May 21, 2012 at 6:00 PM, Colin McCabe wrote: > Hi all, > > We'd like to use CMake instead of autotools to build native (C/C++) code in > Hadoop.  There are a lot

Re: cmake

On Wed, May 23, 2012 at 8:58 AM, Owen O'Malley wrote: > After a quick look at cmake, it seems reasonable. Please use feature tests > rather than OS tests. (By that I mean that if Solaris needs "foobar.h" and > RHEL needs "sys/foobar.h" to get the definition for foobar, the code should > ifdef whet

Hadoop view on Jenkins

Hey gang, fyi, I created a view in jenkins so you can see all the enabled Hadoop builds: https://builds.apache.org/view/Hadoop Thanks, Eli

Re: Clarification about 2.* branches

Hey Vinod, fyi I updated CHANGES.txt yesterday to reflect the 2.0.0-alpha release on trunk and branch-2. branch-2.0.0-alpha was just the release branch Arun used for 2.0.0-alpha, we'll create a new branch (2.0.1-alpha) for the next release. Thanks, Eli On Tue, Jun 5, 2012 at 4:40 PM, Vinod Kuma

Re: Clarification about 2.* branches

Hey Vinod, Out of curiosity, why delete the branch? Might make spelunking for svn revs that correspond to a release hard. If we're deleting old release branches might as well delete the others (branch-0.23.0, branch-0.23.0-rc0, branch-0.23.1, branch-0.23.2, etc) as well? Thanks, Eli On Wed, Ju

Re: Clarification about 2.* branches

On Wed, Jun 6, 2012 at 2:10 PM, Eli Collins wrote: > Hey Vinod, > > Out of curiosity, why delete the branch?  Might make spelunking for > svn revs that correspond to a release hard.  If we're deleting old > release branches might as well delete the others (branch-0.23.0,

Re: Clarification about 2.* branches

when we have dead branches lying around. > > Regarding 0.23.*, you are right, I was about to send a separate email > regarding those too. > > Thanks, > +Vinod > > > On Jun 6, 2012, at 2:10 PM, Eli Collins wrote: > >> Hey Vinod, >> >> Out of curiosity, w

Re: a bug in ViewFs tests

Hey Andrew. I'd open a new jira, and thanks for the find btw! Thanks, Eli On Wed, Jul 11, 2012 at 4:08 PM, Andrey Klochkov wrote: > Hi, > I noticed that the fix done in HADOOP-8036 (failing ViewFs tests) was > reverted later when resolving HADOOP-8129, so the bug exists both in > 0.23 and 2.0.

Common and hdfs Jenkins jobs now running tests

Hey gang, The tests were disabled on the common and hdfs jenkins jobs for some reason. This was hiding test failures in the tests that are not run by test-patch (eg hadoop-dist, see HDFS-3690). I've re-enabled the tests on these jobs and filed HADOOP-8610 to get test-patch on Hadoop to run the ro

Re: 10 failures when run test on hadoop-common-project

Hi Yanbo, You can look at the individual tests output (eg hadoop-common-project/hadoop-common/target/surefire-reports/org.apache.hadoop.net.TestTableMapping.txt for TestTableMapping and so on) for an indication as to why the test failed. Thanks, Eli On Mon, Jul 23, 2012 at 3:50 AM, Yanbo Liang

Re: Build failed in Jenkins: Hadoop-Common-trunk #481

to the ephemeral port. > Contributed by Eli Collins > > [bobby] MAPREDUCE-3893. allow capacity scheduler configs max-apps and > max-am-pct per queue (tgraves via bobby) > > [todd] HDFS-3697. Enable fadvise readahead by default. Contributed by Todd > Lipcon. > > --

Re: Java 7 and Hadoop

ocess for this or is it simply updating the wiki Eli mentioned > after sufficient testing? > > Thanks, > Tom > > > On 4/26/12 4:25 PM, "Eli Collins" wrote: > >> Hey Scott, >> >> Nice. Please update this page with your experience when you get a

Re: Where can I find old Hadoop Common Release Binaries?

Hey Carlos, We don't release common separately from mr and hdfs. You can download the hadoop binaries from this site (or preferably it's mirror). http://apache.claz.org/hadoop/common Eg the 0.20.2 binary is here: http://apache.claz.org/hadoop/common/hadoop-0.20.2/hadoop-0.20.2.tar.gz Thanks, El

Re: adding bin and net categories to JIRA/HADOOP

Works for me. I've been adding missing ones that make sense (eg webhdfs). On Mon, Aug 6, 2012 at 11:34 AM, Steve Loughran wrote: > There aren't bin and net categories in JIRA, yet often bugs go against the > code there? > > Should I add them?

Re: fs.local.block.size vs file.blocksize

Hi Ellis, fs.local.block.size is the default FileSystem block size, note however that most file systems (like HDFS, see DistributedFileSystem) override this, eg when using HDFS the default block size is configured with dfs.blocksize which defaults to 64mb. Note in v1 the default block size for hd

Re: S3 FS tests broken?

Passes for me locally, and the precondition that's failing (passing null to Conf#set) from the backtrace looks like the null is coming from: S3Conf.set(FS_DEFAULT_NAME_DEFAULT, S3Conf.get("test.fs.s3.name")); which is set in core-site.xml so something strange is going on. HADOOP-6296 looks re

Re: S3 FS tests broken?

Trevor, Forgot to ask, since you can reproduce this can you confirm and see why S3Conf.get is returning null for test.fs.s3.name? On Mon, Aug 13, 2012 at 6:35 PM, Eli Collins wrote: > Passes for me locally, and the precondition that's failing (passing > null to Conf#set) from the back

Re: Branch 2 release names

On Tue, Sep 4, 2012 at 11:55 AM, Owen O'Malley wrote: > While cleaning up the subversion branches, I thought more about the > branch 2 release names. I'm concerned if we backtrack and reuse > release numbers it will be extremely confusing to users. It also > creates problems for tools like Maven t

Re: test-patch and native code compile

Yea we want jenkins to run with native. How about adding making native optional in test-patch via a flag and updating the jenkins jobs to use it? On Thu, Sep 6, 2012 at 7:25 AM, Alejandro Abdelnur wrote: > Makes sense, though the Jenkins runs should continue to run w/ native, right? > > On Thu, S

Re: difference between hadoop code and the amazon elastic Mapreduce

Peter, Thanks for the info. Do you guys plan to contribute the rewritten s3 code (assume you're referring to org.apache.hadoop.fs.s3) back to Apache? Thanks, Eli On Sun, Sep 9, 2012 at 12:38 PM, Sirota, Peter wrote: > Hi, > > The major differences are in s3 file system that has been rewritten

Re: Commits breaking compilation of MR 'classic' tests

How about adding this step to the MR PreCommit jenkins job so it's run as part test-patch? On Tue, Sep 25, 2012 at 7:48 PM, Arun C Murthy wrote: > Committers, > > As most people are aware, the MapReduce 'classic' tests (in > hadoop-mapreduce-project/src/test) still need to built using ant since

Re: [VOTE] Hadoop-1.0.4-rc0

+1 Thanks Matt. I verified the signatures and ran some basic MR jobs. Btw, it would be good to update the public KEYS repo with your key as well. See http://wiki.apache.org/hadoop/HowToRelease. Thanks, Eli On Thu, Oct 4, 2012 at 1:59 PM, Matt Foley wrote: > Hi, > There has been a request fr

Re: Need to add fs shim to use QFS

Hey Thilee, Thanks for contributing. We don't process pull request on the git mirrors, please upload a patch against trunk and branch-1 if you'd like this included in Hadoop 1.x and 2.x releases. More info here: http://wiki.apache.org/hadoop/HowToContribute Thanks, Eli On Fri, Oct 5, 2012 at 10:

Re: [VOTE] Hadoop-1.0.4-rc0

On Tue, Oct 9, 2012 at 1:02 PM, Matt Foley wrote: > Hi Eli, > Thanks for the suggestion. Looks like this has gotten fleshed out a little > more since I started doing releases. > > I've had my key posted at MIT since the beginning. I've now also uploaded > it to the PGP Global Directory, and uplo

Re: Need to add fs shim to use QFS

On Wed, Oct 10, 2012 at 12:14 PM, Steve Loughran wrote: > On 10 October 2012 16:03, Thilee Subramaniam wrote: > >> Hi Steve, >> >> Like Harsh said, HADOOP-8886 addresses removing KFS from apache tree. >> >> But I interpret your suggestion as 'moving qfs.jar out of apache tree, and >> keeping the

[DISCUSS] remove packaging

Hey guys, Heads up: I filed HADOOP-8925 to remove the packaging from trunk and branch-2. The packages are not currently being built, were never updated for MR2/YARN, I'm not aware of anyone planning to do this work or maintain them, etc. No sense in letting them continue to bit rot in the code ba

Re: [DISCUSS] remove packaging

me a part of HADOOP-8914 (Automating > the release build). > > --Bobby > > > -Original Message- > From: Eli Collins [mailto:e...@cloudera.com] > Sent: Monday, October 15, 2012 10:33 AM > To: common-dev@hadoop.apache.org > Subject: [DISCUSS] remove packaging > > Hey guys,

Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack

-1, 0, -1 IIUC the only platform we plan to add support for that we can't easily support today (w/o an emulation layer like cygwin) is Windows, and it seems like making the bash scripts simpler and having parallel bat files is IMO a better approach. On Sat, Nov 24, 2012 at 12:13 PM, Matt Foley w

Re: Release numbering for branch-2 releases

Thanks for bringing this up Arun. One of the issues is that we haven't been clear about what type of compatibility breakages are allowed, and which are not. For example, renaming FileSystem#open is incompatible, and not OK, regardless of the alpha/beta tag. Breaking a server/server APIs is OK pr

Re: Release numbering for branch-2 releases

ur Roadmap wiki and other docs post this > discussion. > > thanks, > Arun > > > > On Jan 30, 2013, at 4:21 PM, Eli Collins wrote: > > > Thanks for bringing this up Arun. One of the issues is that we > > haven't been clear about what type of compatibility

Re: Heads up - merge branch-trunk-win to trunk

Thanks for the update Suresh. Has any testing been done on the branch on Linux aside from running the unit tests? Thanks, Eli On Thu, Feb 7, 2013 at 5:42 PM, Suresh Srinivas wrote: > The support for Hadoop on Windows was proposed in > HADOOP-8079

Re: [Vote] Merge branch-trunk-win to trunk

Bobby raises some good questions. A related one, since most current developers won't add Windows support for new features that are platform specific is it assumed that Windows development will either lag or will people actively work on keeping Windows up with the latest? And vice versa in case Wi

Fwd: Compatibility in Apache Hadoop

On Mon, Apr 22, 2013 at 5:42 PM, Steve Loughran wrote: > > On 22 April 2013 14:00, Karthik Kambatla wrote: > > > Hadoop devs, > > > > > > This doc does not intend to propose new policies. The idea is to have one > > document that outlines the various compatibility concerns (lots of areas > > beyo

Re: Use hadoop.relaxed.worker.version.check to allow versions in the same major version?

Hey Karthik, We already support this for HDFS, see HDFS-2983 (Relax the build version check to permit rolling upgrades within a release). We should do so for Yarn as well, MAPREDUCE-4150 tracks this. I don't think Ahmed is working on it so would be great for you or someone else to take it. Thank

Re: Heads up - 2.0.5-beta

On Fri, Apr 26, 2013 at 11:15 AM, Arun C Murthy wrote: > > On Apr 25, 2013, at 7:31 PM, Roman Shaposhnik wrote: > >> On Thu, Apr 25, 2013 at 6:34 PM, Arun C Murthy wrote: >> >>> With that in mind, I really want to make a serious push to lock down APIs >>> and wire-protocols for hadoop-2.0.5-beta

Re: Heads up - 2.0.5-beta

On Fri, Apr 26, 2013 at 2:42 PM, Suresh Srinivas wrote: > Eli, I will post a more detailed reply soon. But one small correction: > > > I'm also not sure there's currently consensus on what an incompatible >> change is. For example, I think HADOOP-9151 is incompatible because it >> broke client/ser

Re: [VOTE] - Release 2.0.5-beta

On Wed, May 15, 2013 at 1:29 PM, Matt Foley wrote: > >> Arun, not sure whether your "Yes to all" already covered this, but I'd > like > >> to throw in support for the compatibility guidelines being a blocker. > > +1 to that. Definitely an overriding concern for me. > > +1 Likewise. Would be g

Re: [VOTE] Plan to create release candidate for 0.23.8

+1 On Friday, May 17, 2013, Thomas Graves wrote: > Hello all, > > We've had a few critical issues come up in 0.23.7 that I think warrants a > 0.23.8 release. The main one is MAPREDUCE-5211. There are a couple of > other issues that I want finished up and get in before we spin it. Those > includ

Re: [PROPOSAL] change in bylaws to remove Release Plan vote

+1 thanks Matt. On Tue, May 21, 2013 at 2:10 PM, Matt Foley wrote: > Hi all, > This has been a side topic in several email threads recently. Currently we > have an ambiguity. We have a tradition in the dev community that any > committer can create a branch, and propose release candidates fro

Re: Cannot update "PoweredBy" wiki page

Hey Adam, If you create a user ID for the Hadoop wiki and send it to me I'll update this page which will give you perms. http://wiki.apache.org/hadoop/ContributorsGroup On Thu, May 30, 2013 at 3:26 PM, Adam Kawa wrote: > Could someone please add me? I promise not to break anything ;) > > > 2013

Re: libhdfs with FileSystem cache issue can causes to memory leak ?

Hey Huy, What version of hadoop are you using? I think HADOOP-4655 may have resolved the issue you're seeing but I think is only in 20 and later. Thanks, Eli On Mon, Oct 12, 2009 at 8:52 PM, Huy Phan wrote: > Hi All, >    I'm writing a multi-thread application using libhdfs in C, a known issue

Re: Hadoop coding style guideline

> My preference is to permit both.  I like to maximize the amount of readable > logic per screen, and find that close braces around one-line expressions > don't improve readability (since indentation already indicates the nesting) > and decrease the amount of per-screen logic. However I know reason

Re: ant test-patch does not work

You can apply the patch for HDFS-423 which hasn't been committed yet. Also, can you clarify what you're working on? HDFS and libhdfs do not yet support symlinks (HDFS-245) so how will you implement fuse-dfs symlinks? Thanks, Eli On Sun, Dec 6, 2009 at 4:25 PM, zhuweimin wrote: > Hi, > > I'm crea

Re: ant test-patch does not work

Forgot to mention, the error you're seeing is because you're not using jdk5 for forest, eg java5.home below needs to point to a jdk v5 installation. On Sun, Dec 6, 2009 at 4:25 PM, zhuweimin wrote: > Hi, > > I'm creating a patch for fuse-dfs symbolic link support. > > I got an error when testing

Re: ant test-patch does not work

>     [exec] Caused by: org.apache.cocoon.sitemap.PatternException: Cannot > get module named 'lm' in expression '{lm:project.{1}{2}.pdf}' >     [exec]     at > org.apache.cocoon.components.treeprocessor.variables.PreparedVariableResolve > r.getNewModuleToken(PreparedVariableResolver.java:142) > >

Re: HDFS-758 in Hadoop-21 , Updates to Namenode health page

On Mon, Dec 7, 2009 at 2:00 PM, Sanjay Radia wrote: > > On Nov 25, 2009, at 12:46 PM, Allen Wittenauer wrote: > > >>Then you'll have no issues patching other things in 0.21 that are >> actual >> bug fixes that also meet this criteria, right? Or does this only apply to >> things that Yahoo! i

Re: Is there a need to rename start-all.sh to start-hdfs-all.sh?

Maybe you mean rename start-dfs.sh? start-all.sh runs both start-dfs.sh and start-mapred.sh so it's appropriately named. Thanks, Eli On Sun, Jan 17, 2010 at 9:47 PM, Hubert Chang wrote: > > I think renaming start-all.sh to start-hdfs-all.sh is more meaningful. > -- > View this message in contex

Re: org.apache.hadoop.fs.Path toString() bug

On Thu, Jan 21, 2010 at 12:27 AM, Victor Hsieh wrote: > Hi, > > I probably find a bug in org.apache.hadoop.fs.Path.  Could you please > verify it for me? > > I suppose the following expression returns "file:///foor/bar", but > what returned is actually "file:/foo/bar". >    new Path("file", "", "/

Re: Namespace partitioning using Locality Sensitive Hashing

On Mon, Mar 1, 2010 at 5:42 PM, Ketan Dixit wrote: > Hello, > Thank you Konstantin and  Allen for your reply. The information > provided really helped to improve my understanding. > However I still have few questions. > How Symlinks/ soft links are used to solve the probem of partitioning. > (Wher

Re: Namespace partitioning using Locality Sensitive Hashing

; out of favor. > > I wouldn't deny that symlinks are an incredible tool to fight namespace > growth - but it's not a 100% solution. > > That said, I'm looking forward to symlinks to solve a few local problems! > > Brian > > On Mar 1, 2010, at 8:15 PM,

Re: adding new filesystems

On Wed, Mar 24, 2010 at 3:28 AM, Steve Loughran wrote: > > I'm looking at what it currently takes to implement new back end > filestores, getting lost in the details. > > This is my current understanding -am I wrong? > > 1. There is an AbstractFileSystem, came in with HADOOP-6223, and is in > SVN

Re: HADOOP-6704 - getResource() usage ok?

Hey Neil, There are other uses of getResource in Hadoop so should be fine. Should you be using getClassLoader#getResource() instead of Class#getResource()? Thanks, Eli On Fri, Apr 30, 2010 at 12:46 PM, Neil Bliss wrote: > Hey folks, > > In the unit tests for my HADOOP-6704 patch, I'm using getR

Re: HADOOP-6704 - getResource() usage ok?

On Fri, Apr 30, 2010 at 1:22 PM, Neil Bliss wrote: >> >>> Thanks Eli, I'll have a look how the other instances are used, and see if >>> I can spot the problem. >>> >>> - Neil >>> >>> >>> On Fri, Apr 30, 2010 at 1:17 PM, Eli Collins

Re: HADOOP-6704 - getResource() usage ok?

Nope, just "submit patch" if you haven't done so already, Huson will pick it up and run the tests. When you upload a new patch and you've already submitted a patch you need to "cancel patch" then "submit patch" again to get Hudson to run again. Thanks, Eli On Fri, Apr 30, 2010 at 4:24 PM, Neil B

Re: data locality on HDFS

Hey Momina, Here's the path on 20: DistributedFileSystem#getFileBlockLocations -> DFSClient#getFileBlockLocations -> callGetBlockLocations -> ClientProtocol#getBlockLocations -> (via proxy) NameNode#getBlockLocations See createNamenode and createRPCNamenode in the DFSClient

Re: Minutes: Hadoop Contributor Meeting 05/06/2010

Hey Arun, I updated the agenda on Meetup. I was assuming the branching would fall out of how to implement HEP but good to discuss separately as well. Also, I added moving contrib out of the repos since a couple people mentioned that at the last meetup but we should do that time permitting the oth

Re: Hadoop Contributors Meeting next Friday

core repos On Fri, May 21, 2010 at 9:51 AM, Eli Collins wrote: > Hey Hadoop Core Developers, > > Just a reminder that the next contributors meeting is next Friday: > > Date/Time: Friday, May 28th 2-4 pm > Venue: Cloudera HQ. 210 Portage Ave Palo Alto, CA 94306 > Agenda: >

Re: RawComparator of BooleanWritable broken ?

Hey Johannes, Nice find. Mind adding the tests you wrote to the patch in HADOOP-6928 which Owen just filed? Thanks, Eli On Thu, Aug 26, 2010 at 4:57 AM, Johannes Zillmann wrote: > Hi folks, > > i'm stumbled into a situation with hadoop-0.20.2 where sorting with a > BooleanWritable doesn't wor

Re: RawComparator of BooleanWritable broken ?

gt; > Johannes > > On Aug 26, 2010, at 8:50 PM, Eli Collins wrote: > >> Hey Johannes, >> >> Nice find.  Mind adding the tests you wrote to the patch in >> HADOOP-6928 which Owen just filed? >> >> Thanks, >> Eli >> >> On Thu, Aug

Re: RawComparator of BooleanWritable broken ?

from git/svn for > hadoop-hdfs and hadoop-mapreduce ? > In hadoop-common there are branches and tags for the release. But how to get > the corresponding version of the other 2 projects ? > -- > > best regards > Johannes > > > On Aug 27

Re: FW: [jira] Moved: (MAPREDUCE-2008) write Junit test cases that run the Hadoop examples

Hey Saikat, Running ant test-core in mapreduce on trunk should pass (modulo a handful of flaky tests). What failures are you seeing? Info on how to add unit tests: http://wiki.apache.org/hadoop/HowToDevelopUnitTests Thanks, Eli On Sat, Sep 4, 2010 at 10:27 PM, Saikat Kanjilal wrote: > > HI Fo

Re: [jira] Moved: (MAPREDUCE-2008) write Junit test cases that run the Hadoop examples

ur thoughts. > > Sent from my iPhone > > On Sep 5, 2010, at 1:55 PM, Eli Collins wrote: > >> Hey Saikat, >> >> Running ant test-core in mapreduce on trunk should pass (modulo a >> handful of flaky tests). What failures are you seeing? >> >> Info on

Re: FW: [jira] Moved: (MAPREDUCE-2008) write Junit test cases that run the Hadoop examples

On Sat, Sep 4, 2010 at 10:27 PM, Saikat Kanjilal wrote: > > HI Folks,I am working on this jira task and had some additional questions:1) > It seems that whenever I go to run the test or the test-core target on the > mapreduce codebase they both fail, I did a fresh sync prior to running these >

Re: (HADOOP-6916) Implement append operation for S3FileSystem

Hey Oleg, Apologies for the slow reply. I'll review on jira. Thanks, Eli On Tue, Oct 5, 2010 at 8:52 AM, Aleshko, Oleg wrote: > Hi, guys! > > Can somebody please help me to get this issue fixed in some of the upcoming > releases? > I've attached a patch, but not sure why the automatic verific

Re: [DISCUSSION]: Future of Hadoop system testing

Hey Cos, I can't comment on the relative amounts of effort between the options here, but it seems like having a single system testing framework is preferable, in which case it makes sense to extend Herriot to work w/o build instrumentation. This new "standard" mode even seems to be a reasonable d

Re: TestDU failures on common RedHat distributions

Hey Kirby, I filed HADOOP-7045. Apologies for the slow reply! Thanks, Eli On Tue, Sep 8, 2009 at 4:50 PM, Kirby Bohling wrote: > All, > >   I was trying to get Hadoop compiling and passing unit tests.  I > have having problems with the TestDU test.  I have searched the issue > tracker and the w

Re: Native code and autoconf generated files

Hey Noah, The short answer is use older versions (RHEL/Centos 5) of autotools. Longer term we need to stop checking in auto-generated files (HADOOP-6436). This pertains to trunk and all releases btw. Thanks, Eli On Sat, Dec 4, 2010 at 3:43 PM, Noah Watkins wrote: > Hi, > > This question perta

Re: *FileSystem and *Fs APIs

Hey Gerald, Yes, there's a new API. FileSystem is replaced by two new classes: FileContext (the user-facing API) and AbstractFileSystem (the fs implementation API). So the AbstractFileSystem derived classes replace the FileSystem derived classes. Thanks, Eli On Thu, Dec 30, 2010 at 8:16 PM, Zhen

Re: *FileSystem and *Fs APIs

On Fri, Dec 31, 2010 at 12:06 AM, Eli Collins wrote: >> Hey Gerald, >> >> Yes, there's a new API. FileSystem is replaced by two new classes: >> FileContext (the user-facing API) and AbstractFileSystem (the fs >> implementation API). So the AbstractFileSyste

Re: *FileSystem and *Fs APIs

On Sun, Jan 2, 2011 at 2:32 PM, Noah Watkins wrote: >> Most of the Hadoop code base (MapReduce, FsShell etc) has not yet >> switched to the new API yet. > > I have a FS implementation that is nearly complete, and designed against the > 0.21 code base, using the old FileSystem interface. Do you hav

Re: Hadoop-common-trunk-Commit is failing since 01/19/2011

Hey Konstantin, The only build breakage I saw from HADOOP-6904 is MAPREDUCE-2290, which was fixed. Trees from trunk are compiling against each other for me (eg each installed to a local maven repo), perhaps the upstream maven repo hasn't been updated with the latest bits yet. Thanks, Eli On Mon

Re: VOTE: Committing HADOOP-6949 to 0.22 branch

+1 On Mon, Mar 28, 2011 at 9:39 PM, Konstantin Shvachko wrote: > HADOOP-6949 introduced a very important optimization to the RPC layer. Based > on the benchmarks presented in HDFS-1583 this provides an order of magnitude > improvement of current RPC implementation. > RPC is a common component of

Re: [VOTE] Release candidate 0.20.203.0-rc0

Hey Owen, I took a quick look at the changes in the branch (specifically the range of 200 or so changes where the first line of the commit doesn't reference a jira). Most of these look like backports of patches on jira, however there also seem to be changes that don't correspond to changes in tru

Re: [VOTE] Release candidate 0.20.203.0-rc0

On Sat, Apr 30, 2011 at 7:17 AM, Owen O'Malley wrote: > On Fri, Apr 29, 2011 at 7:21 PM, Eli Collins wrote: > >> I took a quick look at the changes in the branch > > > Thanks for taking a look. Please continue to inspect and test out the > release and vote. I'

Re: [VOTE] Release candidate 0.20.203.0-rc0

On Sun, May 1, 2011 at 7:20 PM, Arun C Murthy wrote: > Eli, > > On Apr 30, 2011, at 7:19 PM, "Eli Collins" wrote: >> Seems like we need to do this before releasing 0.20.203 to prevent >> blocking the upcoming 0.22 release, due to it being a regression >> aga

Re: [VOTE] Release candidate 0.20.203.0-rc0

On Fri, Apr 29, 2011 at 4:09 PM, Owen O'Malley wrote: > I think everything is ready to go on the 0.20.203.0 release. It includes > security and a lot of improvements in the capacity scheduler and JobTracker. > > Should we release http://people.apache.org/~omalley/hadoop-0.20.203.0-rc0/? > Based

Re: Discussions - Re: [VOTE] Release candidate 0.20.203.0-rc0

Hey Eric, I don't have any objections to a release from branch-0.20-security-203. However when I examined the specific patch set I noticed the are important implications with respect to compatibility (of for 0.20.2 and 0.22), a question about project model (eg not reviewing patches on jira before

Questions wrt security branches

He guys, Do all changes for 0.20.2xx release go through branch-0.20-security, then get merged to a particular -2xx branch? Why create a new branch for every new dot release? Ie if the intent that the branch will be dead after release, why not release from a single branch? It's hard to see that e

  1   2   3   4   5   >