Re: [VOTE] Release hadoop-2.0.3-alpha

I downloaded the binary package and ran a few example jobs on a 3 node cluster. Everything seems to be working OK on it, I did see WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable For every shell command, but just l

timeout is now requested to be on all tests

Sorry about cross posting, but this will impact all developers and I wanted to give you all a heads-up. HADOOP-9112 was just checked it. This means that the pre commit build will now give a –1 for any patch with junit tests that do not include

Re: [Vote] Merge branch-trunk-win to trunk

After this is merged in is Windows still going to be a second class citizen but happens to work for more than just development or is it a fully supported platform where if something breaks it can block a release? How do we as a community intend to keep Windows support from breaking? We don't have

Re: [Vote] Merge branch-trunk-win to trunk

out is an interesting one >>-- >> ie the idea that we would not merge windows support to trunk, but rather >> treat is as a "parallel code line" which lives in the ASF and has its >>own >> builds and releases. The windows team would periodically merge >>tru

Re: [VOTE] Release Apache Hadoop 0.23.7

+1 (binding) I downloaded the release and ran a few sanity tests on it. --Bobby On 4/11/13 2:55 PM, "Thomas Graves" wrote: >I've created a release candidate (RC0) for hadoop-0.23.7 that I would like >to release. > >This release is a sustaining release with several important bug fixes in >it. >

Re: [VOTE] Release Apache Hadoop 2.0.4-alpha

+1 (binding) Downloaded the tar ball and ran some simple jobs. --Bobby Evans On 4/17/13 2:01 PM, "Siddharth Seth" wrote: >+1 (binding) >Verified checksums and signatures. >Built from the source tar, deployed a single node cluster and tested a >couple of simple MR jobs. > >- Sid > > >On Fri, Ap

Re: For release 2.0.X, about when will have a stable release?

What do you consider stable? Do you mean that it will not have the -alpha tag? That should hopefully be in a month or two. Do you mean that it is stable enough for production use? That is likely to be before the end of the year depending on the size of your cluster and your use cases, but a lot o

Re: Heads up - 2.0.5-beta

I agree that "destructive" is not the correct word to describe features like snapshots and windows support. However, I also agree with Konstantin that any large feature will have a destabilizing effect on the code base, even if it is done on a branch and thoroughly tested before being merged in. H

Re: [VOTE] Plan to create release candidate for 0.23.8

+1 On 5/17/13 4:10 PM, "Thomas Graves" wrote: >Hello all, > >We've had a few critical issues come up in 0.23.7 that I think warrants a >0.23.8 release. The main one is MAPREDUCE-5211. There are a couple of >other issues that I want finished up and get in before we spin it. Those >include HDFS-

Re: [VOTE] Release Apache Hadoop 0.23.8

+1 Downloaded the release and ran a few basic tests. --Bobby On 5/28/13 11:00 AM, "Thomas Graves" wrote: > >I've created a release candidate (RC0) for hadoop-0.23.8 that I would like >to release. > >This release is a sustaining release with several important bug fixes in >it. The most critica

Re: [VOTE] Release Apache Hadoop 0.23.9

+1 downloaded the release. Ran a couple of simple jobs and everything worked. On 7/1/13 12:20 PM, "Thomas Graves" wrote: >I've created a release candidate (RC0) for hadoop-0.23.9 that I would like >to release. > >The RC is available at: >http://people.apache.org/~tgraves/hadoop-0.23.9-candidate

Re: mapred replication

Without the stack trace of the exceptions it is hard to tell. The pruning is asynchronous, but so is a node crashing with a replica on it. The client is supposed to detect this situation and find a new replica that works. I am not that familiar with the code, but I believe in some if not all of

Re: Hoop & webhdfs, common HTTP API and more

Jersey is a library that can sit on top of any java web container, Jetty, Netty, Tomcat, etc... On 10/19/11 5:09 AM, "Steve Loughran" wrote: On 17/10/11 23:12, Alejandro Abdelnur wrote: > Calling for comments, > >https://issues.apache.org/jira/browse/HDFS-2178 > > Thanks. > > Alejandro >

Re: replication in HDFS

Our largest cluster is several thousand nodes and we still run with a replication factor of 3. We have not seen any benefit from having a larger replication factor except when it is a resource that lots of machines will use, aka distributed cache. Other then that 3 seems just fine for most map

Re: Proposal: abandon protocol translator layer for cluster-internal RPCs

I think what we are talking about here is removing some of the extra layers of abstraction in java. The wire protocol used will be identical in either case. It is just that we would have to use the Protocol Buffer Builder APIs instead of wrapping them with our own custom getters/setters. I am

Re: ioreply

It definitely sounds interesting, kind of like gridmix. I think that there are three big issues here. 1. Where are you going to store all of the data, or are you just going to generate random data? If it is random data then you can do this almost totally form an anonymised version of the au

Re: OSGi and classloaders

Guillaume, I am not super familiar with OSGi. I have used it a little in the past, but that was 5+ years ago. I am in favor of something that will fix the CLASSPATH problems that we currently have and would allow for CLASSPATH isolation between Hadoop itself and the applications that use Hadoop.

Re: OSGi and classloaders

i and a >performance improvement by avoiding re-parsing the xml configuration >multiple times. > >On Mon, Jul 9, 2012 at 4:37 PM, Robert Evans wrote: > >> Guillaume, >> >> I am not super familiar with OSGi. I have used it a little in the past, >> but that was 5

Re: StreamXMLRecordReader

It wasn't removed ./hadoop-tools/hadoop-streaming/src/main/java/org/apache/hadoop/streaming/S treamXmlRecordReader.java It just got mavenized and moved to a different location. And please don't cross post, hdfs-dev is not really going to help with a streaming question. --Bobby Evans On 7/12/12

Re: Commits breaking compilation of MR 'classic' tests

As per my comment on the bug. I though we were going to remove them. MAPREDUCE-4266 only needs a little bit more work, change a patch to a script, before they disappear entirely. I would much rather see dead code die then be maintained for a few tests that are mostly testing the dead code itself

Re: Commits breaking compilation of MR 'classic' tests

MR 'classic' tests Fair, however there are still tests which need to be ported over. We can remove them after the port. On Sep 26, 2012, at 9:54 AM, Robert Evans wrote: As per my comment on the bug. I though we were going to remove them. MAPREDUCE-4266 only needs a little bit more wo

Re: Fix versions for commits branch-0.23

I don't see much of a reason to have the same JIRA listed under both 0.23 and 2.0. I can see some advantage of being able to see what went into 0.23.X by looking at a 2.0.X CHANGES.txt, but unless the two are released at exactly the same time they will be out of date with each other in the best ca

Re: Is there a way to mount HAR the same way HDFS can be mounted with FUSE?

Using Viewfs, you probably could mount a HAR and then use fuse to mount the entire file system. I am not sure how well it would work though. --Bobby On 10/17/12 12:31 PM, "Peter Litvak" wrote: >I couldn't find anything that talks about this use case although I'd >imagine it is not that uncommo

Re: Is there a way to mount HAR the same way HDFS can be mounted with FUSE?

mples of using Viewfs? Which >tool set is it included in? > >On Wed, Oct 17, 2012 at 2:49 PM, Robert Evans wrote: > >> Using Viewfs, you probably could mount a HAR and then use fuse to mount >> the entire file system. I am not sure how well it would work though. >> &

New competition

I saw an article yesterday saying the GlusterFS 3.3 now has Hadoop bindings. I also ran across XtreemFS a while back, which also supports Hadoop bindings. Both of them claim to be faster and more scalable than HDFS. Has anyone in the community done some actual benchmarks on the same hardware for