Re: Hadoop API performance testcases

2012-07-11 Thread Ravi Prakash
Hi Amir, Did you have any specific APIs in mind? There's obviously, sort, shuffle, gridmix, DFSIO etc. which measure different aspects of Hadoop job execution. Ravi On Tue, Jul 10, 2012 at 12:58 PM, Amir Sanjar wrote: > > Hi all, are there any performance testcases for hadoop APIs? We are loo

Re: Jetty fixes for Hadoop

2012-07-11 Thread Thomas Graves
One question I did have on this is if anyone is now seeing more jetty issues on the datanode with possibly increased usage via webhdfs - do we need something similar to https://issues.apache.org/jira/browse/MAPREDUCE-3184 on the datanode side? Tom On 7/10/12 5:19 PM, "Todd Lipcon" wrote: > +1

[jira] [Created] (HADOOP-8588) SerializationFactory shouldn't throw a NullPointerException if the serializations list is empty

2012-07-11 Thread Harsh J (JIRA)
Harsh J created HADOOP-8588: --- Summary: SerializationFactory shouldn't throw a NullPointerException if the serializations list is empty Key: HADOOP-8588 URL: https://issues.apache.org/jira/browse/HADOOP-8588

Re: Jetty fixes for Hadoop

2012-07-11 Thread Robert Evans
I am +1 on this also, although I think we need to look at moving to Jetty-7 or possibly dropping Jetty completely and look at Netty or even Tomcat long term. Jetty has just been way too unstable at Hadoop scale and that has not really changed with newer versions of Jetty. Sticking with an old for

Re: Jetty fixes for Hadoop

2012-07-11 Thread Suresh Srinivas
I agree that we should explore alternatives the forked version of Jetty. This is a longer term goal. In the interim, lets do jetty-6.1.26 + fixes. On Wed, Jul 11, 2012 at 6:03 AM, Robert Evans wrote: > I am +1 on this also, although I think we need to look at moving to > Jetty-7 or possibly drop

Re: Jetty fixes for Hadoop

2012-07-11 Thread Owen O'Malley
In 2.0 we've reimplemented the shuffle using Netty for a significant speed up. Moving webhdfs to Netty would be interesting. -- Owen On Jul 11, 2012, at 6:03, Robert Evans wrote: > I am +1 on this also, although I think we need to look at moving to > Jetty-7 or possibly dropping Jetty complet

Re: Problem setting up 1st generation Hadoop-0.20 (ANT build) in Eclipse

2012-07-11 Thread Pavan Kulkarni
I resolved the issue of "*Exception in thread "main" java.lang.InternalError: Can't connect to X11 window server using ':0' as the value of the DISPLAY variable."*. The problem was if the "ant tar" is being issued from root whereas the owner of the X Display Socket belongs to different user then, t

[jira] [Resolved] (HADOOP-8565) AuthenticationFilter#doFilter warns unconditionally when using SPNEGO

2012-07-11 Thread Alejandro Abdelnur (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-8565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alejandro Abdelnur resolved HADOOP-8565. Resolution: Duplicate This is a dup of HADOOP-8355 (which I've missed to backport

[jira] [Resolved] (HADOOP-8587) HarFileSystem access of harMetaCache isn't threadsafe

2012-07-11 Thread Eli Collins (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-8587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins resolved HADOOP-8587. - Resolution: Fixed Fix Version/s: 2.0.1-alpha 1.2.0 Target

a bug in ViewFs tests

2012-07-11 Thread Andrey Klochkov
Hi, I noticed that the fix done in HADOOP-8036 (failing ViewFs tests) was reverted later when resolving HADOOP-8129, so the bug exists both in 0.23 and 2.0. I'm going to provide an alternative fix. Should I reopen HADOOP-8036 or create a new one instead? Thanks. -- Andrey Klochkov

[jira] [Resolved] (HADOOP-8365) Add flag to disable durable sync

2012-07-11 Thread Suresh Srinivas (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-8365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas resolved HADOOP-8365. - Resolution: Fixed Release Note: This patch enables durable sync by default. Install

Re: a bug in ViewFs tests

2012-07-11 Thread Eli Collins
Hey Andrew. I'd open a new jira, and thanks for the find btw! Thanks, Eli On Wed, Jul 11, 2012 at 4:08 PM, Andrey Klochkov wrote: > Hi, > I noticed that the fix done in HADOOP-8036 (failing ViewFs tests) was > reverted later when resolving HADOOP-8129, so the bug exists both in > 0.23 and 2.0.

[jira] [Created] (HADOOP-8589) ViewFs tests fail when tests dir is under Jenkins home dir

2012-07-11 Thread Andrey Klochkov (JIRA)
Andrey Klochkov created HADOOP-8589: --- Summary: ViewFs tests fail when tests dir is under Jenkins home dir Key: HADOOP-8589 URL: https://issues.apache.org/jira/browse/HADOOP-8589 Project: Hadoop Commo

Re: plz help! HDFS to S3 copy issues

2012-07-11 Thread Steve Loughran
This is a hadoop-user q, not a development one -please use the right as user questions get ignored on the dev ones. also: http://wiki.apache.org/hadoop/ConnectionRefused On 11 July 2012 19:23, Momina Khan wrote: > i use the following command to try to copy data from hdfs to my s3 bucket > > ubu

[jira] [Created] (HADOOP-8590) Backport HADOOP-7318 (MD5Hash factory should reset the digester it returns) to branch-1

2012-07-11 Thread Todd Lipcon (JIRA)
Todd Lipcon created HADOOP-8590: --- Summary: Backport HADOOP-7318 (MD5Hash factory should reset the digester it returns) to branch-1 Key: HADOOP-8590 URL: https://issues.apache.org/jira/browse/HADOOP-8590