Re: Latency and speed of HDFS

2011-01-28 Thread Harsh J
Moving discussion to hdfs-user mailing list: hdfs-u...@hadoop.apache.com On Sat, Jan 29, 2011 at 5:06 AM, Pei HE wrote: > Hi all, > I want to know the detailed performance of Hadoop. > > I am writing a client code to test the latency, transfer speed of > HDFS, and the time of initiating a JVM for

Re: Hadoop-common-trunk-Commit is failing since 01/19/2011

2011-01-28 Thread Konstantin Boudnik
Wow. looks like it has been happening for about a month now. Looks like there's something strange happening to the build host, because it can't overcome this step of the build: [exec] checking whether we are cross compiling... no It handing right on that step. I wonder if any compiler/librar

Re: Latency and speed of HDFS

2011-01-28 Thread Da Zheng
Have you tried TestDFSIO? I think it's quite a good benchmark to measure the performance of HDFS. If you want to know how to write data to HDFS directly, you can read its code. Da On 1/28/11 6:36 PM, Pei HE wrote: > Hi all, > I want to know the detailed performance of Hadoop. > > I am writing a

Hadoop-common-trunk-Commit is failing since 01/19/2011

2011-01-28 Thread Konstantin Shvachko
I see Hadoop-common-trunk-Commit is failing and not sending any emails. It times out on native compilation and aborts. Therefore changes are not integrated, and now it lead to hdfs and mapreduce both not compiling. Can somebody please take a look at this. The last few lines of the build are below.

Latency and speed of HDFS

2011-01-28 Thread Pei HE
Hi all, I want to know the detailed performance of Hadoop. I am writing a client code to test the latency, transfer speed of HDFS, and the time of initiating a JVM for a task. Does anyone know that which classes can be called without running a whole map/reduce job to find out the performance of H

[jira] Created: (HADOOP-7126) TestDFSShell fails in trunk

2011-01-28 Thread Po Cheung (JIRA)
TestDFSShell fails in trunk --- Key: HADOOP-7126 URL: https://issues.apache.org/jira/browse/HADOOP-7126 Project: Hadoop Common Issue Type: Bug Affects Versions: 0.22.0 Environment: Windows Rep

[jira] Created: (HADOOP-7125) Datanode Disk Fail Inplace

2011-01-28 Thread Bharath Mundlapudi (JIRA)
Datanode Disk Fail Inplace -- Key: HADOOP-7125 URL: https://issues.apache.org/jira/browse/HADOOP-7125 Project: Hadoop Common Issue Type: Sub-task Reporter: Bharath Mundlapudi This sub-task will be a plac

[jira] Created: (HADOOP-7124) TaskTracker Disk Fail Inplace

2011-01-28 Thread Bharath Mundlapudi (JIRA)
TaskTracker Disk Fail Inplace - Key: HADOOP-7124 URL: https://issues.apache.org/jira/browse/HADOOP-7124 Project: Hadoop Common Issue Type: Sub-task Reporter: Bharath Mundlapudi This sub-task will be

[jira] Created: (HADOOP-7123) Hadoop Disk Fail Inplace

2011-01-28 Thread Bharath Mundlapudi (JIRA)
Hadoop Disk Fail Inplace Key: HADOOP-7123 URL: https://issues.apache.org/jira/browse/HADOOP-7123 Project: Hadoop Common Issue Type: Improvement Reporter: Bharath Mundlapudi This is an umbrella Jira for

Example for "Strings Comparison"

2011-01-28 Thread Rawan AlSaad
Dear all, I am looking for an example for a Mapreduce java implementation for strings comparison [pair-wise comparison]. If anybody have gone through a similar example before, could you please help pointing me to a code example for this? Thanks Rawan