See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/649/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 741848 lines...]
[junit] at
org.apache.had
On Apr 26, 2011, at 10:40 PM, Konstantin Boudnik wrote:
Oops, the message came out garbled. I meant to say
I assume the outlined changes won't prevent an earlier version of
HDFS from
upgrades to the federation version, right?
Yes absolutely. We have tested upgrades .
Besides our ops will
I posted the TestDFSIO comparison with and without federation to HDFS-1052.
Please let me know if it addresses your concern. I am also adding it here:
TestDFSIO read tests
*Without federation:*
- TestDFSIO - : read
Date & time: Wed Apr 27 02:04:24 PDT 2011
Number of files
It is not a surprise that the performance of Federation is better than trunk
since, as Suresh mentioned previously, we improved some components of HDFS when
we were developing Federation.
Regards,
Nicholas
From: suresh srinivas
To: hdfs-dev@hadoop.apache.or
Good to see the performance improvements with federation. Curious to know
whether it is because of the associated refactoring?
On 4/27/11 10:02 AM, "suresh srinivas" wrote:
I posted the TestDFSIO comparison with and without federation to HDFS-1052.
Please let me know if it addresses your conce
Interesting... while the read performance has only marginally improved
<4% (still a good thing) the write performance shows significantly
better improvements >10%. Very interesting asymmetry, indeed.
Suresh, what was the size of the cluster in the testing?
Cos
On Wed, Apr 27, 2011 at 10:02, sur
Nice performance data! The federation branch definitely adds code
complexity to HDFS, but this is a long waited feature to improve HDFS
scalability and is a step forward to separating the namespace management
from the storage management. I am for merging this to trunk.
Hairong
On 4/27/11 10:02 AM
Document dfs.datanode.max.transfer.threads in hdfs-default.xml
--
Key: HDFS-1866
URL: https://issues.apache.org/jira/browse/HDFS-1866
Project: Hadoop HDFS
Issue Type: Improvement
On Apr 26, 2011, at 11:34 PM, suresh srinivas wrote:
>> 2. I assume that merging requires a vote. I am sure people who know bylaws
>> better than I do will correct me if it is not true.
>> Did I miss the vote?
>>
>
>
> As regards to voting, since I was not sure about the procedure, I had
> cons
I ran these tests on my laptop. I would like to use this data to emphasize
that there is no regression in performance. I am not sure with just the
tests that I ran we could conclude there is a huge gain in performance with
federation. When out performance test team runs tests at scale we will get
m
Hey Suresh,
Do you plan to update the patch on HDFS-1052 soon? Trunk has moved on
a little bit since the last patch. I assume we vote on the patch
there. I think additional review feedback (beyond what's already been
done) can be handled after the code is merged, I know what a pain it
is to keep
If there are no further issues by tonight, I will merge the branch into
trunk.
Regards,
Suresh
On Wed, Apr 27, 2011 at 1:53 PM, Owen O'Malley wrote:
> On Apr 26, 2011, at 11:34 PM, suresh srinivas wrote:
>
> >> 2. I assume that merging requires a vote. I am sure people who know
> bylaws
> >> be
See https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/37/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 3319 lines...]
compile-hdfs-test:
[delete] D
Thanks Eli.
The merge of latest changes in trunk is not straight forward. I will get it
done tonight and post a new patch. That means the earlier the merge can
happen is tomorrow.
On Wed, Apr 27, 2011 at 2:36 PM, Eli Collins wrote:
> Hey Suresh,
>
> Do you plan to update the patch on HDFS-1052
Yes, I can talk about append as an example.
Some differences with federation project are:
- append had a comprehensive test plan document, which was designed an
executed;
- append was independently evaluated by HBase guys;
- it introduced new benchmark for append;
- We ran both DFSIO and NNThroughp
Owen,
The question is whether this is a
* Code Change,
which requires Lazy consensus of active committers or a
* Adoption of New Codebase,
which needs Lazy 2/3 majority of PMC members
Lazy consensus requires 3 binding +1 votes and no binding vetoes.
If I am looking at the current bylaws, then it
Suresh,
Showing no degradation in performance on one-node cluster is a good start
for benchmarking.
You still have a dev cluster to run benchmarks, don't you?
--Konstantin
On Wed, Apr 27, 2011 at 2:36 PM, suresh srinivas wrote:
> I ran these tests on my laptop. I would like to use this data to em
17 matches
Mail list logo
