Re: Strategy Of Replica

To get the best performance from Hadoop, we can configure network topology. Based on that, it will apply RackAwareness algorithms and write/read the files. Also HDFS-2246 will improve performance by reading directly local replicas. If you have good algorithm and will get good performance than t

Re: Strategy Of Replica

On 2011/10/11 11:13, Uma Maheswara Rao G 72686 wrote: I did not get your proposed strategy implementations. Note that, already you can set the replication levels for files. If you set less replication, then obviously your perf and space will get benefits and also risk will be high in this case

Re: Strategy Of Replica

I did not get your proposed strategy implementations. Note that, already you can set the replication levels for files. If you set less replication, then obviously your perf and space will get benefits and also risk will be high in this case. I think we can manage your requirements using that re

Strategy Of Replica

Hi guys, What do you think of the strategy of replication in hdfs? How about the customized strategy that users customized their strategy of replication such as price, performance and so on? Thank you in advance.

Re: kfs and hdfs

On Mon, Oct 10, 2011 at 12:39 PM, Arun C Murthy wrote: > Agree with Cos completely. +1. Though our employers may spar on blog posts, as developers we still get along pretty well for the most part. Let's keep it that way :) Back to work. -Todd -- Todd Lipcon Software Engineer, Cloudera

RE: kfs and hdfs

Guys, I'm sorry. I really couldn't resist. :-) Yeah, Arun is right. I've given grief to a couple of vendors making claims about their products. It was just too easy and well... it's a Monday so I hope everyone took my post with its intended humor value. We now return you back to the original ra

Re: kfs and hdfs

Agree with Cos completely. As Apache Hadoop becomes more successful, we can expect more of these marketing messages from all sorts of vendors on Apache lists. Of course it's preaching to the choir when it comes to Mike - I've seen him make similar arguments against other vendors on various foru

Re: kfs and hdfs

Mike, I am not speaking for Owen in any way and appreciate your sense of humor, but the position he has expressed was (and hopefully remain) a stance of these lists: they aren't a marketing materials distribution channels. These are dev. mailing lists. Cos On Mon, Oct 10, 2011 at 01:59PM, Segel,

RE: kfs and hdfs

Owen, Are you still a bit touchy over Mike Olson's rebuttal to your blog? :-P (I kee-id, I kee-id) -Original Message- From: Owen O'Malley [mailto:o...@hortonworks.com] Sent: Monday, October 10, 2011 1:14 PM To: common-dev@hadoop.apache.org Subject: Re: kfs and hdfs Ted, Please keep th

Re: kfs and hdfs

Ted, Please keep the marketing of closed-source insecure filesystems such as MapR (and FAT32 *laugh*) off of the Apache Hadoop lists. -- Owen

[jira] [Created] (HADOOP-7732) 204 java docs missing hdfs package

204 java docs missing hdfs package -- Key: HADOOP-7732 URL: https://issues.apache.org/jira/browse/HADOOP-7732 Project: Hadoop Common Issue Type: Bug Components: documentation Affects Versions: 0.

Re: making file system block size bigger to improve hdfs performance ?

I can provide another data point here: xfs works very well in modern Linuxes (in the 2.6.9 era, it had many memory management headaches, especially around the switch to 4k stacks), and its advantage is significant when you run file systems over 95% occupied. Brian On Oct 10, 2011, at 8:51 AM,

Re: making file system block size bigger to improve hdfs performance ?

XFS was created in 1991 by Silicon Graphics. It was designed for streaming. The Linux port was in 2002 or so. I've used it extensively for the past 8 years. It is very stable, and many NAS companies have embedded it in their products. In particular, it works well even when the disk starts getting

Re: making file system block size bigger to improve hdfs performance ?

On 09/10/11 07:01, M. C. Srivas wrote: If you insist on HDFS, try using XFS underneath, it does a much better job than ext3 or ext4 for Hadoop in terms of how data is layed out on disk. But its memory footprint is alteast twice of that of ext3, so it will gobble up a lot more memory on your box.

Re: kfs and hdfs

On 09/10/11 08:57, Ted Dunning wrote: On Sun, Oct 9, 2011 at 12:33 AM, gschen wrote: what is the differences between hdfs and kfs(kosmos file system)? The biggest difference is that kfs is not very active (but not quite dead!) and hdfs has a pretty active development community. If you are

Build failed in Jenkins: Hadoop-Common-trunk #193

See Changes: [acmurthy] MAPREDUCE-3154. Fix JobSubmitter to check for output specs before copying job submission files to fail fast. Contributed by Abhijit Suresh Shingate. [acmurthy] MAPREDUCE-3123. Fix NM to quote symlink names