Re: persistent under-replicated blocks

2014-01-08 Thread Chris Embree
Hm, I had hoped this would have been fixed in hdfs 2. I have a script that I run several times per day that identifies under replicated blocks and increases the replication factor by 1. It then reduces the replication factor back to normal. I can dig up a link if you need it. On Jan 8, 2014 9:00

Re: persistent under-replicated blocks

2014-01-08 Thread Chris Embree
It's too bad that this hasn't been corrected in HDFS 2.0 I have a script that I run several times a day to ensure that blocks are replicated correctly. Here a link to an article about it: http://dataforprofit.com/?p=427 On Wed, Jan 8, 2014 at 9:00 PM, Cooper Bethea wrote: > Following on--

Re: Measuring bandwidth in 2.1.x

2013-09-13 Thread Chris Embree
vnstat should be able to do this. It has a "live" mode or can generate hourly reports much like sar. Google for vnstat to find it. On 9/13/13, hilfi alkaff wrote: > Hi, > > Is there any way that we could measure the actual bandwidth used by the > nodes when running MapReduce jobs (For example,

Re: Can I move block data directly?

2013-07-08 Thread Chris Embree
I know nothing. It seems that circumventing normal operations could be very bad. There was an example of something similar at hadoop summit. Some very experienced contributors decided they should edit meta data. they broke their cluster. Just say no ! ;) On Jul 8, 2013 9:01 PM, "Azuryy Yu"

Hurray for NN sanity! Even Pre 2.x!

2013-04-10 Thread Chris Embree
I work for a largish healthcare co. We finally started using (exploiting?) Hadoop this year. The day before our big, C-Suite sponsored launch of Hadoop celebration, we realized that we could no longer ssh to our NN. Fail of all fails! Sorta. Nothing was really wrong! NN was up, running, and

Re: Release numbering for branch-2 releases

2013-01-30 Thread Chris Embree
Hi Arun, et. al., I hope you don't mind a non-contributor butting in here. I'm currently a Hadoop administrator and former application developer (non-hadoop). regarding GA release changes, I think Arun has got a lot of good ideas here. I think it's better to add new features via new flags, para

[jira] [Created] (HDFS-4327) Update documentation to more accurately describe Replication Pipelining

2012-12-19 Thread Chris Embree (JIRA)
Chris Embree created HDFS-4327: -- Summary: Update documentation to more accurately describe Replication Pipelining Key: HDFS-4327 URL: https://issues.apache.org/jira/browse/HDFS-4327 Project: Hadoop HDFS