+ hdfs-dev for posterity
Thanks, Andrew. . I've been able to manually replicate one of the
under-replicated blocks by scp-ing the block file and its .meta file to
other datanodes and restarting them as you suggest. Once I get all the data
fully replicated I'll try to retrieve the information you'v
Hi Chris,
BCC'ing hdfs-dev@ since you're using CDH, moving us to cdh-user@.
You should be able to manually copy the under-replicated blocks and md5
files to a different datanode and restart it. I'm curious that you're
having this issue though, I haven't encountered it before. Can you send
your NN
I have only 9 under-replicated blocks on the cluster, and it is very
important that I restore my cluster to a fully-replicated state. Is there a
way I can manually copy these blocks to other datanodes, or perhaps new
datanodes?
On Thu, Jan 9, 2014 at 10:34 AM, Cooper Bethea wrote:
> Chris, Steve
Chris, Steve, thanks for responding.
Overnight I ran a script to bump replication, then lower it, as Chris
suggested. There has been no effect--all underreplicated blocks still have
only 1 replica.
Steve, I am running the rebalancer.
On Thu, Jan 9, 2014 at 1:33 AM, Steve Loughran wrote:
> are
are you running the rebalancer?
On 9 January 2014 04:40, Chris Embree wrote:
> It's too bad that this hasn't been corrected in HDFS 2.0 I have a
> script that I run several times a day to ensure that blocks are replicated
> correctly. Here a link to an article about it:
> http://dataforp
Hm, I had hoped this would have been fixed in hdfs 2. I have a script that
I run several times per day that identifies under replicated blocks and
increases the replication factor by 1. It then reduces the replication
factor back to normal.
I can dig up a link if you need it.
On Jan 8, 2014 9:00
It's too bad that this hasn't been corrected in HDFS 2.0 I have a
script that I run several times a day to ensure that blocks are replicated
correctly. Here a link to an article about it:
http://dataforprofit.com/?p=427
On Wed, Jan 8, 2014 at 9:00 PM, Cooper Bethea wrote:
> Following on--
Following on--is there a way that I can forcibly replicate these blocks,
perhaps by rsyncing the underlying files to other datanodes? As you might
imagine under-replicated data makes me very uneasy.
On Wed, Jan 8, 2014 at 12:00 PM, Cooper Bethea wrote:
> Hi HDFS developers,
>
> I have a worrying
Hi HDFS developers,
I have a worrying problem in a 2.0.0-cdh4.4.0 HDFS cluster I am running. 9
blocks in the cluster are persistently reported to be under-replicated per
"hdfs fsck".
I am able to fetch the files that contain these blocks, so I know that the
data is there, but for some reason repl
