Rebalancing data across partitions on a datanode.

2010-08-25 Thread Doug Balog
We've just added a couple of new drives to our datanodes. Each new drive has a single filesystem which we added to dfs.data.dir, and mapred.{local,tmp}.dir. Now I want to rebalance the data across the new filesystems so that they are equally utilized. My plan is to write a script that does the

Question about hadoop namenode -format -clusterid

2011-05-11 Thread Doug Balog
I'm at the hackathon in SF just trying to setup a single node cluster from my trunk checkout. I'm at the point where I need to format a new namenode, and the old way of just running "hadoop namenode -format" is failing because I'm not specifying a clusterID. So I started poking around the code