Re: problem of lost name-node

2011-09-29 Thread Ravi Prakash
Hi, @Mirko: Please file a JIRA. This seems an appropriate time. @Steve: If we store the absolute filenames (i.e. the whole path), would we still have the problem you outlined in the 2nd para? I do agree updating would have to be pushed out and that might be cumbersome, but hey, we are processing

Re: problem of lost name-node

2011-09-28 Thread Steve Loughran
One of the issues here is keeping that list up to date. You don't want filename operations on the NN to push out changes to datanodes (which may not be there, after all), and you don't necessarily want every block creation operation on a DN to force an update on what effectively becomes a min

Re: problem of lost name-node

2011-09-28 Thread Robert Evans
There is also some work underway to add in HA and failover to the namenode. You might get more success if you send your note to hdfs-dev instead of common-dev. One other thing that can sometimes get a discussion going is to just file a JIRA for it. People interested in it are likely to start

Re: problem of lost name-node

2011-09-28 Thread Ravi Prakash
Hi Mirko, Its seems like a great idea to me!! The architects and senior developers might have some more insight on this though. I think part of the reason why the community might be lazy about implementing this is because the Namenode being a single point of failure is usually regarded as FUD. Th