problem of lost name-node

Hello, during the Cloudera Developer Training at Berlin I came up with an idea, regarding a lost name-node. As in this case all data blocks are lost, the solution could be, to have a table which relates *filenames *and *block_ids* on that data node a block is stored. This table can be scaned after

[jira] [Created] (HADOOP-7689) Process cannot exit when there is many RPC readers more that actual client

Process cannot exit when there is many RPC readers more that actual client -- Key: HADOOP-7689 URL: https://issues.apache.org/jira/browse/HADOOP-7689 Project: Hadoop Common

[jira] [Reopened] (HADOOP-7683) hdfs-site.xml template has properties that are not used in 20

[ https://issues.apache.org/jira/browse/HADOOP-7683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley reopened HADOOP-7683: Reopening until Harsh's comments are addressed. > hdfs-site.xml template has properti

Build failed in Jenkins: Hadoop-0.20.203-Build #18

See Changes: [shv] MAPREDUCE-3088. Turn off Clover 2.4.3 until upgraded. Contributed by Konstantin Shvachko. -- [...truncated 3969 lines...] [exec] checking for a sed that does not trun

Re: problem of lost name-node

Hi Mirko, Its seems like a great idea to me!! The architects and senior developers might have some more insight on this though. I think part of the reason why the community might be lazy about implementing this is because the Namenode being a single point of failure is usually regarded as FUD. Th

Re: problem of lost name-node

There is also some work underway to add in HA and failover to the namenode. You might get more success if you send your note to hdfs-dev instead of common-dev. One other thing that can sometimes get a discussion going is to just file a JIRA for it. People interested in it are likely to start

Re: problem of lost name-node

One of the issues here is keeping that list up to date. You don't want filename operations on the NN to push out changes to datanodes (which may not be there, after all), and you don't necessarily want every block creation operation on a DN to force an update on what effectively becomes a min

[jira] [Created] (HADOOP-7690) cleanup up use of properties dfs.namenode.http-address and dfs.namenode.https-address

cleanup up use of properties dfs.namenode.http-address and dfs.namenode.https-address - Key: HADOOP-7690 URL: https://issues.apache.org/jira/browse/HADOOP-7690 Projec

Re: configuring different number of slaves for MR jobs

This (slaves) is a configuration parameter (look at bin/hadoop-config.sh) which is set once you start mapreduce cluster (execute "start-mapred.sh"). You can change the slaves by executing "stop-mapred.sh", changing the slaves file and running "start-mapred.sh" between the jobs. You can also stop/st

[jira] [Resolved] (HADOOP-7683) hdfs-site.xml template has properties that are not used in 20

[ https://issues.apache.org/jira/browse/HADOOP-7683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley resolved HADOOP-7683. Resolution: Fixed Thanks, then we'll consider this one closed and finish the rest in HADOOP-7690

[jira] [Reopened] (HADOOP-7668) Add a NetUtils method that can tell if an InetAddress belongs to local host

[ https://issues.apache.org/jira/browse/HADOOP-7668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened HADOOP-7668: Assignee: Steve Loughran (was: Suresh Srinivas) deleting the two printlns

[jira] [Resolved] (HADOOP-7668) Add a NetUtils method that can tell if an InetAddress belongs to local host

[ https://issues.apache.org/jira/browse/HADOOP-7668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-7668. Resolution: Fixed Release Note: closing again > Add a NetUtils method that can t

[jira] [Created] (HADOOP-7691) hadoop deb pkg should take a diff group id

hadoop deb pkg should take a diff group id -- Key: HADOOP-7691 URL: https://issues.apache.org/jira/browse/HADOOP-7691 Project: Hadoop Common Issue Type: Bug Environment: ubuntu - 11.04

[jira] [Created] (HADOOP-7692) hadoop single node setup script to create mapred dir

hadoop single node setup script to create mapred dir Key: HADOOP-7692 URL: https://issues.apache.org/jira/browse/HADOOP-7692 Project: Hadoop Common Issue Type: Bug Affects Versions: 0.

[jira] [Created] (HADOOP-7693) fix RPC.Server#addProtocol to work in AvroRpcEngine

fix RPC.Server#addProtocol to work in AvroRpcEngine --- Key: HADOOP-7693 URL: https://issues.apache.org/jira/browse/HADOOP-7693 Project: Hadoop Common Issue Type: Improvement Componen

[jira] [Created] (HADOOP-7694) Calling toString() on an RPC proxy object backed by WritableRpcEngine fails

Calling toString() on an RPC proxy object backed by WritableRpcEngine fails --- Key: HADOOP-7694 URL: https://issues.apache.org/jira/browse/HADOOP-7694 Project: Hadoop Common

The configuration loading behavior

All - Can someone please help me to understand this (org.apache.hadoop.conf.Configuration, 20.204). The following code reads like an exception will be thrown if debug is on, no matter what, why? /** A new configuration where the behavior of reading from the default * resources can be turned

Re: The configuration loading behavior

Hello Ruby, It is just logging the trace of configuration object invocations. It will not throw exception. Regards, Uma - Original Message - From: Ruby Stevenson Date: Thursday, September 29, 2011 6:15 am Subject: The configuration loading behavior To: common-dev@hadoop.apache.org > A

[jira] [Created] (HADOOP-7695) RPC.stopProxy can throw unintended exception while logging error

RPC.stopProxy can throw unintended exception while logging error Key: HADOOP-7695 URL: https://issues.apache.org/jira/browse/HADOOP-7695 Project: Hadoop Common Issue Type: Bug

Re: The configuration loading behavior

hmm ... I tried a small test program, if I put log4j configuration in the classpath and turn on debug level, I am actually seeing the exception thrown. The seemingly silly issue I had have been bugging me for a while. DEBUG hadoop.conf.Configuration - java.io.IOException: config() at org.apac

Re: The configuration loading behavior

Ruby, Uma has already explained it isn't. Its just a 'technique' to print out a stack trace during DEBUG logging. It isn't an exception, and it isn't 'thrown' either - as you notice in the code. Creating an Exception instance doesn't 'throw' it. The line you see in logs is a proper, DEBUG-only

[jira] [Created] (HADOOP-7696) Add a note regarding the debug-log stack trace printing done in Configuration class

Add a note regarding the debug-log stack trace printing done in Configuration class --- Key: HADOOP-7696 URL: https://issues.apache.org/jira/browse/HADOOP-7696 Project: H

Re: The configuration loading behavior

FWIW, since this is the second time I saw this come up, I've opened https://issues.apache.org/jira/browse/HADOOP-7696 as a trivial JIRA to address it and avoid confusion :) The usefulness of that statement is that it will tell you, during debug logging, what piece of code instantiated Configuratio

[jira] [Resolved] (HADOOP-7696) Add a note regarding the debug-log stack trace printing done in Configuration class

[ https://issues.apache.org/jira/browse/HADOOP-7696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J resolved HADOOP-7696. - Resolution: Not A Problem My bad. No such statement remains in trunk, so not a problem anymore. For rec

[jira] [Created] (HADOOP-7697) Remove dependency on different version of slf4j in avro

Remove dependency on different version of slf4j in avro --- Key: HADOOP-7697 URL: https://issues.apache.org/jira/browse/HADOOP-7697 Project: Hadoop Common Issue Type: Bug Componen