Hello, I have set my HADOOP_CONF_DIR to the conf folder and still not loading. I have to manually set the options when I create my conf. Have you resolved this?
Regards Saptarshi On Tue, Dec 30, 2008 at 5:25 PM, g00dn3ss <[email protected]> wrote: > Hey all, > > I have a similar issue. I am specifically having problems with the config > option "mapred.child.java.opts." I set it to -Xmx1024m and it uses -Xmx200m > regardless. I am running Hadoop 0.18.2 and I'm pretty sure this option was > working in the previous versions of Hadoop I was using. > > I am not explicitly setting HADOOP_CONF_DIR. My site config is in > ${HADOOP_HOME}/conf. Just to test things further, I wrote a small map task > to print out the ENV values and it has the correct value for HADOOP_HOME, > HADOOP_LOG_DIR, HADOOP_OPTS, etc... I also printed out the key/values in > the JobConf passed to the mapper and it has my specified values for > fs.default.name and mapred.job.tracker. Other settings like dfs.name.dir, > dfs.data.dir, and mapred.child.java.opts do not have my values. > > Any suggestion where to look at next? > > Thanks! > > > > On Mon, Dec 29, 2008 at 10:27 PM, Amareshwari Sriramadasu < > [email protected]> wrote: > >> Saptarshi Guha wrote: >> >>> Hello, >>> I had previously emailed regarding heap size issue and have discovered >>> that the hadoop-site.xml is not loading completely, i.e >>> Configuration defaults = new Configuration(); >>> JobConf jobConf = new JobConf(defaults, XYZ.class); >>> System.out.println("1:"+jobConf.get("mapred.child.java.opts")); >>> System.out.println("2:"+jobConf.get("mapred.map.tasks")); >>> System.out.println("3:"+jobConf.get("mapred.reduce.tasks")); >>> >>> >>> System.out.println("3:"+jobConf.get("mapred.tasktracker.reduce.tasks.maximum")); >>> >>> returns -Xmx200m, 2,1,2 respectively, even though the numbers in the >>> hadoop-site.xml are very different. >>> >>> Is there a way for hadoop to dump the parameters read in from >>> hadoop-site.xml and hadoop-default.xml? >>> >>> >>> >> Is your hadoop-site.xml present in the conf (HADOOP_CONF_DIR) directory? >> http://hadoop.apache.org/core/docs/r0.19.0/cluster_setup.html#Configuration >> >> -Amareshwari >> > -- Saptarshi Guha - [email protected]
