Hi everybody,
I developped an application using Hadoop. It runs perfectly with a
stand-alone mode but when I try to run it with the pseudo-distributed mode I
get an error "java.lang.OutOfMemoryError: GC overhead limit exceeded" (but
with the stand-alone mode I don't get this error. The application runs until
its end.). But the JVM is initialized with 2048MB (-Xmx2048m -Xms2048m).
Moveover I find that's slower than the stand-alone mode, is normal (1h20 to
process just 30% of the file whereas the file is 1MB size big and it must be
processed in 1 minute!!!)?
For information, the application is used to index a documents pool and it
uses Hadoop 0.20.2.
My config files are the above :
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>localhost:8021</value>
</property>
<property>
<name>mapred.child.java.opts</name>
<value>-Xmx2048m -Xms2048m</value>
</property>
</configuration>
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost/</value>
</property>
</configuration>
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
I start Hadoop as this :
./hadoop-0.20.2/bin/start-all.sh
And after I run my application as :
hadoop jar myApp.jar [my parameters...]
Could anybody help me, please?
Thanks.
--
View this message in context:
http://old.nabble.com/Hadoop%27s-slow-and-returns-a-OutOfMemoryError-in-pseudo-distributed-mode-tp31705364p31705364.html
Sent from the Hadoop core-dev mailing list archive at Nabble.com.
