Building Hadoop

2011-12-26 Thread Ronald Petty
Hello, Does anyone have notes on how to build Hadoop on a EC2 instance based on the trunk? I am trying to use http://wiki.apache.org/hadoop/HowToContribute. I used a micro ec2 instance and when I ran mvn compile, some test ran out of memory and the build failed. I then tried a large ec2 instan

mapreduce combiner

2011-12-26 Thread 27g
I have biuld a distribute index using the source code of hadoop/contrib/index,but I found that when the input files become big(such as one file is 16G),the OOM exception will be throwed .The cause is that: in combiner ,"writer.addIndexNoOptimize()",this use much memory cause to OOM, it's the Lucene