my installation notes for lzo-hadoop (might be wrong or incomplete): we run centos 5.6 and cdh3
yum -y install lzo git checkout https://github.com/toddlipcon/hadoop-lzo.git cd hadoop-lzo ant cd build cp hadoop-lzo-0.4.10/hadoop-lzo-0.4.10.jar /usr/lib/hadoop/lib cp -r hadoop-lzo-0.4.10/lib/native /usr/lib/hadoop/lib in core.site.xml: <property> <name>io.compression.codecs</name> <value>org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,com.hadoop.compression.lzo.LzoCodec,com.hadoop.compression.lzo.LzopCodec,org.apache.hadoop.io.compress.BZip2Codec</value> <final>true</final> </property> <property> <name>io.compression.codec.lzo.class</name> <value>com.hadoop.compression.lzo.LzoCodec</value> <final>true</final> </property> in mapred.site.xml: <property> <name>mapred.compress.map.output</name> <value>true</value> <final>false</final> </property> <property> <name>mapred.map.output.compression.codec</name> <value>com.hadoop.compression.lzo.LzoCodec</value> <final>false</final> </property> <property> <name>mapred.output.compress</name> <value>true</value> <final>false</final> </property> <property> <name>mapred.output.compression.codec</name> <value>com.hadoop.compression.lzo.LzoCodec</value> <final>false</final> </property> <property> <name>mapred.output.compression.type</name> <value>BLOCK</value> <final>false</final> </property> On Mon, Jul 25, 2011 at 7:14 PM, Alejandro Abdelnur <t...@cloudera.com>wrote: > Vikas, > > You should be able to use the Snappy codec doing some minor tweaks > from http://code.google.com/p/hadoop-snappy/ until a Hadoop releases > with Snappy support. > > Thxs. > > Alejandro. > > On Mon, Jul 25, 2011 at 4:04 AM, Vikas Srivastava > <vikas.srivast...@one97.net> wrote: > > Hey , > > > > i just want to use any compression in hadoop so i heard about lzo which > is > > best among all the compression (after snappy) > > > > please any1 tell me who is already using any kind of compression in > hadoop > > 0.20.2 > > > > > > > > -- > > With Regards > > Vikas Srivastava > > > > DWH & Analytics Team > > Mob:+91 9560885900 > > One97 | Let's get talking ! > > >
