Error while indexing the LZO file

[Ankit Jain]

Hi all,
I tried to index the lzo file but got the following error while indexing the
lzo file :

java.lang.ClassCastException:
com.hadoop.compression.lzo.LzopCodec$LzopDecompressor cannot be cast to
com.hadoop.compression.lzo.LzopDecompressor
    at
com.hadoop.mapreduce.LzoSplitRecordReader.initialize(LzoSplitRecordReader.java:52)
    at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:418)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:620)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
    at org.apache.hadoop.mapred.Child.main(Child.java:170)


I have performed following steps:-

1. $sudo apt-get install liblzo2-dev
2. Download the Hadoop-lzo clone from github repo (
https://github.com/kevinweil/hadoop-lzo )
3. Build the Hadoop-lzo project.
4. Copy the hadoop-lzo-*.jar file at $HADOOP_HOME/lib dir of cluster nodes
5. Copy the hadoop-lzo-install-dir/build/hadoop-lzo-*/native library at
$HADOOP_HOME/lib dir of cluster nodes.
6. core-site.xml:

            <property>
            <name>io.compression.codecs</name>
        <value>org.apache.hadoop.io.compress.GzipCodec
            ,org.apache.hadoop.io.compress.DefaultCodec,
            com.hadoop.compression.lzo.LzoCodec,
            com.hadoop.compression.lzo.LzopCodec,
            org.apache.hadoop.io.compress.BZip2Codec
        </value>
      </property>
      <property>
            <name>io.compression.codec.lzo.class</name>
            <value>com.hadoop.compression.lzo.LzoCodec</value>
      </property>

7. mapred-site.xml:

    <property>
                    <name>mapred.child.env</name>

<value>JAVA_LIBRARY_PATH=/opt/ladap/common/hadoop-0.20.2/lib/native/Linux-i386-32/*</value>

            </property>

          <property>
                <name>mapred.map.output.compression.codec</name>
                <value>com.hadoop.compression.lzo.LzoCodec</value>
          </property>

8. hadoop-env.sh

export
HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/home/ankit/hadoop-0.20.1/lib/hadoop-lzo-0.4.12.jar
export JAVA_LIBRARY_PATH=/home/ankit/hadoop-0.20.1/lib/native/Linux-i386-32/

9. Restart the cluster

10. uploaded lzo file into hdfs

11. Runned the following command for indexing:

bin/hadoop jar path/to/hadoop-lzo-*.jar
com.hadoop.compression.lzo.LzoIndexer lzofile.lzo