Hi,

How do I read regular text files on HDFS from configure() in my
Mapper? I am doing the following and my jobs appear to fail randomly
(i.e., it works sometimes but mostly fails).

                        FileSystem fs = FileSystem.get(conf);
                        Path path = new Path("/path/to/file.txt");
                        FSDataInputStream fsin = fs.open(path);
                        LineRecordReader reader = new LineRecordReader(fsin, 0, 
4*1024, conf);
                        LongWritable key = new LongWritable();
                        Text value = new Text("");
                        while(reader.next(key, value)) {
                              String line = value.toString();
                              ...
                        }

Thanks,
Delip

Reply via email to