Hi,
How do I read regular text files on HDFS from configure() in my
Mapper? I am doing the following and my jobs appear to fail randomly
(i.e., it works sometimes but mostly fails).
FileSystem fs = FileSystem.get(conf);
Path path = new Path("/path/to/file.txt");
FSDataInputStream fsin = fs.open(path);
LineRecordReader reader = new LineRecordReader(fsin, 0,
4*1024, conf);
LongWritable key = new LongWritable();
Text value = new Text("");
while(reader.next(key, value)) {
String line = value.toString();
...
}
Thanks,
Delip