Thanks for the replies. If I have fs.default.name = file://my_lustre_mount_point , then only the lustre filesystem will be used. I would like to have something like
fs.default.name=file://my_lustre_mount_point , hdfs://localhost:9123 so that both local filesystem and lustre are in use. Kindly correct me if I am missing something here. Thanks, Vikas On Thu, Jun 10, 2010 at 8:09 PM, Owen O'Malley <omal...@apache.org> wrote: > > Your local storage should get used for MR. Use Lustre via file:// > (LocalFileSystem, iirc) > > instead of HDFS via hdfs:// (DistributedFileSystem, irrc) as the default > file system type. > > If Lustre has integrated checksums, you'll want to use the > RawLocalFileSystem instead of LocalFileSystem. You'll want to make it > accessible via: > > fs.raw.impl = org.apache.hadoop.fs.RawLocalFileSystem > > so that urls like raw:///my/path won't go through the Hadoop checksum code. > > -- Owen >
