A map-only job does not write to local disk, but rather directly to the configured FileSystem the job's running on (HDFS usually).
It has always been so as far as I know, as map-only jobs do not do any form of sorting/partitioning and hence do not require transient storage. On Thu, Apr 26, 2012 at 2:19 AM, Xun TANG <tangxun.al...@gmail.com> wrote: > Hi, > > We are running Hadoop jobs with mappers only, no reducers. > After each map computation, it writes its result (i.e. intermediate > results) to local disk. How fast is this speed, compared with write-speed > to HDFS? > We used TestFDSIO to do the benchmark to get write-speed to HDFS, but we > wonder how much the gap is. > > Another question is, when did Hadoop move this chuck to HDFS? > > Any thoughts, guys? Thanks ahead. > > Xun -- Harsh J
