I am running 0.19.1-dev, r744282. I have searched the issues but
found nothing about the compression.
AFAIK, there are no open issues that prevent intermediate compression
from working. The following might be useful:
http://hadoop.apache.org/core/docs/current/mapred_tutorial.html#Data+Compression
Should the intermediate results not be compressed also if the map
output files are set to be compressed?
These are controlled by separate options.
FileOutputFormat::setCompressOutput enables/disables compression on
the final output
JobConf::setCompressMapOutput enables/disables compression of the
intermediate output
If not then why do we have the map compression option just to save
network traffic?
That's part of it. Also to save on disk bandwidth and intermediate
space. -C
