Hey Tarandeep, regarding the network part: not possible at the moment. It's pretty straight forward to add support for it, but no one ever got around to actually implementing it. If you would like to contribute, I am happy to give some hints about which parts of the system would need to be modified.
– Ufuk On Mon, Apr 18, 2016 at 12:56 PM, Tarandeep Singh <tarand...@gmail.com> wrote: > Hi, > > How can I set compression for AvroOutputFormat when writing files on HDFS? > Also, can we set compression for intermediate data that is sent over network > (from map to reduce phase) ? > > Thanks, > Tarandeep
