Re: Failures on DataSet programs

2016-09-28 Thread Ufuk Celebi
Hey Paulo! I think it's not possible out of the box at the moment, but you can try the following as a work around: 1) Create a custom OutputFormat that extends TextOutputFormat and override the clean up method: public class NoCleanupTextOutputFormat extends TextOutputFormat { @Override p

Failures on DataSet programs

2016-09-27 Thread Paulo Cezar
Hi Folks, I was wondering if it's possible to keep partial outputs from dataset programs. I have a batch pipeline that writes its output on HDFS using writeAsFormattedText. When it fails the output file is deleted but I would like to keep it so that I can generate new inputs for the pipeline to av