Hello Everyone, I would like to log certain stats during iterations in a bulk iterative job. The way I do this is store the things I want at each iteration and plan to flush everything to HDFS once all the iterations are done. To do that I would need to know when the last iteration is invoked in order to flush the data. However, the close() method in the RichMapFunction is executed at the end of each iteration.
Is there anyway to know when I am in the last invocation? Or would you have a better suggestion to achieve what I am trying to do? Thank you and best regards, Tran Nam-Luc
