Spark Streaming - Small file in HDFS

Were you able to find a solution to your problem?

Spark Streaming - Small file in HDFS

Hi People, I'm using java kafka spark streaming and saving the result file into hdfs. As per my understanding, spark streaming write every processed message or event to hdfs file. Reason to creating one file per message or event could be to ensure fault tolerance. Is there any way spark handle th