d through? if the
> default is different in Hadoop 3 we could mention that in the docs. i
> don't know if we want to always set it to 1 as a Spark default, even
> in Hadoop 3 right?
>
> On Thu, Jun 25, 2020 at 2:43 PM Waleed Fateem
> wrote:
> >
> > Hello!
>
Hello!
I noticed that in the documentation starting with 2.2.0 it states that the
parameter spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version is 1
by default:
https://issues.apache.org/jira/browse/SPARK-20107
I don't actually see this being set anywhere explicitly in the Spark code
and
