Re: Why is Spark 3.0.x faster than Spark 3.1.x

2021-05-18 Thread Maziyar Panahi
7:42, Rao, Abhishek (Nokia - IN/Bangalore) > wrote: > > Hi Maziyar, Mich > > Do we have any ticket to track this? Any idea if this is going to be fixed in > 3.1.2? > > Thanks and Regards, > Abhishek > > From: Mich Talebzadeh > Sent: Friday, April 9, 2021

Re: Why is Spark 3.0.x faster than Spark 3.1.x

2021-04-09 Thread Maziyar Panahi
this email's technical content is explicitly disclaimed. The > author will in no case be liable for any monetary damages arising from such > loss, damage or destruction. > > > > On Thu, 8 Apr 2021 at 16:19, Maziyar Panahi <mailto:maziyar.pan...@iscpif.fr>> wr

Re: Why is Spark 3.0.x faster than Spark 3.1.x

2021-04-08 Thread Maziyar Panahi
ying on this email's technical content is explicitly disclaimed. The > author will in no case be liable for any monetary damages arising from such > loss, damage or destruction. > > > > On Thu, 8 Apr 2021 at 15:05, Maziyar Panahi <mailto:maziyar.pan...@iscpif.fr>

Re: Why is Spark 3.0.x faster than Spark 3.1.x

2021-04-08 Thread Maziyar Panahi
Thanks Sean, I have already tried adding that and the result is absolutely the same. The reason that config cannot be the reason (at least not alone) it's because my comparison is between Spark 3.0.2 and Spark 3.1.1. This config has been set to true the beginning of 3.0.0 and hasn't changed: