[SQL] s.s.a.coalescePartitions.parallelismFirst true but recommends false

2021-09-03 Thread Jacek Laskowski
Hi, Found this new spark.sql.adaptive.coalescePartitions.parallelismFirst config property [1] with the default value `true` but the descriptions says the opposite: > It's recommended to set this config to false Is this OK and I'm misreading it? [1] https://github.com/apache/spark/blob/54cca7f82

[SQL] When SQLConf vals gets own accessor defs?

2021-09-03 Thread Jacek Laskowski
Hi, Just found something I'd consider an inconsistency in how SQLConf constants (vals) get their own accessor method for the current value (defs). I thought that internal config prop vals might not have defs (and that would make sense) but DYNAMIC_PARTITION_PRUNING_PRUNING_SIDE_EXTRA_FILTER_RATIO