Re: [DISCUSSION] Esoteric Spark function `TRIM/LTRIM/RTRIM`

2020-02-15 Thread Wenchen Fan
It's unfortunate that we don't have a clear document to talk about breaking changes (I'm working on it BTW). I believe the general guidance is: *avoid breaking changes unless we have to*. For example, the previous result was so broken that we have to fix it, moving to Scala 2.12 makes us have to br

Re: [DISCUSSION] Esoteric Spark function `TRIM/LTRIM/RTRIM`

2020-02-15 Thread Maxim Gekk
Also if we look at possible combination of trim parameters: 1. foldable srcStr + foldable trimStr 2. foldable srcStr + non-foldable trimStr 3. non-foldable srcStr + foldable trimStr 4. non-foldable srcStr + non-foldable trimStr The case # 2 seems a rare case, and # 3 is probably the most common ca

[DOCS] Spark SQL Upgrading Guide

2020-02-15 Thread Jacek Laskowski
Hi, Just noticed that http://spark.apache.org/docs/latest/sql-migration-guide-upgrade.html (Spark 2.4.5) has formatting issues in "Upgrading from Spark SQL 2.4.3 to 2.4.4" [1] which got fixed in master [2]. That's OK. What made me wonder was the other change to the section "Upgrading from Spark S