Re: Question using multiple partition for Window cumulative functions when partition is not specified.

Hi haejoon, I think you can check the discuss in https://github.com/apache/spark/pull/27861 Best regards Angers Haejoon Lee 于2021年8月30日周一 下午1:59写道： > Hi all, > > I noticed that Spark uses only one partition when performing Window > cumulative functions without specifying the partition, so all

Re: Question using multiple partition for Window cumulative functions when partition is not specified.

You just have 1 partition here because the input is so small. You can always repartition this further for parallelism. Is the issue that you're not partitioning the window itself, maybe? On Mon, Aug 30, 2021 at 12:59 AM Haejoon Lee wrote: > Hi all, > > I noticed that Spark uses only one partitio

Re: spark 3.2 release date

Hi, There is not exact release date now. As per https://spark.apache.org/release-process.html , we need a Release Candidate which passes the release vote. Spark 3.2 RC1 failed recently. I will cut RC2 after https://issues.apache.org/jira/browse/SP