[ANNOUNCE] Apache Spark 3.5.3 released

We are happy to announce the availability of Apache Spark 3.5.3! Spark 3.5.3 is the third maintenance release containing security and correctness fixes. This release is based on the branch-3.5 maintenance branch of Spark. We strongly recommend all 3.5 users to upgrade to this stable release. To d

Re: [ANNOUNCE] Apache Spark 3.5.3 released

Hi, I find download page does not contain 3.5.3 link, but release notes link exists. 3.5.3 release notes contains some commits that does not exists in tag/v3.5.3, e.g `[SPARK-49628]` exists in branch-3.5 but not in tag/v3.5.3, would you help explain more about that? Thank you On 2024/09/25 0

Re: [DISCUSS] [Spark SQL] Single-pass Analyzer SPIP

Let me add a bit more color since I'm the Shepherd. I've fixed quite some bugs in the analyzer due to rule order issues. The recent ones are https://github.com/apache/spark/pull/45718 and https://github.com/apache/spark/pull/45350 . Dealing with rule order is very tricky and making all the analyze

Re: [ANNOUNCE] Apache Spark 3.5.3 released

Hi, Yang. For this question, please try with `Private` mode or `Incognito` mode in your browser. > I find download page does not contain 3.5.3 link, but release notes link exists. Dongjoon. On Tue, Sep 24, 2024 at 8:39 PM Yang Zhang wrote: > Hi, > > I find download page does not contain 3.5.3

Re: [ANNOUNCE] Apache Spark 3.5.3 released

Thank you for the release, Heajoon. Could you publish docker images too like the following? - https://github.com/apache/spark-docker/pull/64 (Publish 3.5.2 to docker registry) Dongjoon. On Tue, Sep 24, 2024 at 10:29 PM Haejoon Lee wrote: > Hi, Yang! > > And thanks Dongjoon for answering th

Re: [ANNOUNCE] Apache Spark 3.5.3 released

Hi, Yang! And thanks Dongjoon for answering the question! For the second question, I got the commit list from the JIRA release note , but just realized that some commits are not resolved yet & missing from

Re: [DISCUSS][Spark SQL] Update API

All the existing DML APIs we support today have a source query so they all start with the source DataFrame, e.g. sourceDf.write.insertInto... sourceDf.write.saveAsTable... sourceDf.mergeInto... However, this is not the case for UPDATE and DELETE, as there is no source query. We need a different st