Re: Avoiding unnnecessary sort in FileFormatWriter/DynamicPartitionDataWriter

2020-09-08 Thread Cheng Su
Thanks, Ximo. On our side, we do see the similar cases in production as well and we added this feature internally couple years ago. Let me submit new PR (which is mostly to rebase https://github.com/apache/spark/pull/23163 to latest master and try to have better code structure), if there’s no ob

Re: [VOTE] Release Spark 2.4.7 (RC3)

2020-09-08 Thread Mridul Muralidharan
+1 Signatures, digests, etc check out fine. Checked out tag and built/tested with -Pyarn -Phadoop-2.7 -Phive -Phive-thriftserver -Pmesos -Pkubernetes Thanks, Mridul On Tue, Sep 8, 2020 at 8:55 AM Prashant Sharma wrote: > Please vote on releasing the following candidate as Apache Spark > versi

Re: [VOTE] Release Spark 2.4.7 (RC3)

2020-09-08 Thread Sean Owen
+1 from as with the last RC. (This no no big deal, but https://repository.apache.org/content/repositories/orgapachespark-1361/ says it is not exposed.) I got a few failures in tests, but I think it is likely due to the system I'm running on. If nobody else sees these (and Jenkins seems OK) then I

[VOTE] Release Spark 2.4.7 (RC3)

2020-09-08 Thread Prashant Sharma
Please vote on releasing the following candidate as Apache Spark version 2.4.7. The vote is open until Sep 11th at 9AM PST and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 2.4.7 [ ] -1 Do not release this package because ...