Re:[VOTE] Release Apache Spark 3.4.0 (RC7)

2023-04-11 Thread beliefer
+1 At 2023-04-08 07:29:46, "Xinrong Meng" wrote: Please vote on releasing the following candidate(RC7) as Apache Spark version 3.4.0. The vote is open until 11:59pm Pacific time April 12th and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this pa

Re: [VOTE] Release Apache Spark 3.2.4 (RC1)

2023-04-11 Thread Jungtaek Lim
+1 (non-binding) Thanks for driving the release! On Wed, Apr 12, 2023 at 3:41 AM Xinrong Meng wrote: > +1 non-binding > > Thank you Doogjoon! > > Wenchen Fan 于2023年4月10日 周一下午11:32写道: > >> +1 >> >> On Tue, Apr 11, 2023 at 10:09 AM Hyukjin Kwon >> wrote: >> >>> +1 >>> >>> On Tue, 11 Apr 2023 at

Re: [VOTE] Release Apache Spark 3.4.0 (RC7)

2023-04-11 Thread Jungtaek Lim
+1 (non-binding) Thanks for driving the release! On Wed, Apr 12, 2023 at 10:42 AM Ye Zhou wrote: > +1 non-binding > > On Tue, Apr 11, 2023 at 18:40 Ye Zhou wrote: > >> Yes, it is not a regression issue. We can fix it after the release. >> >> Thanks >> Ye >> >> On Tue, Apr 11, 2023 at 17:42 Xia

Spark Multiple Hive Metastore Catalog Support

2023-04-11 Thread Ankit Gupta
Hi All The question is regarding the support of multiple Remote Hive Metastore catalogs with Spark. Starting Spark 3, multiple catalog support is added in spark, but have we implemented any CatalogPlugin that can help us configure multiple Remote Hive Metastore Catalogs ? If yes, can anyone help m

Re: [VOTE] Release Apache Spark 3.4.0 (RC7)

2023-04-11 Thread Ye Zhou
+1 non-binding On Tue, Apr 11, 2023 at 18:40 Ye Zhou wrote: > Yes, it is not a regression issue. We can fix it after the release. > > Thanks > Ye > > On Tue, Apr 11, 2023 at 17:42 Xiao Li wrote: > >> Thanks for testing it in your environment! >> >> >>> This is a minor issue itself, and only imp

Re: [VOTE] Release Apache Spark 3.4.0 (RC7)

2023-04-11 Thread Ye Zhou
Yes, it is not a regression issue. We can fix it after the release. Thanks Ye On Tue, Apr 11, 2023 at 17:42 Xiao Li wrote: > Thanks for testing it in your environment! > > >> This is a minor issue itself, and only impacts the metrics for push-based >> shuffle, but it will essentially completely

Re: [VOTE] Release Apache Spark 3.4.0 (RC7)

2023-04-11 Thread Xiao Li
Thanks for testing it in your environment! > This is a minor issue itself, and only impacts the metrics for push-based > shuffle, but it will essentially completely eliminate the effort > in SPARK-36620. Based on my understanding, this is not a regression. It only affects the new enhancements h

Re: [VOTE] Release Apache Spark 3.4.0 (RC7)

2023-04-11 Thread Ye Zhou
Manually tested the binary in our cluster. Started spark-shell application with some shuffle. Found one issue which is related to push based shuffle client side metrics introduced in https://github.com/apache/spark/pull/36165. Filed a ticket https://issues.apache.org/jira/browse/SPARK-43100, posted

Re: [SparkSQL, SparkUI, RESTAPI] How to extract the WholeStageCodeGen ids from SparkUI

2023-04-11 Thread Chitral Verma
try explain codegen on your DF and then pardee the string On Fri, 7 Apr, 2023, 3:53 pm Chenghao Lyu, wrote: > Hi, > > The detailed stage page shows the involved WholeStageCodegen Ids in its > DAG visualization from the Spark UI when running a SparkSQL. (e.g., under > the link > node:18088/histor

Re: [VOTE] Release Apache Spark 3.2.4 (RC1)

2023-04-11 Thread Xinrong Meng
+1 non-binding Thank you Doogjoon! Wenchen Fan 于2023年4月10日 周一下午11:32写道: > +1 > > On Tue, Apr 11, 2023 at 10:09 AM Hyukjin Kwon wrote: > >> +1 >> >> On Tue, 11 Apr 2023 at 11:04, Ruifeng Zheng wrote: >> >>> +1 (non-binding) >>> >>> Thank you for driving this release! >>> >>> ---

Re: [VOTE] Release Apache Spark 3.4.0 (RC7)

2023-04-11 Thread Peter Toth
+1 Jia Fan ezt írta (időpont: 2023. ápr. 11., K, 9:09): > +1 > > Wenchen Fan 于2023年4月11日周二 14:32写道: > >> +1 >> >> On Tue, Apr 11, 2023 at 9:57 AM Yuming Wang wrote: >> >>> +1. >>> >>> On Tue, Apr 11, 2023 at 9:14 AM Yikun Jiang wrote: >>> +1 (non-binding) Also ran the docker im

Re: [VOTE] Release Apache Spark 3.4.0 (RC7)

2023-04-11 Thread Jia Fan
+1 Wenchen Fan 于2023年4月11日周二 14:32写道: > +1 > > On Tue, Apr 11, 2023 at 9:57 AM Yuming Wang wrote: > >> +1. >> >> On Tue, Apr 11, 2023 at 9:14 AM Yikun Jiang wrote: >> >>> +1 (non-binding) >>> >>> Also ran the docker image related test (signatures/standalone/k8s) with >>> rc7: https://github.co