Re: [VOTE] Release Spark 4.1.0-preview1 (RC1)

+1 On 2025/07/11 04:23:27 Ángel Álvarez Pascua wrote: > +1 (non-binding) > > El jue, 10 jul 2025, 21:07, Jules Damji escribió: > > > +1 (non-binding) > > — > > Sent from my iPhone > > Pardon the dumb thumb typos :) > > > > On Jul 10, 2025, at 8:04 AM, Peter Toth wrote: > > > >  > > +1 > > > >

Re: [VOTE] SPIP: Monthly preview release

+1 On 2025/07/03 11:03:48 Mark Hamstra wrote: > +0 I'm not sure that it is a good idea, but I'm also not certain that > it isn't an idea worth trying. > > On Wed, Jul 2, 2025 at 9:37 PM Hyukjin Kwon wrote: > > > > Hi all, > > > > I would like to start a vote on the monthly preview releases. > >

Re: [VOTE] Release Apache Spark K8s Operator 0.4.0 (RC1)

+1 On 2025/07/01 00:35:56 Wenchen Fan wrote: > +1 > > On Tue, Jul 1, 2025 at 6:23 AM L. C. Hsieh wrote: > > > +1 > > > > Thanks Dongjoon. > > > > On Mon, Jun 30, 2025 at 8:52 AM Dongjoon Hyun > > wrote: > > > > > > Please vote on releasing the following candidate as Apache Spark K8s > > Operat

Re: [DISCUSS] Dropping LevelDB support in Spark

> >> > > >> > Thanks for initiating this. > >> > > >> > I wonder if we don't have any compatibility issue on every component - > >> SS area does not have an issue, but I don't quite remember if the history > >> server would

Re: [DISCUSS] Automation of RC email

Option 1 +1, thank you, Hyukjin, for the efforts you've put into this. On 2025/06/06 02:59:32 Jungtaek Lim wrote: > Thanks for the confirmation. That sounds great as long as the ASF account > information is required per run and never be stored somewhere after the run. > > 2025년 6월 6일 (금) 오전 11:39

Re: [VOTE] Release Apache Spark Connect Swift Client 0.3.0 (RC1)

+1 On 2025/06/01 08:09:38 Peter Toth wrote: > +1 > > On Sun, Jun 1, 2025 at 9:00 AM L. C. Hsieh wrote: > > > +1 > > > > On Sat, May 31, 2025 at 9:18 PM Dongjoon Hyun > > wrote: > > > > > > Please vote on releasing the following candidate as Apache Spark Connect > > Swift Client 0.3.0. This is

Re: [VOTE] Release Apache Spark K8s Operator 0.3.0 (RC1)

+1 On 2025/06/01 08:10:10 Peter Toth wrote: > +1 > > On Sun, Jun 1, 2025 at 9:01 AM L. C. Hsieh wrote: > > > +1 > > > > On Sat, May 31, 2025 at 10:38 PM Dongjoon Hyun > > wrote: > > > > > > Please vote on releasing the following candidate as Apache Spark K8s > > Operator 0.3.0. This is the fir

Re: [DISCUSS] SPIP: Real-Time Mode in Apache Spark Structured Streaming

+1 On 2025/05/29 16:25:19 Xiao Li wrote: > +1 > > Yuming Wang 于2025年5月29日周四 02:22写道： > > > +1. > > > > On Thu, May 29, 2025 at 3:36 PM DB Tsai wrote: > > > >> +1 > >> Sent from my iPhone > >> > >> On May 29, 2025, at 12:15 AM, John Zhuge wrote: > >> > >>  > >> +1 Nice feature > >> > >> On We

Re: [DISCUSS] Dropping LevelDB support in Spark

The project "org.fusesource.leveldbjni:leveldbjni" released its last version 12 years ago, and its code repository was last updated 8 years ago. Consequently, I believe it's challenging for us to receive ongoing maintenance and support from this project. On the flip side, when developers implem

Re: [VOTE] Release Spark 3.5.6 (RC1)

+1 On 2025/05/26 01:10:23 Hyukjin Kwon wrote: > The key issue was fixed. > > On Mon, 26 May 2025 at 10:05, Hyukjin Kwon wrote: > > > Probably should avoid backporting it for improvements but If there is a > > CVE that directly affects Spark, let's upgrade. > > > > On Mon, 26 May 2025 at 00:27,

Re: [VOTE] New Spark Connect Client Repository for Rust

+1 On 2025/05/20 07:14:30 Wenchen Fan wrote: > +1 > > On Tue, May 20, 2025 at 2:34 PM Jungtaek Lim > wrote: > > > Sounds great, thanks for the clarification. > > > > +1 (non-binding) > > > > On Tue, May 20, 2025 at 3:20 PM Renjie Liu > > wrote: > > > >> Hi, Jungtaek, Yuanjian: > >> > >> Yes, I

Re: [VOTE] Release Spark 4.0.0 (RC7)

+1 On 2025/05/20 04:54:50 Yuming Wang wrote: > +1 > > On Tue, May 20, 2025 at 12:19 PM Szehon Ho wrote: > > > +1 (non-binding) > > > > Checked signature, checksum, ran basic tests on spark-4.0.0-bin-hadoop3 > > Thanks > > Szehon > > > > > > > > On Mon, May 19, 2025 at 9:07 PM Denny Lee wrote:

Re: [DISCUSS] Apache Spark 3.5.6 (automated release)

+1, It would be truly awesome if we could use GitHub Actions for release. On 2025/05/20 05:08:16 Wenchen Fan wrote: > +1, look forward to Spark's first automated release! > > On Tue, May 20, 2025 at 12:55 PM Yuming Wang wrote: > > > +1. > > > > On Tue, May 20, 2025 at 12:51 PM Hyukjin Kwon > >

Re: [VOTE] Release Apache Spark K8s Operator 0.2.0 (RC1)

+1 On 2025/05/19 04:53:52 Ruifeng Zheng wrote: > +1 > > On Mon, May 19, 2025 at 12:34 PM Wenchen Fan wrote: > > > +1 > > > > On Mon, May 19, 2025 at 7:07 AM Liu Cao wrote: > > > >> +1 (non-binding) > >> > >> On Mon, May 19, 2025 at 03:07 Rozov, Vlad > >> wrote: > >> > >>> +1 (non-binding) > >

Re: [VOTE] Release Apache Spark Connect Swift Client 0.2.0 (RC1)

+1 On 2025/05/19 04:53:38 Ruifeng Zheng wrote: > +1 > > On Mon, May 19, 2025 at 12:34 PM Wenchen Fan wrote: > > > +1 > > > > On Mon, May 19, 2025 at 4:48 AM Jules Damji wrote: > > > >> + 1 (non-binding) > >> — > >> Sent from my iPhone > >> Pardon the dumb thumb typos :) > >> > >> > >> — > >>

Re: [VOTE] Release Apache Spark Connect Swift Client 0.2.0 (RC1)

+1 On 2025/05/19 04:53:38 Ruifeng Zheng wrote: > +1 > > On Mon, May 19, 2025 at 12:34 PM Wenchen Fan wrote: > > > +1 > > > > On Mon, May 19, 2025 at 4:48 AM Jules Damji wrote: > > > >> + 1 (non-binding) > >> — > >> Sent from my iPhone > >> Pardon the dumb thumb typos :) > >> > >> > >> — > >>

Re: [VOTE] Release Spark 4.0.0 (RC6)

+1 On 2025/05/14 00:21:11 Ruifeng Zheng wrote: > +1 > > On Wed, May 14, 2025 at 7:01 AM Gengliang Wang wrote: > > > +1 > > > > On Tue, May 13, 2025 at 3:57 PM Hyukjin Kwon wrote: > > > >> +1 > >> > >> On Wed, 14 May 2025 at 07:29, Wenchen Fan wrote: > >> > >>> Same as before, I'll start with

Re: [VOTE] Release Spark 4.0.0 (RC5)

+1, thank you Wenchen On 2025/05/13 02:11:02 "Rozov, Vlad" wrote: > +1 (non-binding) > > Thank you, > > Vlad > > On May 12, 2025, at 5:44 PM, huaxin gao wrote: > > +1 > > On Mon, May 12, 2025 at 5:34 PM Hyukjin Kwon > mailto:gurwls...@apache.org>> wrote: > +1 > > On Tue, 13 May 2025 at 03:

Re: [VOTE] Release Apache Spark Connect Swift Client 0.1.0 (RC1)

+1, A big thank you to Dongjoon for all the hard work you've put into this! On 2025/05/05 18:19:33 DB Tsai wrote: > +1, it’s exciting to see Spark Connect Swift client, showcasing Spark Connect > as a truly language-agnostic protocol, and also powering Swift users to use > Spark! > > > > Sent

Re: [VOTE] Release Apache Spark K8s Operator 0.1.0 (RC1)

+1, Thanks Dongjoon On 2025/05/06 02:45:40 Kent Yao wrote: > +1, Thank you Dongjoon. > > Kent Yao > > Hyukjin Kwon 于2025年5月6日周二 10:31写道： > > > +1 > > > > On Tue, 6 May 2025 at 10:55, Ruifeng Zheng wrote: > > > >> +1 > >> > >> On Tue, May 6, 2025 at 9:00 AM Wenchen Fan wrote: > >> > >>> +1, t

Re: [VOTE] SPIP: Declarative Pipelines

+1 (binding) On 2025/04/10 02:20:02 Cheng Pan wrote: > +1 (non-binding) > > Thanks, > Cheng Pan > > > > > On Apr 9, 2025, at 22:22, Sandy Ryza wrote: > > > > We started to get some votes on the discussion thread, so I'd like to move > > to a formal vote on adding support for declarative pip

Re: [DISCUSS] SPIP: Declarative Pipelines

+1 On 2025/04/09 01:07:57 Hyukjin Kwon wrote: > +1 > > I am actually pretty excited to have this. Happy to see this being proposed. > > On Wed, 9 Apr 2025 at 01:55, Chao Sun wrote: > > > +1. Super excited about this effort! > > > > On Tue, Apr 8, 2025 at 9:47 AM huaxin gao wrote: > > > >> +1

Re: [VOTE] Release Spark 4.0.0 (RC3)

-1, The pull request at https://github.com/apache/spark/pull/49604 introduced a connection-related example module and successfully merged it into branch-4.0, which is a commendable achievement. However, it is recommended that this example module should rely on the snapshot version of its cor

Re: [VOTE] Retain migration logic of incorrect `spark.databricks.*` configuration in Spark 4.0.x

> Adam Binford 于2025年3月11日周二 09:08写道： > >>> > >>>> +1 (non-binding) > >>>> > >>>> It's a pretty in the weeds issue with how Structured Streaming works > >>>> under the hood that's kinda hard to understand if you&#

Re: [VOTE] Retain migration logic of incorrect `spark.databricks.*` configuration in Spark 4.0.x

like people are not quite understanding the change, and objecting > >> to something that doesn't exist. > >> It's a shame, as this seems like something not even worth discussing. I > >> don't know why this triggered this much discussion. We have kept depre

Re: [VOTE] Retain migration logic of incorrect `spark.databricks.*` configuration in Spark 4.0.x

-1 Remove migration logic of incorrect `spark.databricks.*` configuration in Spark 4.0.0 because I think this configuration was initially introduced accidentally in Spark 3.5.4, lacking a clear design intent. Although the immediate maintenance cost of retaining this configuration currently seems

Re: [DISCUSS] New Spark Connect Client repository for Swift language

Great! Really happy to see that spark-connect supports more programming languages. On 2025/03/10 07:00:32 Martin Grund wrote: > Great work and proposal! > > I'm supportive. > > On Sun, Mar 9, 2025 at 23:31 Dongjoon Hyun wrote: > > > Hi, All. > > > > I'd like to propose to add a new Apache Sp

Re: [VOTE] Release Spark 4.0.0 (RC2)

-1, I think SPARK-49489 should be fixed before the 4.0 release, because it addresses the regression caused by an unreleased change (SPARK-47018). https://issues.apache.org/jira/browse/SPARK-49489 Thanks, Jie Yang On 2025/03/01 07:25:06 Wenchen Fan wrote: > Please vote on releasing the following

Re: [DISCUSS] SPARK-51318: Remove `jar` files from Apache Spark repository and disable affected tests

+1， Agree to remove the jar files from the Apache Spark repository and disable the affected tests. For the current test scenarios that use jar files, I believe we can definitely find a more reasonable testing approach. Thanks, Jie Yang On 2025/02/26 16:57:45 "Rozov, Vlad" wrote: > +1 on fixi

Re: [VOTE] SPIP: Add the TIME data type

+1 On 2025/02/24 05:30:34 Jungtaek Lim wrote: > +1 (non-binding) > > On Mon, Feb 24, 2025 at 2:26 PM Gengliang Wang wrote: > > > +1 > > > > On Sun, Feb 23, 2025 at 8:40 PM Hyukjin Kwon wrote: > > > >> +1 > >> > >> On Mon, 24 Feb 2025 at 11:47, Wenchen Fan wrote: > >> > >>> +1 > >>> > >>> On M

Re: [VOTE] Release Spark 3.5.5 (RC1)

+1 On 2025/02/24 04:04:22 Dongjoon Hyun wrote: > Thank you for your voting. > > I have been aware of SPARK-51281 since Wenchen pinged me three days ago. > > I just thought it was not going to be ready because the PR was idle for the > last two days after my comment. > > Since SPARK-51281 is not

Re: [VOTE] Release Apache Spark 3.5.5 deprecating `spark.databricks.*` configuration

+1 On 2025/02/19 05:57:53 Mark Hamstra wrote: > +1 > > On Tue, Feb 18, 2025 at 9:46 PM dongjoon.hyun wrote: > > > > Please vote to deprecate `spark.databricks.*` configuration at Apache Spark > > 3.5.5. > > This is a part of the following on-going discussion. > > > > - DISCUSSION: > > https://

Re: [VOTE] Publish additional Spark distribution with Spark Connect enabled

+1 Jie Yang On 2025/02/05 07:38:08 Sakthi wrote: > +1 (non-binding) > > On Tue, Feb 4, 2025 at 11:25 PM DB Tsai wrote: > > > +1 > > > > DB Tsai | https://www.dbtsai.com/ | PGP 42E5B25A8F7A82C1 > > > > On Feb 4, 2025, at 11:13 PM, Xiao Li wrote: > > > > +1 > > > > Xiao > > > > Jungtaek Lim

Re: Quick question: can you guys navigate 3.5.4 Java API documentation?

ide too? > > Seems like the issue does not exist in the development documentation FWIW. > > Would be great to fix it if this happens to you too. > > cc @Pan Bingkun @Yang Jie FYI > > [image: Screenshot 2025-02-05 at 2.38.20 PM.png] > - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: FYI: SPARK-49700 Unified Scala Interface for Connect and Classic

I reported a test issue that is suspected to be related to this pr： - https://github.com/apache/spark/pull/48818/files#r1929652392 and it seems to be causing the failure of the Maven daily test. Thanks, Jie Yang On 2025/01/24 20:24:57 Dongjoon Hyun wrote: > Hi, All. > > SPARK-49700 landed one

Re: [ANNOUNCE] Apache Spark 3.5.4 released

Thanks Vlad and Dongjoon On 2025/01/14 22:47:55 Dongjoon Hyun wrote: > Thank you for reporting. > > Now, I've synced it. > > FROM: > https://dist.apache.org/repos/dist/dev/spark/KEYS > > TO: > https://dist.apache.org/repos/dist/release/spark/KEYS > https://downloads.apache.org/spark/KEYS > > D

Re: [VOTE] Use plain text logs by default

+1 On 2025/01/09 06:01:01 Yuming Wang wrote: > +1. > > On Thu, Jan 9, 2025 at 12:52 PM beliefer wrote: > > > +1 > > > > > > I think there are some learning curves when using it. Users must provide a > > good, visual, and easily readable JSON rendering tool themselves before > > they can turn on

Re: [VOTE][RESULT] Release Spark 3.5.4 (RC3)

Thank you very much for your guidance and assistance throughout this process. @Dongjoon On 2024/12/20 15:10:32 Dongjoon Hyun wrote: > Thank you so much for driving all 3.5.4 RC votes successfully, Jie! > > Dongjoon. > > On Fri, Dec 20, 2024 at 2:23 AM 杨杰 wrote: > > > The vote[1] passes with 1

Re: [VOTE] Release Spark 3.5.4 (RC3)

Thank you all. This vote passed. Let me conclude this vote. Jie Yang On 2024/12/17 09:23:46 杨杰 wrote: > Please vote on releasing the following candidate as Apache Spark version > 3.5.4. > > The vote is open until Dec 20, 10:00:00 UTC and passes if a majority +1 PMC > votes are cast, with > a min

Re: [VOTE] Release Spark 3.5.4 (RC3)

Thank you all. This vote passed. Let me conclude this vote. Jie Yang On 2024/12/20 02:36:51 roryqi wrote: > +1， Thanks > > huaxin gao 于2024年12月20日周五 05:16写道： > > > +1 > > > > On Thu, Dec 19, 2024 at 1:02 PM Rozov, Vlad > > wrote: > > > >> +1 (not binding) > >> > >> Thanks > >> > >> Vlad Rozov

Re: [VOTE] Release Spark 3.5.4 (RC2)

Thank you all for your votes, but I'm very sorry to inform you that due to an issue discovered in the rc2 release script, I have decided to abandon RC2. I will prepare RC3 as soon as possible. Thanks, Jie Yang On 2024/12/16 16:16:03 Dongjoon Hyun wrote: > +1 > > Thank you again. > > Dongjoo

Re: [DISCUSS] Use plain text logs by default

+1 On 2024/12/11 02:34:02 Kent Yao wrote: > +1 > > On 2024/11/23 02:50:36 Wenchen Fan wrote: > > Hi Martin, > > > > Yea, we should be more deliberate about when to use Structured Logging. Let > > me start with when people prefer plain text logs: > > - Spark engine developers like us. When runnin

Re: [VOTE] Release Spark 3.5.4 (RC1)

Thank you everyone for your votes. But I have found that branch-3.5 is missing the fix for the issue introduced in https://github.com/apache/spark/pull/48767, which is addressed by https://github.com/apache/spark/pull/49021，and this has caused a regression on branch-3.5. So I suggest abandoni

Re: [DISCUSS] Release Apache Spark 3.5.4

Thank you again for all the positive feedback ~ On 2024/12/06 06:14:02 Jungtaek Lim wrote: > +1 for 3.5.4. I was thinking of doing this as I've fixed a couple blockers > in this branch, but was very busy and forgot about this. Thanks for > volunteering! > > On Fri, Dec 6,

Re: [DISCUSS] Release Apache Spark 3.5.4

Thank you everyone for the positive feedback. A special thanks to Dongjoon for offering to help. And I plan to release the first release candidate (RC1) next Monday, December 9th, 2024. Bests, Jie Yang On 2024/12/05 16:34:01 Dongjoon Hyun wrote: > Thank you for sharing, Terry. > > I merged

答复: [外部邮件] Re: Spark Connect the default API in Spark 4.0

+1 原始邮件 发件人：Hyukjin Kwon 时间：2024-11-27 08:04:06 主题：[外部邮件] Re： Spark Connect the default API in Spark 4.0 收件人：Bjørn Jørgensen; 抄送人：Herman van Hovell;Spark dev list; +1 On Mon, 25 Nov 2024 at 23:33, Bjørn Jørgensen mailto:bjornjorgen...@gmail.com>> wrote: +1 man. 25. nov. 2024 k

Re: [外部邮件] Re: New committer: Bingkun Pan

Congratulations, Bingkun ~ 发件人: Hyukjin Kwon 日期: 2024年11月17日 星期日 14:05 收件人: Jungtaek Lim 抄送: Ruifeng Zheng , Dongjoon Hyun , dev 主题: [外部邮件] Re: New committer: Bingkun Pan Congrats!! On Sat, Nov 16, 2024 at 11:46 AM Jungtaek Lim mailto:kabhwan.opensou...@gmail.com>> wrote: Congrats Bingkun!

Re: [VOTE] Release Spark 3.4.4 (RC1)

+1, thank you Dongjoon ~ Jie Yang On 2024/10/22 03:10:46 Kent Yao wrote: > +1, thank you Dongjoon! > > Kent Yao > > L. C. Hsieh 于2024年10月22日周二 10:21写道： > > > > +1 > > > > Thanks Dongjoon! > > > > On Mon, Oct 21, 2024 at 5:45 PM Xinrong Meng wrote: > > > > > > +1 > > > > > > Thank you Dongjoon

Re: [VOTE] Single-pass Analyzer for Catalyst

+1, Thanks Jie Yang On 2024/10/01 03:26:40 John Zhuge wrote: > +1 (non-binding) > > On Mon, Sep 30, 2024 at 7:42 PM Gengliang Wang > wrote: > > > +1 > > > > On Mon, Sep 30, 2024 at 6:22 PM Jungtaek Lim > > wrote: > > > >> +1 (non-binding), promising proposal! > >> > >> 2024년 10월 1일 (화) 오전 8:0

Re: [VOTE] Single-pass Analyzer for Catalyst

+1, Thanks Jie Yang On 2024/10/01 03:26:40 John Zhuge wrote: > +1 (non-binding) > > On Mon, Sep 30, 2024 at 7:42 PM Gengliang Wang > wrote: > > > +1 > > > > On Mon, Sep 30, 2024 at 6:22 PM Jungtaek Lim > > wrote: > > > >> +1 (non-binding), promising proposal! > >> > >> 2024년 10월 1일 (화) 오전 8:0

Re: [VOTE] Officialy Deprecate GraphX in Spark 4

+1, Thanks Jie Yang On 2024/10/01 02:53:11 Mridul Muralidharan wrote: > +1. > > Regards, > Mridul > > PS: In the past, I did have fun with graphx ... unfortunate that it has > come to this :-( > > > On Mon, Sep 30, 2024 at 6:23 PM Sean Owen wrote: > > > For reasons in the previous thread, y

Re: [VOTE] Release Spark 4.0.0-preview2 (RC1)

+1 On 2024/09/19 23:33:39 Shubham Patel wrote: > +1 > > On Thu, 19 Sept 2024, 01:05 Gengliang Wang, wrote: > > > +1 > > > > On Wed, Sep 18, 2024 at 10:47 PM Xiao Li wrote: > > > >> +1 > >> > >> On Wed, Sep 18, 2024 at 18:22 Yuming Wang wrote: > >> > >>> +1 > >>> > >>> On Wed, Sep 18, 2024 at

Re: [VOTE] Deprecate SparkR

+1 On 2024/08/23 00:19:40 Chao Sun wrote: > +1 > > On Thu, Aug 22, 2024 at 5:19 PM Yuming Wang wrote: > > > +1 > > > > On Fri, Aug 23, 2024 at 5:42 AM Allison Wang > > wrote: > > > >> +1 > >> > >> On Thu, Aug 22, 2024 at 10:10 AM Zhou Jiang > >> wrote: > >> > >>> +1 (non-binding) > >>> > >>>

Re: [ANNOUNCE] Apache Spark 3.5.1 released

hmm... I guess this is meant to cc @Bingkun Pan ? On 2024/03/05 02:16:12 Hyukjin Kwon wrote: > Is this related to https://github.com/apache/spark/pull/42428? > > cc @Yang,Jie(INF) > > On Mon, 4 Mar 2024 at 22:21, Jungtaek Lim > wrote: > > > Shall we revisit this

Re: [VOTE] Release Spark 3.3.4 (RC1)

+1 On 2023/12/11 03:03:39 "L. C. Hsieh" wrote: > +1 > > On Sun, Dec 10, 2023 at 6:15 PM Kent Yao wrote: > > > > +1(non-binding > > > > Kent Yao > > > > Yuming Wang 于2023年12月11日周一 09:33写道： > > > > > > +1 > > > > > > On Mon, Dec 11, 2023 at 5:55 AM Dongjoon Hyun wrote: > > >> > > >> +1 > > >> >

Re: Apache Spark 3.3.4 EOL Release?

+1 for a 3.3.4 EOL Release. Thanks Dongjoon. Jie Yang On 2023/12/04 15:08:25 Tom Graves wrote: > +1 for a 3.3.4 EOL Release. Thanks Dongjoon. > Tom > On Friday, December 1, 2023 at 02:48:22 PM CST, Dongjoon Hyun > wrote: > > Hi, All. > > Since the Apache Spark 3.3.0 RC6 vote passed o

Re: Remove HiveContext from Apache Spark 4.0

Thank you very much for the feedback from Dongjoon and Xiao Li. After carefully reading https://lists.apache.org/thread/mrx0y078cf3ozs7czykvv864y6dr55xq, I have decided to abandon the deletion of HiveContext. As Xiao Li said, its maintenance cost is not high, but it will increase the cost of

Re: [VOTE] Release Spark 3.4.2 (RC1)

+1(non-binding) Jie Yang On 2023/11/29 02:08:04 Kent Yao wrote: > +1(non-binding) > > Kent Yao > > On 2023/11/27 01:12:53 Dongjoon Hyun wrote: > > Hi, Marc. > > > > Given that it exists in 3.4.0 and 3.4.1, I don't think it's a release > > blocker for Apache Spark 3.4.2. > > > > When the patch

Re: What else could be removed in Spark 4?

I would like to know how we should handle the two Kinesis-related modules in Spark 4.0. They have a very low frequency of code updates, and because the corresponding tests are not continuously executed in any GitHub Actions pipeline, so I think they significantly lack quality assurance. On top o

Re: Time for Spark 3.4.0 release?

+1 for me YangJie 发件人: Hyukjin Kwon 日期: 2023年1月4日 星期三 13:16 收件人: Xinrong Meng 抄送: dev 主题: Re: Time for Spark 3.4.0 release? SGTM +1 On Wed, Jan 4, 2023 at 2:13 PM Xinrong Meng mailto:xinrong.apa...@gmail.com>> wrote: Hi All, Shall we cut branch-3.4 on January 16th, 2023? We proposed Januar

Re: maven build failing in spark sql w/BouncyCastleProvider CNFE

bcpkix-jdk15on test ``` Yang Jie 发件人: "Yang,Jie(INF)" 日期: 2022年12月6日 星期二 18:27 收件人: Steve Loughran 抄送: Hyukjin Kwon , Apache Spark Dev 主题: Re: maven build failing in spark sql w/BouncyCastleProvider CNFE I think we can try scala-maven-plugin 4.8.0 发件人: Steve Loughran

Re: maven build failing in spark sql w/BouncyCastleProvider CNFE

I think we can try scala-maven-plugin 4.8.0 发件人: Steve Loughran 日期: 2022年12月6日 星期二 18:19 收件人: "Yang,Jie(INF)" 抄送: Hyukjin Kwon , Apache Spark Dev 主题: Re: maven build failing in spark sql w/BouncyCastleProvider CNFE On Tue, 6 Dec 2022 at 04:10, Yang,Jie(INF) mailto:yangji...

Re: maven build failing in spark sql w/BouncyCastleProvider CNFE

Steve, did compile failed happen when mvn build Spark master with hadoop 3.4.0-SNAPSHOT? 发件人: Hyukjin Kwon 日期: 2022年12月6日 星期二 10:27 抄送: Apache Spark Dev 主题: Re: maven build failing in spark sql w/BouncyCastleProvider CNFE Steve, does the lower version of scala plugin work for you? If that solv

Re: [ANNOUNCE] Apache Spark 3.2.3 released

Thanks, Chao! 发件人: Maxim Gekk 日期: 2022年11月30日 星期三 19:40 收件人: Jungtaek Lim 抄送: Wenchen Fan , Chao Sun , dev , user 主题: Re: [ANNOUNCE] Apache Spark 3.2.3 released Thank you, Chao! On Wed, Nov 30, 2022 at 12:42 PM Jungtaek Lim mailto:kabhwan.opensou...@gmail.com>> wrote: Thanks Chao for drivin

Re: [VOTE] Release Spark 3.2.3 (RC1)

+1,non-binding The test combination of Java 11 + Scala 2.12 and Java 11 + Scala 2.13 has passed. Yang Jie 发件人: Chris Nauroth 日期: 2022年11月17日 星期四 04:27 收件人: Yuming Wang 抄送: "Yang,Jie(INF)" , Dongjoon Hyun , huaxin gao , "L. C. Hsieh" , Chao Sun , dev 主题: Re: [VOTE]

Re: [VOTE][SPIP] Better Spark UI scalability and Driver stability for large applications

+1, non-binding Yang Jie 发件人: Mridul Muralidharan 日期: 2022年11月16日 星期三 17:35 收件人: Kent Yao 抄送: Gengliang Wang , dev 主题: Re: [VOTE][SPIP] Better Spark UI scalability and Driver stability for large applications +1 Would be great to see history server performance improvements and lower

Re: [VOTE] Release Spark 3.2.3 (RC1)

I switched Scala 2.13 to Scala 2.12 today. The test is still in progress and it has not been hung. Yang Jie 发件人: Dongjoon Hyun 日期: 2022年11月16日 星期三 01:17 收件人: "Yang,Jie(INF)" 抄送: huaxin gao , "L. C. Hsieh" , Chao Sun , dev 主题: Re: [VOTE] Release Spark 3.2.3 (RC1) Did y

Re: [VOTE] Release Spark 3.2.3 (RC1)

ase being executed is `SPARK-28323: PythonUDF should be able to use in join condition`, does anyone have the same problem? Yang Jie 发件人: huaxin gao 日期: 2022年11月15日 星期二 13:59 收件人: "L. C. Hsieh" 抄送: Dongjoon Hyun , Chao Sun , dev 主题: Re: [VOTE] Release Spark 3.2.3 (RC1) +1 Thanks Chao!

Re: Upgrade guava to 31.1-jre and remove hadoop2

Hi, Bjørn Currently, I don't think we can consider upgrading Guava for the following reasons: 1. Although Spark 3.4.0 will no longer release the hadoop-2 distribution, but the build and testing process is still running. We need to keep it and will not consider upgrading Guava until it is reall

Re: [ANNOUNCE] Apache Spark 3.3.1 released

Thanks Yuming and all developers ~ Yang Jie 发件人: Maxim Gekk 日期: 2022年10月26日 星期三 15:19 收件人: Hyukjin Kwon 抄送: "L. C. Hsieh" , Dongjoon Hyun , Yuming Wang , dev , User 主题: Re: [ANNOUNCE] Apache Spark 3.3.1 released Congratulations everyone with the new release, and thanks to Yumi

Re: Apache Spark 3.2.3 Release?

+1 发件人: vaquar khan 日期: 2022年10月19日 星期三 10:08 收件人: "416161...@qq.com" 抄送: Yuming Wang , kazuyuki tanimura , Gengliang Wang , huaxin gao , Dongjoon Hyun , Sean Owen , Chao Sun , dev 主题: Re: Apache Spark 3.2.3 Release? +1 On Tue, Oct 18, 2022, 8:58 PM 416161...@qq.com

Re: [VOTE] Release Spark 3.3.1 (RC4)

Use maven to test Java 17 + Scala 2.13 and test passed, +1 for me 发件人: Sean Owen 日期: 2022年10月17日 星期一 21:34 收件人: Yuming Wang 抄送: dev 主题: Re: [VOTE] Release Spark 3.3.1 (RC4) +1 from me, same as last time On Sun, Oct 16, 2022 at 9:14 PM Yuming Wang mailto:wgy...@gmail.com>> wrote: Please vote

答复: Welcome Yikun Jiang as a Spark committer

Congratulations Yikun! Regards, Yang Jie 发件人: Mridul Muralidharan 发送时间: 2022年10月8日 14:16:02 收件人: Yuming Wang 抄送: Hyukjin Kwon; dev; Yikun Jiang 主题: Re: Welcome Yikun Jiang as a Spark committer Congratulations ! Regards, Mridul On Sat, Oct 8, 2022 at 12:19 AM

Re: Dropping Apache Spark Hadoop2 Binary Distribution?

Hi, Dongjoon Our company(Baidu) is still using the combination of Spark 3.3 + Hadoop 2.7.4 in the production environment. Hadoop 2.7.4 is an internally maintained version compiled by Java 8. Although we are using Hadoop 2, I still support this proposal because it is positive and exciting. Rega

Re: [VOTE] SPIP: Support Docker Official Image for Spark

+1 (non-binding) Regards, Yang Jie 发件人: Gengliang Wang 日期: 2022年9月22日 星期四 12:22 收件人: Xiangrui Meng 抄送: Kent Yao , Hyukjin Kwon , dev 主题: Re: [VOTE] SPIP: Support Docker Official Image for Spark +1 On Wed, Sep 21, 2022 at 7:26 PM Xiangrui Meng mailto:men...@gmail.com>> wrote: +1

答复: [DISCUSS] SPIP: Support Docker Official Image for Spark

+1 (non-binding) Yang Jie 发件人: Yikun Jiang 发送时间: 2022年9月19日 14:23:14 收件人: Denny Lee 抄送: bo zhaobo; Yuming Wang; Kent Yao; Gengliang Wang; Hyukjin Kwon; dev; zrf 主题: Re: [DISCUSS] SPIP: Support Docker Official Image for Spark Thanks for your support! @all

Re: Time for Spark 3.3.1 release?

+1 Thanks Yuming ~ 发件人: Hyukjin Kwon 日期: 2022年9月13日 星期二 08:19 收件人: Gengliang Wang 抄送: "L. C. Hsieh" , Dongjoon Hyun , Yuming Wang , dev 主题: Re: Time for Spark 3.3.1 release? +1 On Tue, 13 Sept 2022 at 06:45, Gengliang Wang mailto:ltn...@gmail.com>> wrote: +1. Thank you, Yuming! On Mon, Se

Re: memory module of yarn container

Hi, vtygoss In my memory, the memoryOverhead in Spark 2.3 includes all the memories that are not executor onHeap memory, including the memory used by Spark offheapMemoryPool(executorOffHeapMemory, this concept also exists in Spark 2.3), PySparkWorker, PipeRDD used, netty memory pool, JVM di

Re: Welcoming three new PMC members

Congratulations! Regards, Yang Jie 发件人: Yuming Wang 日期: 2022年8月10日 星期三 16:42 收件人: Yikun Jiang 抄送: dev 主题: Re: Welcoming three new PMC members Congratulations! On Wed, Aug 10, 2022 at 4:35 PM Yikun Jiang mailto:yikunk...@gmail.com>> wrote: Congratulations! Regards, Yikun On Wed,

Re: Welcome Xinrong Meng as a Spark committer

Congratulations! Regards, Yang Jie 发件人: Hyukjin Kwon 日期: 2022年8月9日 星期二 16:12 收件人: dev 主题: Welcome Xinrong Meng as a Spark committer Hi all, The Spark PMC recently added Xinrong Meng as a committer on the project. Xinrong is the major contributor of PySpark especially Pandas API on Spark

Re: [VOTE] Release Spark 3.2.2 (RC1)

+1 (non-binding) Yang Jie 发件人: Dongjoon Hyun 日期: 2022年7月12日 星期二 16:03 收件人: dev 抄送: Cheng Su , "Yang,Jie(INF)" , Sean Owen 主题: Re: [VOTE] Release Spark 3.2.2 (RC1) +1 Dongjoon. On Mon, Jul 11, 2022 at 11:34 PM Cheng Su mailto:scnj...@gmail.com>> wrote: +1 (non-binding). B

Re: [VOTE] Release Spark 3.2.2 (RC1)

Does this happen when running all UTs? I ran this suite several times alone using OpenJDK(zulu) 8u322-b06 on my Mac, but no similar error occurred 发件人: Sean Owen 日期: 2022年7月12日 星期二 10:45 收件人: Dongjoon Hyun 抄送: dev 主题: Re: [VOTE] Release Spark 3.2.2 (RC1) Is anyone seeing this error? I'm on Op

Re: Apache Spark 3.2.2 Release?

+1 (non-binding) Thank you Dongjoon ~ 发件人: Ruifeng Zheng 日期: 2022年7月7日 星期四 16:28 收件人: dev 主题: Re: Apache Spark 3.2.2 Release? +1 thank you Dongjoon! [图像已被发件人删除。] Ruifeng Zheng ruife...@foxmail.com -- Original -- From: "Yikun

Contributor data in github-page no longer updated after May 1

Hi, teams The contributors data in the following page seems no longer updated after May 1, Can anyone fix it? https://github.com/apache/spark/graphs/contributors?from=2022-05-01&to=2022-05-11&type=c Warm regards, YangJie

Re: [VOTE] Release Spark 3.1.1 (RC1)

+1 发件人: Gengliang Wang 日期: 2021年1月19日 星期二 下午3:04 收件人: Jungtaek Lim 抄送: Yuming Wang , Hyukjin Kwon , dev 主题: Re: [VOTE] Release Spark 3.1.1 (RC1) +1 (non-binding) On Tue, Jan 19, 2021 at 2:05 PM Jungtaek Lim mailto:kabhwan.opensou...@gmail.com>> wrote: +1 (non-binding) * verified signature