Re: [VOTE] SPIP: Declarative Pipelines

+1 On Wed, Apr 9, 2025 at 11:57 AM Martin Grund wrote: > +1 > > On Wed, Apr 9, 2025 at 8:43 PM Denny Lee wrote: > >> +1 (non-binding) >> >> On Wed, Apr 9, 2025 at 11:03 Chao Sun wrote: >> >>> +1 >>> >>> On Wed, Apr 9, 2025 at 10:55 AM L. C. Hsieh wrote: >>> +1 On Wed, Apr 9, 20

Re: [VOTE] SPIP: Constraints in DSv2

sponsible for something it does >>> not control. >>> >>> El vie, 21 mar 2025 a las 20:18, L. C. Hsieh () >>> escribió: >>> >>>> +1 >>>> >>>> On Fri, Mar 21, 2025 at 12:13 PM huaxin gao >>>> wrote: &

Re: [VOTE] SPIP: Constraints in DSv2

+1 On Fri, Mar 21, 2025 at 11:46 AM Anton Okolnychyi wrote: > Hi all, > > I would like to start a vote on adding support for constraints to DSv2. > > *Discussion thread: * > https://lists.apache.org/thread/njqjcryq0lot9rkbf10mtvf7d1t602bj > *SPIP:* > https://docs.google.com/document/d/1EHjB4W1Lj

Re: [Discuss] SPIP: Support NanoSecond Timestamps

Hi Qi, Thanks for the proposal. I am generally +1 with the idea. Could you clarify which option is preferred in “Q1. What are you trying to do?”? Understanding this will help us align our discussion. On Mon, Mar 17, 2025 at 3:05 PM Reynold Xin wrote: > Pretty much anything (say vs current tim

Re: [VOTE] SPIP: Add the TIME data type

+1 On Sun, Feb 23, 2025 at 8:40 PM Hyukjin Kwon wrote: > +1 > > On Mon, 24 Feb 2025 at 11:47, Wenchen Fan wrote: > >> +1 >> >> On Mon, Feb 24, 2025 at 7:51 AM John Zhuge wrote: >> >>> +1 (non-binding) >>> >>> John Zhuge >>> >>> >>> On Sun, Feb 23, 2025 at 2:37 PM huaxin gao >>> wrote: >>> >>>

Re: [DISCUSS] SPIP: Constraints in DSv2

+1, the proposal will unify constraint management in DSv2 and reduce redundant work across connectors On Thu, Feb 13, 2025 at 9:20 PM Anton Okolnychyi wrote: > Hi folks, > > I'd like to start a discussion on SPARK-51207 that aims to extend the DSv2 > API to let users define, modify, and enforce

Spark Website Styling Issues Partially Resolved

pache.org/jira/browse/SPARK-51104> to track the remaining fixes. I will work with community developers to address these, and contributions are welcome. Gengliang Wang

Re: Docs look weird; can't build locally

FYI, I’ve created a fix for this issue: [SPARK-51104][DOC] Self-host JavaScript and CSS in Spark website On Wed, Feb 5, 2025 at 4:27 PM Hyukjin Kwon wrote: > I updated Jeykll version at > https://github.com/apache/spark/commit/3675568a34d42054686eb213

Re: Quick question: can you guys navigate 3.5.4 Java API documentation?

FYI, I’ve created a fix for this issue: [SPARK-51104][DOC] Self-host JavaScript and CSS in Spark website On Tue, Feb 4, 2025 at 11:10 PM wrote: > 1.It seems to be a `CSP` issue, as follows: > [image: d14447a60a387863b505f6b66b8e012a.png] > > 2.What is

Re: [VOTE] Publish additional Spark distribution with Spark Connect enabled

+1 On Tue, Feb 4, 2025 at 11:38 PM Sakthi wrote: > +1 (non-binding) > > On Tue, Feb 4, 2025 at 11:25 PM DB Tsai wrote: > >> +1 >> >> DB Tsai | https://www.dbtsai.com/ | PGP 42E5B25A8F7A82C1 >> >> On Feb 4, 2025, at 11:13 PM, Xiao Li wrote: >> >> +1 >> >> Xiao >> >> Jungtaek Lim 于2025年2月4日

Re: A documentation change is a user-facing change

Thanks for pointing it out! Based on the discussion, I’ve created a PR: https://github.com/apache/spark/pull/49534. Let me know what you think! On Thu, Jan 16, 2025 at 2:25 PM Xiao Li wrote: > Thank you for pointing it out! Let’s update the template to exclude > documentation changes from the b

Re: [VOTE] Use plain text logs by default

+1 On Wed, Jan 8, 2025 at 10:17 PM Jungtaek Lim wrote: > +1 (non-binding) > > 2025년 1월 9일 (목) 오후 3:06, Cheng Pan 님이 작성: > >> +1 >> >> Thanks, >> Cheng Pan >> >> >> >> On Jan 9, 2025, at 12:28, Wenchen Fan wrote: >> >> Hi all, >> >> Following the discussion[1], I'd like to start the vote for 'Us

Re: [DISCUSS] Use plain text logs by default

eal-world feedback is a part of that process. I'm looking forward to hearing your thoughts and discussing how we can continue to improve our logging practices. Best regards, Gengliang Wang On Fri, Nov 22, 2024 at 3:32 PM bo yang wrote: > +1 for default using plain text logging. It is go

Re: New committer: Bingkun Pan

Congrats! On Sat, Nov 16, 2024 at 4:12 AM xianjin wrote: > Congrats! > > Sent from my iPhone > > On Nov 16, 2024, at 5:46 PM, Hyukjin Kwon wrote: > >  > Congrats!! > > On Sat, Nov 16, 2024 at 11:46 AM Jungtaek Lim < > kabhwan.opensou...@gmail.com> wrote: > >> Congrats Bingkun! >> >> 2024년 11월

Re: [VOTE] Single-pass Analyzer for Catalyst

+1 On Mon, Sep 30, 2024 at 6:22 PM Jungtaek Lim wrote: > +1 (non-binding), promising proposal! > > 2024년 10월 1일 (화) 오전 8:04, Dongjoon Hyun 님이 작성: > >> Thank you for the swift clarification, Reynold and Xiao. >> >> It seems that the Target Version was set mistakenly initially. >> >> I removed the

Re: [VOTE] Release Spark 4.0.0-preview2 (RC1)

+1 On Wed, Sep 18, 2024 at 10:47 PM Xiao Li wrote: > +1 > > On Wed, Sep 18, 2024 at 18:22 Yuming Wang wrote: > >> +1 >> >> On Wed, Sep 18, 2024 at 6:07 PM Cheng Pan wrote: >> >>> +1 (non-binding) >>> >>> I checked >>> - Signatures and checksums are good. >>> - Build success from source code. >

Re: [VOTE] Document and Feature Preview via GitHub Pages

+1 On Wed, Sep 11, 2024 at 6:30 AM Wenchen Fan wrote: > +1 > > On Wed, Sep 11, 2024 at 5:15 PM Martin Grund > wrote: > >> +1 >> >> On Wed, Sep 11, 2024 at 9:39 AM Kent Yao wrote: >> >>> Hi all, >>> >>> Following the discussion[1], I'd like to start the vote for 'Document and >>> Feature Previe

Re: [VOTE] Release Apache Spark 3.5.3 (RC3)

+1 On Mon, Sep 9, 2024 at 6:01 PM Wenchen Fan wrote: > +1 > > On Tue, Sep 10, 2024 at 7:42 AM Rui Wang > wrote: > >> +1 (non-binding) >> >> >> -Rui >> >> On Mon, Sep 9, 2024 at 4:22 PM Hyukjin Kwon wrote: >> >>> +1 >>> >>> On Tue, Sep 10, 2024 at 5:39 AM Haejoon Lee >>> wrote: >>> Hi, de

Re: [VOTE] Move Variant to Parquet

+1 On Tue, Sep 3, 2024 at 4:29 PM rdb...@gmail.com wrote: > +1 > > On Tue, Sep 3, 2024 at 3:38 PM Gene Pang wrote: > >> Hi Micah, >> >> I wanted to open this vote to get official alignment on where the Spark >> community wants to move the Variant spec and implementation. There are >> several po

Re: [DISCUSS] release Spark 3.5.3?

+1 for the new release On Mon, Sep 2, 2024 at 4:09 AM Wenchen Fan wrote: > Thanks for the support! > > @Yuming Wang I looked into the tickets you mentioned. > I think the first one is not an issue, the second one is bad error message > and not a blocker. > > @Haejoon Lee There is a newly > rep

Re: [VOTE] Deprecate SparkR

+1 On Thu, Aug 22, 2024 at 11:36 PM Yang Jie wrote: > +1 > > On 2024/08/23 00:19:40 Chao Sun wrote: > > +1 > > > > On Thu, Aug 22, 2024 at 5:19 PM Yuming Wang wrote: > > > > > +1 > > > > > > On Fri, Aug 23, 2024 at 5:42 AM Allison Wang > > > wrote: > > > > > >> +1 > > >> > > >> On Thu, Aug 22,

Re: Welcome new Apache Spark committers

Congratulations, everyone! On Mon, Aug 12, 2024 at 7:10 PM Denny Lee wrote: > Congrats Allison, Martin, and Haejoon! > > On Tue, Aug 13, 2024 at 9:59 AM Jungtaek Lim > wrote: > >> Congrats everyone! >> >> On Tue, Aug 13, 2024 at 9:21 AM Xiao Li wrote: >> >>> Congratulations! >>> >>> Hyukjin Kw

Re: Welcoming a new PMC member

Congratulations! On Mon, Aug 12, 2024 at 7:12 PM Denny Lee wrote: > Congrats, Kent! > > On Tue, Aug 13, 2024 at 9:06 AM Dongjoon Hyun > wrote: > >> Congratulations, Kent. >> >> Dongjoon. >> >> On Mon, Aug 12, 2024 at 5:22 PM Xiao Li wrote: >> >>> Congratulations ! >>> >>> Hyukjin Kwon 于2024年8

Re: [VOTE] Archive Spark Documentations in Apache Archives

+1 On Mon, Aug 12, 2024 at 2:01 PM Xiao Li wrote: > +1 > > Mich Talebzadeh 于2024年8月12日周一 13:11写道： > >> Hi Kent, >> >> Can you if possible please provide a heuristic estimate of storage >> reduction that will be achieved through this approach? >> >> Thanks >> >> Mich Talebzadeh, >> >> Architect

Re: [VOTE] Release Spark 3.5.2 (RC5)

+1. Thanks for creating this new RC. I confirmed that SPARK-49054 is fixed. On Fri, Aug 9, 2024 at 6:54 AM Wenchen Fan wrote: > +1 > > On Fri, Aug 9, 2024 at 6:04 PM Peter Toth wrote: > >> +1 >> >> huaxin gao ezt írta (időpont: 2024. aug. 8.,

Re: [VOTE] Release Spark 3.5.2 (RC4)

apologize for the late vote, as I understand the voting period closed on July 29 at 14:00:00 UTC. I appreciate your understanding and consideration of this important issue. Gengliang Wang On Tue, Jul 30, 2024 at 4:22 AM Cheng Pan wrote: > +1 (non-binding) > > - All links are valid a

Re: [VOTE] Differentiate Spark without Spark Connect from Spark Connect

+1 On Mon, Jul 22, 2024 at 5:19 PM Hyukjin Kwon wrote: > Starting with my own +1. > > On Tue, 23 Jul 2024 at 09:12, Hyukjin Kwon wrote: > >> Hi all, >> >> I’d like to start a vote for differentiating "Spark without Spark >> Connect" as "Spark Classic". >> >> Please also refer to: >> >>- Dis

Re: [VOTE] Move Spark Connect server to builtin package (Client API layer stays external)

+1 On Wed, Jul 3, 2024 at 4:48 PM Reynold Xin wrote: > +1 > > On Wed, Jul 3, 2024 at 4:45 PM L. C. Hsieh wrote: > >> +1 >> >> On Wed, Jul 3, 2024 at 3:54 PM Dongjoon Hyun >> wrote: >> > >> > +1 >> > >> > Dongjoon >> > >> > On Wed, Jul 3, 2024 at 10:58 Xinrong Meng wrote: >> >> >> >> +1 >> >>

Re: [VOTE] SPARK 4.0.0-preview1 (RC3)

+1 On Fri, May 31, 2024 at 11:06 AM Xiao Li wrote: > +1 > > Cheng Pan 于2024年5月30日周四 09:48写道： > >> +1 (non-binding) >> >> - All links are valid >> - Run some basic quires using YARN client mode with Apache Hadoop v3.3.6, >> HMS 2.3.9 >> - Pass integration tests with Apache Kyuubi v1.9.1 RC0 >> >

Re: [VOTE] SPIP: Stored Procedures API for Catalogs

+1 On Mon, May 13, 2024 at 12:30 PM Zhou Jiang wrote: > +1 (non-binding) > > On Sat, May 11, 2024 at 2:10 PM L. C. Hsieh wrote: > >> Hi all, >> >> I’d like to start a vote for SPIP: Stored Procedures API for Catalogs. >> >> Please also refer to: >> >>- Discussion thread: >> https://lists.ap

Re: [VOTE] SPARK-46122: Set spark.sql.legacy.createHiveTableByDefault to false

+1 On Fri, Apr 26, 2024 at 10:01 AM Dongjoon Hyun wrote: > I'll start with my +1. > > Dongjoon. > > On 2024/04/26 16:45:51 Dongjoon Hyun wrote: > > Please vote on SPARK-46122 to set > spark.sql.legacy.createHiveTableByDefault > > to `false` by default. The technical scope is defined in the follo

Re: [VOTE] Release Spark 3.4.3 (RC2)

+1 On Tue, Apr 16, 2024 at 11:57 AM L. C. Hsieh wrote: > +1 > > On Tue, Apr 16, 2024 at 4:08 AM Wenchen Fan wrote: > > > > +1 > > > > On Mon, Apr 15, 2024 at 12:31 PM Dongjoon Hyun > wrote: > >> > >> I'll start with my +1. > >> > >> - Checked checksum and signature > >> - Checked Scala/Java/R/

Re: [VOTE] SPARK-44444: Use ANSI SQL mode by default

+1 On Sat, Apr 13, 2024 at 3:26 PM Dongjoon Hyun wrote: > I'll start from my +1. > > Dongjoon. > > On 2024/04/13 22:22:05 Dongjoon Hyun wrote: > > Please vote on SPARK-4 to use ANSI SQL mode by default. > > The technical scope is defined in the following PR which is > > one line of code chan

Re: [DISCUSS] SPARK-44444: Use ANSI SQL mode by default

or Attribution Framework <https://issues.apache.org/jira/browse/SPARK-38615> will also be beneficial in migrating to ANSI SQL mode. Gengliang On Thu, Apr 11, 2024 at 7:56 PM Dongjoon Hyun wrote: > Hi, All. > > Thanks to you, we've been achieving many things and have on-going SPI

Re: [VOTE] SPIP: Pure Python Package in PyPI (Spark Connect)

+1 On Sun, Mar 31, 2024 at 8:24 PM Dongjoon Hyun wrote: > +1 > > Thank you, Hyukjin. > > Dongjoon > > On Sun, Mar 31, 2024 at 19:07 Haejoon Lee > wrote: > >> +1 >> >> On Mon, Apr 1, 2024 at 10:15 AM Hyukjin Kwon >> wrote: >> >>> Hi all, >>> >>> I'd like to start the vote for SPIP: Pure Python

Re: Allowing Unicode Whitespace in Lexer

+1, this is a reasonable change. Gengliang On Wed, Mar 27, 2024 at 9:54 AM serge rielau.com wrote: > Going once, going twice, …. last call for objections > On Mar 23, 2024 at 5:29 PM -0700, serge rielau.com , > wrote: > > Hello, > > I have a PR https://github.com/apache/sp

[VOTE][RESULT] SPIP: Structured Logging Framework for Apache Spark

(*) - Scott - Jungtaek Lim - Reynold Xin (*) - Holden Karau (*) - Xiao Li (*) - Chao Sun (*) - Liang-Chi Hsieh (*) - rhatlnux - Robyn Nameth - John Zhuge - Ruifeng Zheng (*) - Tom Graves (*) - Bo Yang +0: None -1: None Thanks, Gengliang Wang

Re: [VOTE] SPIP: Structured Logging Framework for Apache Spark

e test result is worth one-thousand >> expert opinions (Werner >> <https://en.wikipedia.org/wiki/Wernher_von_Braun>Von Braun >> <https://en.wikipedia.org/wiki/Wernher_von_Braun>)". >> >> >> On Mon, 11 Mar 2024 at 09:27, Hyukjin Kwon wrote: >> &g

Re: [VOTE] SPIP: Structured Logging Framework for Apache Spark

com/document/d/1rATVGmFLNVLmtxSpWrEceYm7d-ocgu8ofhryVs4g3XU/edit#bookmark=id.8zbyavz648i6> Thanks, Gengliang On Mon, Mar 11, 2024 at 9:30 AM huaxin gao wrote: > +1 > > On Mon, Mar 11, 2024 at 7:02 AM Wenchen Fan wrote: > >> +1 >> >> On Mon, Mar 11, 2024 at 5:26 PM Hyukjin Kwon >&g

[VOTE] SPIP: Structured Logging Framework for Apache Spark

it?usp=sharing> - Discussion thread <https://lists.apache.org/thread/gocslhbfv1r84kbcq3xt04nx827ljpxq> Please vote on the SPIP for the next 72 hours: [ ] +1: Accept the proposal as an official SPIP [ ] +0 [ ] -1: I don’t think this is a good idea because … Thanks! Gengliang Wang

Re: [DISCUSS] SPIP: Structured Spark Logging

Thanks everyone for the valuable feedback! Given the generally positive feedback received, I plan to move forward by initiating the voting thread. I encourage you to participate in the upcoming thread. Warm regards, Gengliang On Sat, Mar 9, 2024 at 12:55 PM Mich Talebzadeh wrote: > Splen

Re: [DISCUSS] SPIP: Structured Spark Logging

treaming" clearly. Thanks for helping shape this! Best, Gengliang On Sat, Mar 2, 2024 at 12:19 PM Mich Talebzadeh wrote: > Hi Gengliang, > > Thanks for taking the initiative to improve the Spark logging system. > Transitioning to structured logs seems like a worthy way to en

[DISCUSS] SPIP: Structured Spark Logging

Hi All, I propose to enhance our logging system by transitioning to structured logs. This initiative is designed to tackle the challenges of analyzing distributed logs from drivers, workers, and executors by allowing them to be queried using a fixed schema. The goal is to improve the informativene

Re: Re: [DISCUSS] Release Spark 3.5.1?

+1 On Sun, Feb 4, 2024 at 1:57 PM Hussein Awala wrote: > +1 > > On Sun, Feb 4, 2024 at 10:13 PM John Zhuge wrote: > >> +1 >> >> John Zhuge >> >> >> On Sun, Feb 4, 2024 at 11:23 AM Santosh Pingale >> wrote: >> >>> +1 >>> >>> On Sun, Feb 4, 2024, 8:18 PM Xiao Li >>> wrote: >>> +1

Re: Algolia search on website is broken

r" is now ".container". I've updated the crawler accordingly, and it's working properly now. Gengliang On Sun, Dec 10, 2023 at 8:15 AM Nicholas Chammas wrote: > Pinging Gengliang and Xiao about this, per these docs > <https://github.com/apache/spark-websit

Re: [VOTE] SPIP: Testing Framework for Spark UI Javascript files

+1 On Sat, Nov 25, 2023 at 2:50 AM yangjie01 wrote: > +1 > > > > *发件人**: *Reynold Xin > *日期**: *2023年11月25日 星期六 14:35 > *收件人**: *Dongjoon Hyun > *抄送**: *Ye Zhou , Mridul Muralidharan < > mri...@gmail.com>, Kent Yao , dev > *主题**: *Re: [VOTE] SPIP: Testing Framework for Spark UI Javascript fil

Re: Welcome to Our New Apache Spark Committer and PMCs

Congratulations to all! Well deserved! On Mon, Oct 2, 2023 at 10:16 PM Xiao Li wrote: > Hi all, > > The Spark PMC is delighted to announce that we have voted to add one new > committer and two new PMC members. These individuals have consistently > contributed to the project and have clearly demo

Re: [VOTE] Release Apache Spark 3.5.0 (RC5)

+1 On Mon, Sep 11, 2023 at 11:28 AM Xiao Li wrote: > +1 > > Xiao > > Yuanjian Li 于2023年9月11日周一 10:53写道： > >> @Peter Toth I've looked into the details of this >> issue, and it appears that it's neither a regression in version 3.5.0 nor a >> correctness issue. It's a bug related to a new feature

Re: [VOTE] Release Apache Spark 3.5.0 (RC4)

+1 On Wed, Sep 6, 2023 at 9:46 PM Yuanjian Li wrote: > +1 (non-binding) > > Xiao Li 于2023年9月6日周三 15:27写道： > >> +1 >> >> Xiao >> >> Herman van Hovell 于2023年9月6日周三 22:08写道： >> >>> Tested connect, and everything looks good. >>> >>> +1 >>> >>> On Wed, Sep 6, 2023 at 8:11 AM Yuanjian Li >>> wrote:

Re: Welcome two new Apache Spark committers

Congratulations! Peter and Xiduo! On Sun, Aug 6, 2023 at 7:37 PM Jungtaek Lim wrote: > Congrats Peter and Xiduo! > > On Mon, Aug 7, 2023 at 11:33 AM yangjie01 > wrote: > >> Congratulations, Peter and Xiduo ~ >> >> >> >> *发件人**: *Hyukjin Kwon >> *日期**: *2023年8月7日 星期一 10:30 >> *收件人**: *Ruifeng Z

Re: [Reminder] Spark 3.5 Branch Cut

Hi Yuanjian, Besides the abovementioned changes, it would be great to include the UI page for Spakr Connect: SPARK-44394 <https://issues.apache.org/jira/browse/SPARK-44394>. Best Regards, Gengliang On Fri, Jul 14, 2023 at 11:44 AM Julek Sompolski wrote: > Thank you, > My chan

Introducing English SDK for Apache Spark - Seeking Your Feedback and Contributions

hable and user-friendly. Thank you in advance for your attention and involvement. We look forward to hearing your thoughts and seeing your contributions! Best, Gengliang Wang

Re: [VOTE] Release Spark 3.4.1 (RC1)

+1 On Thu, Jun 22, 2023 at 11:14 AM Driesprong, Fokko wrote: > Thank you for running the release Dongjoon > > +1 > > Tested against Iceberg and it looks good. > > > Op do 22 jun 2023 om 18:03 schreef yangjie01 : > >> +1 >> >> >> >> *发件人**: *Dongjoon Hyun >> *日期**: *2023年6月22日 星期四 23:35 >> *收件人*

Re: [ANNOUNCE] Apache Spark 3.4.0 released

Congratulations everyone! Thank you Xinrong for driving the release! On Fri, Apr 14, 2023 at 12:47 PM Xinrong Meng wrote: > Hi All, > > We are happy to announce the availability of *Apache Spark 3.4.0*! > > Apache Spark 3.4.0 is the fifth release of the 3.x line. > > To download Spark 3.4.0, hea

Re: [VOTE] Release Apache Spark 3.4.0 (RC7)

+1 On Sun, Apr 9, 2023 at 3:17 PM Dongjoon Hyun wrote: > +1 > > I verified the same steps like previous RCs. > > Dongjoon. > > > On Sat, Apr 8, 2023 at 7:47 PM Mridul Muralidharan > wrote: > >> >> +1 >> >> Signatures, digests, etc check out fine. >> Checked out tag and build/tested with -Phive

Re: Apache Spark 3.2.4 EOL Release?

+1 On Wed, Apr 5, 2023 at 11:27 AM kazuyuki tanimura wrote: > +1 > > On Apr 5, 2023, at 6:53 AM, Tom Graves > wrote: > > +1 > > Tom > > On Tuesday, April 4, 2023 at 12:25:13 PM CDT, Dongjoon Hyun < > dongjoon.h...@gmail.com> wrote: > > > Hi, All. > > Since Apache Spark 3.2.0 passed RC7 vote on

Re: [VOTE] Release Apache Spark 3.4.0 (RC5)

Hi Anton, +1 for adding the old constructors back! Could you raise a PR for this? I will review it ASAP. Thanks Gengliang On Wed, Apr 5, 2023 at 9:37 AM Anton Okolnychyi wrote: > Sorry, I think my last message did not land on the list. > > I have a question about changes to exception

Re: [VOTE] Release Apache Spark 3.4.0 (RC1)

fc14282b> Gengliang On Wed, Feb 22, 2023 at 3:09 PM Tom Graves wrote: > It looks like there are still blockers open, we need to make sure they are > addressed before doing a release: > > https://issues.apache.org/jira/browse/SPARK-41793 > https://issues.apache.org/jira/browse/SPARK-

Re: Time for Spark 3.4.0 release?

+1, thanks for driving the release! Gengliang On Tue, Jan 3, 2023 at 10:55 PM Dongjoon Hyun wrote: > +1 > > Thank you! > > Dongjoon > > On Tue, Jan 3, 2023 at 9:44 PM Rui Wang wrote: > >> +1 to cut the branch starting from a workday! >> >> Great to s

[VOTE][RESULT] SPIP: Better Spark UI scalability and Driver stability for large applications

The vote passes with 11 +1s(3 binding +1s) +1: Kent Yao Mridul Muralidharan* Jie Yang Yuming Wang Maciej Szymkiewicz* Chris Nauroth Jungtaek Lim Ye Zhou Wenchen Fan* Ruifeng Zheng Peter Toth 0: None -1: None (* = binding) Thank you all for chiming in and for your votes! Cheers, Gengliang

[VOTE][SPIP] Better Spark UI scalability and Driver stability for large applications

lease vote on the SPIP for the next 72 hours: [ ] +1: Accept the proposal as an official SPIP [ ] +0 [ ] -1: I don’t think this is a good idea because … Kind Regards, Gengliang

Re: [DISCUSS] SPIP: Better Spark UI scalability and Driver stability for large applications

With the positive feedback from Mridul and Wenchen, I will officially start the vote. On Tue, Nov 15, 2022 at 8:57 PM Wenchen Fan wrote: > This looks great! UI stability/scalability has been a pain point for a > long time. > > On Sat, Nov 12, 2022 at 5:24 AM Gengliang Wang wr

Re: Apache Spark 3.2.3 Release?

+1. Thanks Chao! On Tue, Oct 18, 2022 at 11:45 AM huaxin gao wrote: > +1 Thanks Chao! > > Huaxin > > On Tue, Oct 18, 2022 at 11:29 AM Dongjoon Hyun > wrote: > >> +1 >> >> Thank you for volunteering, Chao! >> >> Dongjoon. >> >> >> On Tue, Oct 18, 2022 at 9:55 AM Sean Owen wrote: >> >>> OK by me

Re: [VOTE] Release Spark 3.3.1 (RC4)

+1 from me, same as last time. On Tue, Oct 18, 2022 at 11:45 AM L. C. Hsieh wrote: > +1 > > Thanks Yuming! > > On Tue, Oct 18, 2022 at 11:28 AM Dongjoon Hyun > wrote: > > > > +1 > > > > Thank you, Yuming and all! > > > > Dongjoon. > > > > > > On Tue, Oct 18, 2022 at 9:22 AM Yang,Jie(INF) > wro

Re: Welcome Yikun Jiang as a Spark committer

Congratulations, Yikun! On Sun, Oct 9, 2022 at 12:33 AM 416161...@qq.com wrote: > Congrats, Yikun! > > -- > Ruifeng Zheng > ruife...@foxmail.com > >

Re: [VOTE] Release Spark 3.3.1 (RC2)

+1. I ran some simple tests and also verified that SPARK-40389 is fixed. Gengliang On Mon, Oct 3, 2022 at 8:56 AM Thomas Graves wrote: > +1. ran out internal tests and everything looks good. > > Tom Graves > > On Wed, Sep 28, 2022 at 12:20 AM Yuming Wang wrote: > &g

Re: [VOTE] SPIP: Support Docker Official Image for Spark

+1 On Wed, Sep 21, 2022 at 7:26 PM Xiangrui Meng wrote: > +1 > > On Wed, Sep 21, 2022 at 6:53 PM Kent Yao wrote: > >> +1 >> >> *Kent Yao * >> @ Data Science Center, Hangzhou Research Institute, NetEase Corp. >> *a spark enthusiast* >> *kyuubi is a >> unified

Re: [DISCUSS] SPIP: Support Docker Official Image for Spark

+1, thanks for the work! On Sun, Sep 18, 2022 at 6:20 PM Hyukjin Kwon wrote: > +1 > > On Mon, 19 Sept 2022 at 09:15, Yikun Jiang wrote: > >> Hi, all >> >> I would like to start the discussion for supporting Docker Official Image >> for Spark. >> >> This SPIP is proposed to add Docker Official I

Re: Time for Spark 3.3.1 release?

+1. Thank you, Yuming! On Mon, Sep 12, 2022 at 12:10 PM L. C. Hsieh wrote: > +1 > > Thanks Yuming! > > On Mon, Sep 12, 2022 at 11:50 AM Dongjoon Hyun > wrote: > > > > +1 > > > > Thanks, > > Dongjoon. > > > > On Mon, Sep 12, 2022 at 6:38 AM Yuming Wang wrote: > >> > >> Hi, All. > >> > >> > >> >

Re: Welcome Xinrong Meng as a Spark committer

Congratulations, Xinrong! Well deserved. On Tue, Aug 9, 2022 at 7:09 AM Yi Wu wrote: > Congrats Xinrong!! > > > On Tue, Aug 9, 2022 at 7:07 PM Maxim Gekk > wrote: > >> Congratulations, Xinrong! >> >> Maxim Gekk >> >> Software Engineer >> >> Databricks, Inc. >> >> >> On Tue, Aug 9, 2022 at 3:15

Re: [VOTE] Release Spark 3.2.2 (RC1)

Hi Bruce, FYI we had further discussions on https://github.com/apache/spark/pull/35313#issuecomment-1185195455. Thanks for pointing that out, but this document issue should not be a blocker of the release. +1 on the RC. Gengliang On Thu, Jul 14, 2022 at 10:22 PM sarutak wrote: > Hi Dongj

Re: Apache Spark 3.2.2 Release?

+1. Thank you, Dongjoon. On Wed, Jul 6, 2022 at 10:21 PM Wenchen Fan wrote: > +1 > > On Thu, Jul 7, 2022 at 10:41 AM Xinrong Meng > wrote: > >> +1 >> >> Thanks! >> >> >> Xinrong Meng >> >> Software Engineer >> >> Databricks >> >> >> On Wed, Jul 6, 2022 at 7:25 PM Xiao Li wrote: >> >>> +1 >>> >

Docker images for Spark 3.3.0 release are now available

Gengliang

Re: Re: [VOTE][SPIP] Spark Connect

+1 (non-binding) On Wed, Jun 15, 2022 at 9:32 AM Dongjoon Hyun wrote: > +1 > > On Wed, Jun 15, 2022 at 9:22 AM Xiao Li wrote: > >> +1 >> >> Xiao >> >> beliefer 于2022年6月14日周二 03:35写道： >> >>> +1 >>> Yeah, I tried to use Apache Livy, so as we can runing interactive query. >>> But the Spark Driver

Re: Stickers and Swag

FYI now you can find the shopping information on https://spark.apache.org/community <https://spark.apache.org/community> as well :) Gengliang > On Jun 14, 2022, at 7:47 PM, Hyukjin Kwon wrote: > > Woohoo > > On Tue, 14 Jun 2022 at 15:04, Xiao Li <mailto:gato

Re: [VOTE] Release Spark 3.3.0 (RC6)

+1 (non-binding) On Mon, Jun 13, 2022 at 10:20 AM Herman van Hovell wrote: > +1 > > On Mon, Jun 13, 2022 at 12:53 PM Wenchen Fan wrote: > >> +1, tests are all green and there are no more blocker issues AFAIK. >> >> On Fri, Jun 10, 2022 at 12:27 PM Maxim Gekk >> wrote: >> >>> Please vote on rel

Re: [VOTE] Release Spark 3.3.0 (RC5)

+1 (non-binding) Gengliang On Tue, Jun 7, 2022 at 12:24 PM Thomas Graves wrote: > +1 > > Tom Graves > > On Sat, Jun 4, 2022 at 9:50 AM Maxim Gekk > wrote: > > > > Please vote on releasing the following candidate as Apache Spark version > 3.3.0. > > >

Re: [VOTE] Release Spark 3.3.0 (RC2)

Hi Kent and Wenchen, Thanks for reporting. I just created https://github.com/apache/spark/pull/36609 to fix the issue. Gengliang On Thu, May 19, 2022 at 5:40 PM Wenchen Fan wrote: > I think it should have been fixed by > https://github.com/apache/spark/

Re: SIGMOD System Award for Apache Spark

Congratulations to the whole spark community! On Fri, May 13, 2022 at 10:14 AM Jungtaek Lim wrote: > Congrats Spark community! > > On Fri, May 13, 2022 at 10:40 AM Qian Sun wrote: > >> Congratulations !!! >> >> 2022年5月13日 上午3:44，Matei Zaharia 写道： >> >> Hi all, >> >> We recently found out that

Re: [VOTE] Release Spark 3.3.0 (RC1)

me. We should have RC2 to include the fix. Thanks Gengliang On Fri, May 6, 2022 at 6:15 PM Maxim Gekk wrote: > Hi Dongjoon, > > > https://issues.apache.org/jira/projects/SPARK/versions/12350369 > > Since RC1 is started, could you move them out from the 3.3.0 milestone? > >

Re: Apache Spark 3.3 Release

g <https://github.com/apache/spark/pull/35896> - [SPARK-38590][SQL] New SQL function: try_to_binary <https://github.com/apache/spark/pull/35897> Gengliang On Thu, Mar 17, 2022 at 7:59 AM Andrew Melo wrote: > Hello, > > I've been trying for a bit to get the followin

Re: [VOTE] Spark 3.1.3 RC4

+1 (non-binding) On Wed, Feb 16, 2022 at 1:28 PM Wenchen Fan wrote: > +1 > > On Tue, Feb 15, 2022 at 3:59 PM Yuming Wang wrote: > >> +1 (non-binding). >> >> On Tue, Feb 15, 2022 at 10:22 AM Ruifeng Zheng >> wrote: >> >>> +1 (non-binding) >>> >>> checked the release script issue Dongjoon mentio

Re: [ANNOUNCE] Apache Spark 3.2.1 released

Thanks to Huaxin for driving the release! Fengyu, this is a known issue that will be fixed in the 3.3 release. Currently, the "hadoop3.2" means 3.2 or higher. See the thread https://lists.apache.org/thread/yov8xsggo3g2qr2p1rrr2xtps25wkbvj for more details. On Sat, Jan 29, 2022 at 3:26 PM FengYu

Re: [VOTE] Release Spark 3.2.1 (RC2)

+1 (non-binding) On Mon, Jan 24, 2022 at 6:26 PM Dongjoon Hyun wrote: > +1 > > Dongjoon. > > On Sat, Jan 22, 2022 at 7:19 AM Mridul Muralidharan > wrote: > >> >> +1 >> >> Signatures, digests, etc check out fine. >> Checked out tag and build/tested with -Pyarn -Pmesos -Pkubernetes >> >> Regards,

Re: [Apache Spark Jenkins] build system shutting down Dec 23th, 2021

Thanks for the works, Shane! On Wed, Dec 8, 2021 at 9:19 AM shane knapp ☠ wrote: > created an issue to track stuff: > > https://issues.apache.org/jira/browse/SPARK-37571 > > On Tue, Dec 7, 2021 at 8:25 AM shane knapp ☠ wrote: > >> Will you be nuking all the Jenkins-related code in the repo afte

Re: Time for Spark 3.2.1?

+1 for new maintenance releases for all 3.x branches as well. On Wed, Dec 8, 2021 at 8:19 AM Hyukjin Kwon wrote: > SGTM! > > On Wed, 8 Dec 2021 at 09:07, huaxin gao wrote: > >> I prefer to start rolling the release in January if there is no need to >> publish it sooner :) >> >> On Tue, Dec 7, 2

Re: [VOTE] SPIP: Row-level operations in Data Source V2

+1 (non-binding) On Tue, Nov 16, 2021 at 9:03 PM Wenchen Fan wrote: > +1 > > On Mon, Nov 15, 2021 at 2:54 AM John Zhuge wrote: > >> +1 (non-binding) >> >> On Sun, Nov 14, 2021 at 10:33 AM Chao Sun wrote: >> >>> +1 (non-binding). Thanks Anton for the work! >>> >>> On Sun, Nov 14, 2021 at 10:01

Re: Update Spark 3.3 release window?

+1, Mid-March 2022 sounds good. Gengliang On Thu, Oct 28, 2021 at 10:54 PM Tom Graves wrote: > +1 for updating, mid march sounds good. I'm also fine with EOL 2.x. > > Tom > > On Thursday, October 28, 2021, 09:37:00 AM CDT, Mridul Muralidharan < > mri...@gmail.com>

Re: [ANNOUNCE] Apache Spark 3.2.0

/spark/spark-3.2.0/spark-3.2.0-bin-hadoop3.3.tgz > > FYI, unable to download from this location. > Also, I don’t see Hadoop 3.3 version in the dist > > > On Oct 19, 2021, at 9:39 AM, Bode, Meikel, NMA-CFD < > meikel.b...@bertelsmann.de> wrote: > >  > > Many

[ANNOUNCE] Apache Spark 3.2.0

Hi all, Apache Spark 3.2.0 is the third release of the 3.x line. With tremendous contribution from the open-source community, this release managed to resolve in excess of 1,700 Jira tickets. We'd like to thank our contributors and users for their contributions and early feedback to this release.

Re: [VOTE][RESULT] Release Spark 3.2.0 (RC7)

Hi all, FYI the size of the PySpark tarball exceeds the file size limit of PyPI. I am still waiting for the issue https://github.com/pypa/pypi-support/issues/1374 to be resolved. Gengliang On Tue, Oct 12, 2021 at 3:26 PM Bode, Meikel, NMA-CFD < meikel.b...@bertelsmann.de> wrote:

[VOTE][RESULT] Release Spark 3.2.0 (RC7)

The vote passes with 28 +1s (10 binding +1s). Thanks to all who helped with the release! (* = binding) +1: - Gengliang Wang - Michael Heuer - Mridul Muralidharan * - Sean Owen * - Ruifeng Zheng - Dongjoon Hyun * - Yuming Wang - Reynold Xin * - Cheng Su - Peter Toth - Mich Talebzadeh - Maxim Gekk

Please take a look at the draft of the Spark 3.2.0 release notes

/1Wvc7K2ep96HeGFOa4gsSUDhpCTj7U7p8EVRCj8dcjM0/edit?usp=sharing Thanks Gengliang

Re: [VOTE] Release Spark 3.2.0 (RC7)

Starting with my +1(non-binding) Thanks, Gengliang On Thu, Oct 7, 2021 at 12:48 AM Gengliang Wang wrote: > Please vote on releasing the following candidate as > Apache Spark version 3.2.0. > > The vote is open until 11:59pm Pacific time October 11 and passes if a > majority +

[VOTE] Release Spark 3.2.0 (RC7)

Please vote on releasing the following candidate as Apache Spark version 3.2.0. The vote is open until 11:59pm Pacific time October 11 and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 3.2.0 [ ] -1 Do not release this package

Re: [VOTE] Release Spark 3.2.0 (RC6)

s week. Thus, I will start RC7 after their tests are completed and the known blockers are resolved, probably next week. Gengliang On Fri, Oct 1, 2021 at 2:26 AM Shardul Mahadik wrote: > I ran into https://issues.apache.org/jira/browse/SPARK-36905 when testing > on some views in our organi

Re: [VOTE] Release Spark 3.2.0 (RC6)

Starting with my +1(non-binding) Thanks, Gengliang On Tue, Sep 28, 2021 at 11:45 PM Gengliang Wang wrote: > Please vote on releasing the following candidate as > Apache Spark version 3.2.0. > > The vote is open until 11:59pm Pacific time September 30 and passes if a > majority +

[VOTE] Release Spark 3.2.0 (RC6)

Please vote on releasing the following candidate as Apache Spark version 3.2.0. The vote is open until 11:59pm Pacific time September 30 and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 3.2.0 [ ] -1 Do not release this packa

Re: [VOTE] Release Spark 3.2.0 (RC5)

OR] [Error] >>>>> /mnt/data/testing/spark-3.2.0/common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java:34: >>>>> package com.google.common.collect does not exist >>>>> ... >>>>> >>>>> I didn't see th

Re: [VOTE] Release Spark 3.2.0 (RC5)

Hi Kousuke, I tend to agree with Sean. It only affects the macOS developers when building Spark with the released Spark 3.2 code tarball without setting JAVA_HOME. I can mention this one as a known issue in the release note if this vote passes. Thanks, Gengliang On Mon, Sep 27, 2021 at 11:47 PM

  1   2   >