Re: ASF board report draft for February 2025

hat spark in of it self is more of a batch centric >>>>> technology, however I strongly believe that DRA in the long lived >>>>> streaming >>>>> context is possibly even more important than DRA in batch context. >>>>> Running >>>&

Re: ASF board report draft for February 2025

long lived >>>>> streaming context is possibly even more important than DRA in batch >>>>> context. Running a large Hadoop/spark cluster 24x7 is expensive and >>>>> could really benefit from the functionality that proper streaming work >>>>>

Re: ASF board report draft for February 2025

;>> from the functionality that proper streaming work load based DRA could >>>> bring. >>>> >>>> Also, knowing that the PR author has been running this DRA code in his >>>> own environment for quite some time now successfully, makes it more

Re: ASF board report draft for February 2025

e PR author has been running this DRA code in his >>> own environment for quite some time now successfully, makes it more >>> frustrating. The code has essentially been tested externally before the PR >>> was even raised. It seems to be more than just a theoretical im

Re: ASF board report draft for February 2025

has essentially been tested externally before the PR >> was even raised. It seems to be more than just a theoretical improvement >> to the codebase. >> >> >> >> Regards, >> >> >> >> Adam Hobbs >> >> >> >> >> C2 -

Re: ASF board report draft for February 2025

oretical improvement > to the codebase. > > > > Regards, > > > > Adam Hobbs > > > > > C2 - Internal Use > From: Jungtaek Lim > *Sent:* Tuesday, 11 February 2025 8:49 AM > *To:* adam.ho...@bendigoadelaide.com.au.invalid > *Cc:* Matei Zah

RE: ASF board report draft for February 2025

theoretical improvement to the codebase. Regards, Adam Hobbs C2 - Internal Use From: Jungtaek Lim Sent: Tuesday, 11 February 2025 8:49 AM To: adam.ho...@bendigoadelaide.com.au.invalid Cc: Matei Zaharia ; Spark dev list Subject: Re: ASF board report draft for February 2025 CAUTION: This email

Re: ASF board report draft for February 2025

352). > > This seems like a bit of a failure in the process > . > Regards, > > Adam Hobbs > > > C2 - Internal Use > -Original Message- > From: Matei Zaharia > Sent: Thursday, 6 February 2025 2:57 PM > To: Spark dev list > Cc: priv...@spark.

Re: ASF board report draft for February 2025

park-wtf-i-like-it-when-a-plan-comes-together-part-i-48c52a667288> >>>>> >>>>> El jue, 6 feb 2025 a las 6:30, Adam Hobbs >>>>> () escribió: >>>>> >>>>>> I'd like to add something around the failure to get any traction o

Re: ASF board report draft for February 2025

t;>>>> shepparding of the structured streaming DRA PR. Multiple times now there >>>>> have been calls for help to get this initiative over the line and the >>>>> response has been disappointing. The github PR has been closed due to >>>>> inacti

Re: ASF board report draft for February 2025

ve been calls for help to get this initiative over the line and the >>>> response has been disappointing. The github PR has been closed due to >>>> inaction (https://github.com/apache/spark/pull/42352). >>>> >>>> This seems like a bit of a failure in t

Re: ASF board report draft for February 2025

to >>> inaction (https://github.com/apache/spark/pull/42352). >>> >>> This seems like a bit of a failure in the process >>> . >>> Regards, >>> >>> Adam Hobbs >>> >>> >>> C2 - Internal Use >>> -O

Re: ASF board report draft for February 2025

PR has been closed due to >> inaction (https://github.com/apache/spark/pull/42352). >> >> This seems like a bit of a failure in the process >> . >> Regards, >> >> Adam Hobbs >> >> >> C2 - Internal Use >> -Original Message-

Re: ASF board report draft for February 2025

Use > -Original Message- > From: Matei Zaharia > Sent: Thursday, 6 February 2025 2:57 PM > To: Spark dev list > Cc: priv...@spark.apache.org > Subject: ASF board report draft for February 2025 > > CAUTION: This email originated from outside of the organisation. Do not &g

RE: ASF board report draft for February 2025

ction (https://github.com/apache/spark/pull/42352). This seems like a bit of a failure in the process . Regards, Adam Hobbs C2 - Internal Use -Original Message- From: Matei Zaharia Sent: Thursday, 6 February 2025 2:57 PM To: Spark dev list Cc: priv...@spark.apache.org Subject: ASF board r

ASF board report draft for February 2025

It’s time to send our next ASF board report again on February 12th. Here’s an initial draft — feel free to suggest changes: = Description: Apache Spark is a fast and general purpose engine for large-scale data processing. It offers high-level APIs in Java, Scala, Python, R

Re: ASF board report draft for February

Np, thanks for addressing the point promptly Mich Talebzadeh, Dad | Technologist | Solutions Architect | Engineer London United Kingdom view my Linkedin profile https://en.everybodywiki.com/Mich_Talebzadeh *Disclaimer:* The info

Re: ASF board report draft for February

Thanks for the clarification. I updated it to say Comet is in the process of being open sourced. > On Feb 18, 2024, at 1:55 AM, Mich Talebzadeh > wrote: > > Hi Matei, > > With regard to your last point > > "- Project Comet, a plugin designed to accelerate Spark query execution by > leveragi

Re: ASF board report draft for February

Hi Matei, With regard to your last point "- Project Comet, a plugin designed to accelerate Spark query execution by leveraging DataFusion and Arrow, has been open-sourced under the Apache Arrow project. For more information, visit https://github.com/apache/arrow-datafusion-comet."; If my underst

Re: ASF board report draft for February

+1, it looks good to me. Thank you, Matei. Dongjoon On Sat, Feb 17, 2024 at 11:21 AM Matei Zaharia wrote: > Hi all, > > I missed some reminder emails about our board report this month, but here > is my draft. I’ll submit it tomorrow if that’s ok. > > == > > Issues for the board: >

ASF board report draft for February

Hi all, I missed some reminder emails about our board report this month, but here is my draft. I’ll submit it tomorrow if that’s ok. == Issues for the board: - None Project status: - We made two patch releases: Spark 3.3.4 (EOL release) on December 16, 2023, and Spark 3.4.2 on

Re: ASF board report draft for February 2022

Thanks, good idea. > On Feb 8, 2022, at 12:25 PM, Mich Talebzadeh > wrote: > > Hi, > > I believe it would be beneficial to provide the links to SPIPs mentioned in > the report > > - Two Spark Project Improvement Proposals (SPIPs) were recently accepted by > the community: namely; 1) Suppor

Re: ASF board report draft for February 2022

Hi, I believe it would be beneficial to provide the links to SPIPs mentioned in the report - Two Spark Project Improvement Proposals (SPIPs) were recently accepted by the community: namely; 1) Support for Customized Kubernetes Schedulers

ASF board report draft for February 2022

It’s time to send our quarterly report to the ASF board again this Wednesday. I’ve written the following draft for it — let me know if you want to add or change anything. == Description: Apache Spark is a fast and general purpose engine for large-scale data

ASF board report draft for February

Hi all, Our project board report needs to be submitted on Feb 12th, and I wanted to include anything notable going on that we want to appear in the board archive. Here is my draft below — let me know if you have suggestions to add or change things. =