Re: [VOTE] Single-pass Analyzer for Catalyst

+1 on the assumption that we should phase this release on an incremental basis. Probably will take us to end of release 5. HTH Mich Talebzadeh, Architect | Data Engineer | Data Science | Financial Crime PhD Imperial College London

Re: [VOTE] Single-pass Analyzer for Catalyst

+1 El jue, 3 oct 2024, 20:06, Wenchen Fan escribió: > +1 > > On Wed, Oct 2, 2024 at 7:50 AM Peter Toth wrote: > >> +1 >> >> On Tue, Oct 1, 2024, 08:33 Yang Jie wrote: >> >>> +1, Thanks >>> >>> Jie Yang >>> >>> On 2024/10/01 03:26:40 John Zhuge wrote: >>> > +1 (non-binding) >>> > >>> > On Mon,

Re: [VOTE] Single-pass Analyzer for Catalyst

Hi Mich! Thank you for this input. Yes, this is exactly the approach I would propose too. Putting the new analyzer under a flag and making the tests pass for both implementations is crucial. We need to compare the logical (analyzed) plans and ensure that they are identical. On Thu, Oct 3, 2024 at

Re: [VOTE] Single-pass Analyzer for Catalyst

With a project manager hat on and having read the SPIP This proposed single-pass Analyzer framework does potentially offer significant long-term benefits in terms of efficiency, maintenance, and stability, especially for large or complex queries.

Re: [VOTE] Single-pass Analyzer for Catalyst

+1 On Thu, Oct 3, 2024 at 7:31 AM Wenchen Fan wrote: > > +1 > > On Wed, Oct 2, 2024 at 7:50 AM Peter Toth wrote: >> >> +1 >> >> >> On Tue, Oct 1, 2024, 08:33 Yang Jie wrote: >>> >>> +1, Thanks >>> >>> Jie Yang >>> >>> On 2024/10/01 03:26:40 John Zhuge wrote: >>> > +1 (non-binding) >>> > >>> > O

Re: [VOTE] Single-pass Analyzer for Catalyst

+1 On Wed, Oct 2, 2024 at 7:50 AM Peter Toth wrote: > +1 > > On Tue, Oct 1, 2024, 08:33 Yang Jie wrote: > >> +1, Thanks >> >> Jie Yang >> >> On 2024/10/01 03:26:40 John Zhuge wrote: >> > +1 (non-binding) >> > >> > On Mon, Sep 30, 2024 at 7:42 PM Gengliang Wang >> > wrote: >> > >> > > +1 >> > >

Re: [VOTE] Single-pass Analyzer for Catalyst

+1 On Tue, Oct 1, 2024, 08:33 Yang Jie wrote: > +1, Thanks > > Jie Yang > > On 2024/10/01 03:26:40 John Zhuge wrote: > > +1 (non-binding) > > > > On Mon, Sep 30, 2024 at 7:42 PM Gengliang Wang > > wrote: > > > > > +1 > > > > > > On Mon, Sep 30, 2024 at 6:22 PM Jungtaek Lim < > kabhwan.opensou..

Re: [VOTE] Single-pass Analyzer for Catalyst

+1, Thanks Jie Yang On 2024/10/01 03:26:40 John Zhuge wrote: > +1 (non-binding) > > On Mon, Sep 30, 2024 at 7:42 PM Gengliang Wang > wrote: > > > +1 > > > > On Mon, Sep 30, 2024 at 6:22 PM Jungtaek Lim > > wrote: > > > >> +1 (non-binding), promising proposal! > >> > >> 2024년 10월 1일 (화) 오전 8:0

Re: [VOTE] Single-pass Analyzer for Catalyst

+1, Thanks Jie Yang On 2024/10/01 03:26:40 John Zhuge wrote: > +1 (non-binding) > > On Mon, Sep 30, 2024 at 7:42 PM Gengliang Wang > wrote: > > > +1 > > > > On Mon, Sep 30, 2024 at 6:22 PM Jungtaek Lim > > wrote: > > > >> +1 (non-binding), promising proposal! > >> > >> 2024년 10월 1일 (화) 오전 8:0

Re: [VOTE] Single-pass Analyzer for Catalyst

Folks, thanks for voting! Yes, the target version was set by my mistake, Dongjoon. Sorry for that. Vladimir. On Tue, Oct 1, 2024, 05:28 John Zhuge wrote: > +1 (non-binding) > > On Mon, Sep 30, 2024 at 7:42 PM Gengliang Wang > wrote: > >> +1 >> >> On Mon, Sep 30, 2024 at 6:22 PM Jungtaek Lim <

Re: [VOTE] Single-pass Analyzer for Catalyst

+1 (non-binding) On Mon, Sep 30, 2024 at 7:42 PM Gengliang Wang wrote: > +1 > > On Mon, Sep 30, 2024 at 6:22 PM Jungtaek Lim > wrote: > >> +1 (non-binding), promising proposal! >> >> 2024년 10월 1일 (화) 오전 8:04, Dongjoon Hyun 님이 작성: >> >>> Thank you for the swift clarification, Reynold and Xiao. >

Re: [VOTE] Single-pass Analyzer for Catalyst

+1 Regards, Mridul On Mon, Sep 30, 2024 at 5:39 PM Reynold Xin wrote: > I don't actually "lead" this. But I don't think this needs to target a > specific Spark version given it should not have any user facing > consequences? > > > On Mon, Sep 30, 2024 at 3:36 PM Dongjoon Hyun wrote: > >> Thank

Re: [VOTE] Single-pass Analyzer for Catalyst

+1 On Mon, Sep 30, 2024 at 6:22 PM Jungtaek Lim wrote: > +1 (non-binding), promising proposal! > > 2024년 10월 1일 (화) 오전 8:04, Dongjoon Hyun 님이 작성: > >> Thank you for the swift clarification, Reynold and Xiao. >> >> It seems that the Target Version was set mistakenly initially. >> >> I removed the

Re: [VOTE] Single-pass Analyzer for Catalyst

+1 (non-binding), promising proposal! 2024년 10월 1일 (화) 오전 8:04, Dongjoon Hyun 님이 작성: > Thank you for the swift clarification, Reynold and Xiao. > > It seems that the Target Version was set mistakenly initially. > > I removed the `Target Version` from the SPIP JIRA. > > https://issues.apache.org/j

Re: [VOTE] Single-pass Analyzer for Catalyst

Thank you for the swift clarification, Reynold and Xiao. It seems that the Target Version was set mistakenly initially. I removed the `Target Version` from the SPIP JIRA. https://issues.apache.org/jira/browse/SPARK-49834 I'm switching my cast to +1 for this SPIP vote. Thanks, Dongjoon. On 202

Re: [VOTE] Single-pass Analyzer for Catalyst

+1 in support of the direction of the Single-pass Analyzer for Catalyst. I think we should not have a target version for the new Catalyst SPARK-49834 . It should not be a blocker for Spark 4.0. When implementing the new analyzer, the code changes

Re: [VOTE] Single-pass Analyzer for Catalyst

I don't actually "lead" this. But I don't think this needs to target a specific Spark version given it should not have any user facing consequences? On Mon, Sep 30, 2024 at 3:36 PM Dongjoon Hyun wrote: > Thank you for leading this, Vladimir, Reynold, Herman. > > I'm wondering if this is really

Re: [VOTE] Single-pass Analyzer for Catalyst

Thank you for leading this, Vladimir, Reynold, Herman. I'm wondering if this is really achievable goal for Apache Spark 4.0.0. If it's expected that we are unable to deliver it, shall we postpone this vote until 4.1.0 planning? Anyway, since SPARK-49834 has a target version 4.0.0 explicitly,

Re: [VOTE] Single-pass Analyzer for Catalyst

+1 On Mon, Sep 30, 2024 at 8:29 AM Reynold Xin wrote: > +1 > > On Mon, Sep 30, 2024 at 6:47 AM Vladimir Golubev > wrote: > >> Hi all, >> >> I’d like to start a vote for a single-pass Analyzer for the Catalyst >> project. This project will introduce a new analysis framework to the >> Catalyst, w

Re: [VOTE] Single-pass Analyzer for Catalyst

+1 On Mon, Sep 30, 2024 at 6:47 AM Vladimir Golubev wrote: > Hi all, > > I’d like to start a vote for a single-pass Analyzer for the Catalyst > project. This project will introduce a new analysis framework to the > Catalyst, which will eventually replace the fixed-point one. > > Please refer to

[VOTE] Single-pass Analyzer for Catalyst

Hi all, I’d like to start a vote for a single-pass Analyzer for the Catalyst project. This project will introduce a new analysis framework to the Catalyst, which will eventually replace the fixed-point one. Please refer to the SPIP jira: https://issues.apache.org/jira/browse/SPARK-49834 [ ] +1: