Re: Re: [DISCUSS][SPIP] Subexpression elimination supporting more physical operators

2023-03-08 Thread Adam Binford
I'm +1 for anything that adds support for subexpression elimination to conditional expressions, I've been trying to get that support in for a long time https://github.com/apache/spark/pull/32987. This is a major performance limitation in Spark in my opinion. On Tue, Mar 7, 2023 at 4:24 PM Kun Wan

Re:Re: [DISCUSS][SPIP] Subexpression elimination supporting more physical operators

2023-03-07 Thread Kun Wan
Hi, Mich Thanks for your reminder. It seems "Support for subexpression elimination with more physical operators" should not be a SPIP, it's just a design doc. I have changed the title of the document. Hope someone who are interested will join in and complete it together. At 2023-03-08 01:50:

Re: [DISCUSS][SPIP] Subexpression elimination supporting more physical operators

2023-03-07 Thread Mich Talebzadeh
Hi Kun, Have you checked the doc procedure for SPIP here in case Spark Project Improvement Proposals (SPIP) | Apache Spark HT view my Linkedin profile https://en.everybodyw

[DISCUSS][SPIP] Subexpression elimination supporting more physical operators

2023-03-07 Thread 万昆
Hi everyone, I would like to start a discussion on "Subexpression elimination supporting more physical operators" Now spark only support subexpression elimination in ProjectExec and AggregateExec. We can support improve subexpression elimination framework to support more physical operato