Re: Apache Spark 3.2 Expectation

[Dongjoon Hyun]

Thank you for volunteering, Gengliang.

Apache Spark 3.2.0 is the first version enabling AQE by default. I'm also
watching some on-going improvements on that.

    https://issues.apache.org/jira/browse/SPARK-33828 (SQL Adaptive Query
Execution QA)

To Liang-Chi, I'm -1 for postponing the branch cut because this is a soft
cut and the committers still are able to commit to `branch-3.3` according
to their decisions.

Given that Apache Spark had 115 commits in a week in various areas
concurrently, we should start QA for Apache Spark 3.2 by creating
branch-3.3 and allowing only limited backporting.

    https://github.com/apache/spark/graphs/commit-activity

Bests,
Dongjoon.


On Wed, Jun 16, 2021 at 9:19 AM Liang-Chi Hsieh <vii...@gmail.com> wrote:

> First, thanks for being volunteer as the release manager of Spark 3.2.0,
> Gengliang!
>
> And yes, for the two important Structured Streaming features, RocksDB
> StateStore and session window, we're working on them and expect to have
> them
> in the new release.
>
> So I propose to postpone the branch cut date.
>
> Thank you!
>
> Liang-Chi
>
>
> Gengliang Wang-2 wrote
> > Thanks, Hyukjin.
> >
> > The expected target branch cut date of Spark 3.2 is *July 1st* on
> > https://spark.apache.org/versioning-policy.html. However, I notice that
> > there are still multiple important projects in progress now:
> >
> > [Core]
> >
> >    - SPIP: Support push-based shuffle to improve shuffle efficiency
> >    &lt;https://issues.apache.org/jira/browse/SPARK-30602&gt;
> >
> > [SQL]
> >
> >    - Support ANSI SQL INTERVAL types
> >    &lt;https://issues.apache.org/jira/browse/SPARK-27790&gt;
> >    - Support Timestamp without time zone data type
> >    &lt;https://issues.apache.org/jira/browse/SPARK-35662&gt;
> >    - Aggregate (Min/Max/Count) push down for Parquet
> >    &lt;https://issues.apache.org/jira/browse/SPARK-34952&gt;
> >
> > [Streaming]
> >
> >    - EventTime based sessionization (session window)
> >    &lt;https://issues.apache.org/jira/browse/SPARK-10816&gt;
> >    - Add RocksDB StateStore as external module
> >    &lt;https://issues.apache.org/jira/browse/SPARK-34198&gt;
> >
> >
> > I wonder whether we should postpone the branch cut date.
> > cc Min Shen, Yi Wu, Max Gekk, Huaxin Gao, Jungtaek Lim, Yuanjian
> > Li, Liang-Chi Hsieh, who work on the projects above.
> >
> > On Tue, Jun 15, 2021 at 4:34 PM Hyukjin Kwon &lt;
>
> > gurwls223@
>
> > &gt; wrote:
> >
> >> +1, thanks.
> >>
> >> On Tue, 15 Jun 2021, 16:17 Gengliang Wang, &lt;
>
> > ltnwgl@
>
> > &gt; wrote:
> >>
> >>> Hi,
> >>>
> >>> As the expected release date is close,  I would like to volunteer as
> the
> >>> release manager for Apache Spark 3.2.0.
> >>>
> >>> Thanks,
> >>> Gengliang
> >>>
>
>
>
>
>
> --
> Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>