Hi Gordon,

Thanks for mention the feature freeze date for 1.9.0, that's very helpful
for contributors to evaluate their dev plan!

Regarding FLIP-29, we are glad to do our best to finish the dev of FLIP-29,
then catch up with the release of 1.9.

Thanks again for push the release of 1.9.0 forward!

Cheers,
Jincheng



Tzu-Li (Gordon) Tai <tzuli...@apache.org> 于2019年5月27日周一 下午5:48写道:

> Hi all,
>
> I want to kindly remind the community that we're now 5 weeks away from the
> proposed feature freeze date for 1.9.0, which is June 28.
>
> This is not yet a final date we have agreed on, so I would like to start
> collecting feedback on how the mentioned features are going, and in
> general, whether or not the date sounds reasonable given the current status
> of the ongoing efforts.
> Please let me know what you think!
>
> Cheers,
> Gordon
>
>
> On Mon, May 27, 2019 at 5:40 PM Tzu-Li (Gordon) Tai <tzuli...@apache.org>
> wrote:
>
> > @Hequn @Jincheng
> >
> > Thanks for bringing up FLIP-29 to attention.
> > As previously mentioned, the original list is not a fixed feature set, so
> > if FLIP-29 has ongoing efforts and can make it before the feature freeze,
> > then of course it should be included!
> >
> > @himansh1306
> >
> > Concerning the ORC format for StreamingFileSink, is there already a JIRA
> > ticket tracking that? If not, I suggest to first open one and see if
> there
> > are similar interests from committers in adding that.
> >
> >
> > On Sun, May 5, 2019 at 11:19 PM Hequn Cheng <chenghe...@gmail.com>
> wrote:
> >
> >> Hi,
> >>
> >> Great job, Gordon! Thanks a lot for driving this and wrapping features
> up
> >> to a detailed list. +1 on it!
> >>
> >> Would be great if we can also add flip29 to the list. @jincheng sun
> >> <sunjincheng...@gmail.com>  and I are focusing on it these days. I
> think
> >> these features in flip29 would bring big enhancements to the Table API.
> >> :-)
> >>
> >> Best, Hequn
> >>
> >> On Sun, May 5, 2019 at 10:41 PM Becket Qin <becket....@gmail.com>
> wrote:
> >>
> >> > Thanks for driving this release, Gordon. +1 on the feature list.
> >> >
> >> > This is a pretty exciting and ambitious release!
> >> >
> >> > Cheers,
> >> >
> >> > Jiangjie (Becket) Qin
> >> >
> >> > On Sun, May 5, 2019 at 4:28 PM jincheng sun <sunjincheng...@gmail.com
> >
> >> > wrote:
> >> >
> >> > > Thanks a lot for being our release manager, Great job!
> >> > >
> >> > > +1 for the feature list and It's better to add FLIP-29
> >> > > <
> >> >
> >>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=97552739
> >> > > >(Support
> >> > > map/flatMap/aggregate/flatAggregate on TableAPI), as the goals of
> >> release
> >> > > 1.9.
> >> > >
> >> > > What do you think?
> >> > >
> >> > > Best,
> >> > > Jincheng
> >> > >
> >> > > Bowen Li <bowenl...@gmail.com> 于2019年5月5日周日 上午12:47写道:
> >> > >
> >> > > > +1, exciting and ambitious goals, the rough timeline looks
> >> reasonable.
> >> > > > Let's make it happen!
> >> > > >
> >> > > > On Sat, May 4, 2019 at 2:47 AM Jark Wu <imj...@gmail.com> wrote:
> >> > > >
> >> > > > > +1 for the 1.9.0 feature list. Excited to see it is happening.
> >> > > > >
> >> > > > > Regards,
> >> > > > > Jark
> >> > > > >
> >> > > > > On Thu, 2 May 2019 at 17:07, himansh1...@gmail.com <
> >> > > > himansh1...@gmail.com>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > +1 for Protobuf, Hive Metastore integration & features related
> >> to
> >> > > > > > savepoint.
> >> > > > > >
> >> > > > > > I was hoping if we could support for ORC File format can be
> >> added
> >> > in
> >> > > > > > StreamingFileSink Writer, Currently only Parquet is supported
> >> with
> >> > > > > respect
> >> > > > > > to columnar file formats
> >> > > > > >
> >> > > > > >
> >> > > > > > On 2019/05/01 05:15:23, "Tzu-Li (Gordon) Tai" <
> >> tzuli...@apache.org
> >> > >
> >> > > > > > wrote:
> >> > > > > > > Hi community,
> >> > > > > > >
> >> > > > > > > Apache Flink 1.8.0 has been released a few weeks ago, so
> >> > naturally,
> >> > > > > it’s
> >> > > > > > > time to start thinking about what we want to aim for 1.9.0.
> >> > > > > > >
> >> > > > > > > Kurt and I had collected some features that would be
> >> reasonable
> >> > to
> >> > > > > > consider
> >> > > > > > > including for the next release, based on talking with
> various
> >> > > people
> >> > > > as
> >> > > > > > > well as observations from mailing list discussions and
> >> questions.
> >> > > > > > >
> >> > > > > > > Note that having specific features listed here does not mean
> >> that
> >> > > no
> >> > > > > > other
> >> > > > > > > pull requests or topics will be reviewed. I am sure that
> there
> >> > are
> >> > > > > other
> >> > > > > > > ongoing efforts that we missed here and will likely make it
> >> as an
> >> > > > > > > improvement or new feature in the next release. This
> >> discussion
> >> > is
> >> > > > > merely
> >> > > > > > > for bootstrapping a discussion for 1.9, as well as to give
> >> > > > contributors
> >> > > > > > an
> >> > > > > > > idea of what the community is looking to focus on in the
> next
> >> > > couple
> >> > > > of
> >> > > > > > > weeks.
> >> > > > > > >
> >> > > > > > > *Proposed features and focus*
> >> > > > > > >
> >> > > > > > > In the previous major release, Apache Flink 1.8.0, the
> >> community
> >> > > had
> >> > > > > > > prepared for some major Table & SQL additions from the Blink
> >> > > branch.
> >> > > > > With
> >> > > > > > > this in mind, for the next release, it would be great to
> wind
> >> up
> >> > > > those
> >> > > > > > > efforts by merging in the Blink-based Table / SQL planner
> and
> >> > > runtime
> >> > > > > for
> >> > > > > > > 1.9.
> >> > > > > > >
> >> > > > > > > Following Stephan’s previous thread [1] in the mailing list
> >> about
> >> > > > > > features
> >> > > > > > > in Blink, we should also start focusing on preparing for
> >> Blink’s
> >> > > > other
> >> > > > > > > several enhancements for batch execution. This includes
> >> resource
> >> > > > > > > optimization, fine-grained failover, pluggable shuffle
> >> service,
> >> > > > > adapting
> >> > > > > > > stream operators for batch execution, as well as better
> >> > integration
> >> > > > > with
> >> > > > > > > commonly used systems by batch executions such as Apache
> Hive.
> >> > > > > > >
> >> > > > > > > Moreover, besides efforts related to the Blink merge, we
> would
> >> > also
> >> > > > > like
> >> > > > > > us
> >> > > > > > > to work towards pushing forward some of the most discussed
> and
> >> > > > > > anticipated
> >> > > > > > > features by the community. Most of these had discussions in
> >> the
> >> > > > mailing
> >> > > > > > > lists that span multiple releases, and are also frequently
> >> > brought
> >> > > up
> >> > > > > in
> >> > > > > > > community events such as Flink Forward. This includes
> features
> >> > such
> >> > > > as
> >> > > > > > > source event-time alignment and the source interface
> rework, a
> >> > > > > savepoint
> >> > > > > > > connector that allows users to manipulate and query state in
> >> > > > > savepoints,
> >> > > > > > > interactive programming, as well as terminating a job with a
> >> > final
> >> > > > > > > savepoint.
> >> > > > > > >
> >> > > > > > > Last but not least, we have several existing contributions
> or
> >> > > > > discussions
> >> > > > > > > for the ecosystem surrounding Flink, which we think is also
> >> very
> >> > > > > valuable
> >> > > > > > > to try to merge in for 1.9. This includes a web UI rework
> >> > (recently
> >> > > > > > already
> >> > > > > > > merged), active K8s integration, Google PubSub connector,
> >> native
> >> > > > > support
> >> > > > > > > for the Protobuf format, Python support in the Table API, as
> >> well
> >> > > as
> >> > > > > > > reworking Flink’s support for machine learning.
> >> > > > > > >
> >> > > > > > > To wrap this up as a list of items, some of which already
> have
> >> > > JIRAs
> >> > > > or
> >> > > > > > > mailing list threads to track them:
> >> > > > > > >
> >> > > > > > >    - Merge Blink runner for Table & SQL [2]
> >> > > > > > >    -
> >> > > > > > >
> >> > > > > > >       Restructure flink-table to separate API from core
> >> runtime
> >> > > > > > >       -
> >> > > > > > >
> >> > > > > > >       Make table planners pluggable
> >> > > > > > >       -
> >> > > > > > >
> >> > > > > > >       Rework Table / SQL type system to integrate better
> with
> >> the
> >> > > SQL
> >> > > > > > >       standard [3]
> >> > > > > > >       -
> >> > > > > > >
> >> > > > > > >       Merge Blink planner and runtime for Table / SQL
> >> > > > > > >       - Further preparations for more batch execution
> >> > optimization
> >> > > > from
> >> > > > > > >    Blink
> >> > > > > > >    -
> >> > > > > > >
> >> > > > > > >       Dedicated scheduler component [4]
> >> > > > > > >       -
> >> > > > > > >
> >> > > > > > >       Fine grained failover for batch [5]
> >> > > > > > >       -
> >> > > > > > >
> >> > > > > > >       Selectable input stream operator [6]
> >> > > > > > >       -
> >> > > > > > >
> >> > > > > > >       Pluggable Shuffle Service [7]
> >> > > > > > >       -
> >> > > > > > >
> >> > > > > > >       FLIP-30: Unified Catalog API & Hive metastore
> >> integration
> >> > [8]
> >> > > > > > >       - Heavily anticipated / discussed features in the
> >> community
> >> > > > > > >    -
> >> > > > > > >
> >> > > > > > >       FLIP-27: Source interface rework [9]
> >> > > > > > >       -
> >> > > > > > >
> >> > > > > > >       Savepoint connector [10]
> >> > > > > > >       -
> >> > > > > > >
> >> > > > > > >       FLIP-34: Terminate / Suspend job with savepoint [11]
> >> > > > > > >       -
> >> > > > > > >
> >> > > > > > >       FLIP-36: Interactive Programming [12]
> >> > > > > > >       - Ecosystem
> >> > > > > > >    -
> >> > > > > > >
> >> > > > > > >       Web UI rework [13]
> >> > > > > > >       -
> >> > > > > > >
> >> > > > > > >       Active K8s integration [14]
> >> > > > > > >       -
> >> > > > > > >
> >> > > > > > >       Google PubSub connector [15]
> >> > > > > > >       -
> >> > > > > > >
> >> > > > > > >       First-class Protobuf support [16]
> >> > > > > > >       -
> >> > > > > > >
> >> > > > > > >       FLIP-38: Python support in Table API [17]
> >> > > > > > >       -
> >> > > > > > >
> >> > > > > > >       FLIP-39: Flink ML pipeline and libraries on top of
> Table
> >> > API
> >> > > > [18]
> >> > > > > > >
> >> > > > > > > *Suggested release timeline*
> >> > > > > > >
> >> > > > > > > Apache Flink 1.8.0 was released earlier this month, so based
> >> on
> >> > our
> >> > > > > usual
> >> > > > > > > timely release schedule, we should aim for releasing 1.9.0
> >> around
> >> > > mid
> >> > > > > to
> >> > > > > > > end July.
> >> > > > > > >
> >> > > > > > > Since it seems that this is going to be a fairly large
> >> release,
> >> > to
> >> > > > give
> >> > > > > > the
> >> > > > > > > community enough testing time, I propose that the feature
> >> freeze
> >> > to
> >> > > > be
> >> > > > > > near
> >> > > > > > > the end of June (8-9 weeks from now, probable June 28). This
> >> is
> >> > of
> >> > > > > > course a
> >> > > > > > > ballpark estimation for now; we should follow-up with a
> >> separate
> >> > > > thread
> >> > > > > > > later in the release cycle to prepare contributors with an
> >> > official
> >> > > > > > feature
> >> > > > > > > freeze date.
> >> > > > > > >
> >> > > > > > > I’d also like to use this opportunity to propose myself and
> >> Kurt
> >> > as
> >> > > > the
> >> > > > > > > release managers for 1.9.
> >> > > > > > > AFAIK, we did not used to have 2 RMs for a single release in
> >> the
> >> > > > past,
> >> > > > > > but
> >> > > > > > > 1.9.0 is definitely quite ambitious so it would not hurt to
> >> have
> >> > > one
> >> > > > > more
> >> > > > > > > on board :) Cheers, Gordon [1]
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Flink-1-6-features-td22632.html
> >> > > > > > >
> >> > > > > > > [2] https://issues.apache.org/jira/browse/FLINK-11439
> >> > > > > > >
> >> > > > > > > [3] https://issues.apache.org/jira/browse/FLINK-12251
> >> > > > > > >
> >> > > > > > > [4] https://issues.apache.org/jira/browse/FLINK-10429
> >> > > > > > >
> >> > > > > > > [5]
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Backtracking-for-failover-regions-td28293.html
> >> > > > > > >
> >> > > > > > > [6]
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Enhance-Operator-API-to-Support-Dynamically-Selective-Reading-and-EndOfInput-Event-td26753.html
> >> > > > > > >
> >> > > > > > > [7] https://issues.apache.org/jira/browse/FLINK-10653
> >> > > > > > >
> >> > > > > > > [8] https://issues.apache.org/jira/browse/FLINK-11275
> >> > > > > > >
> >> > > > > > > [9]
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-27-Refactor-Source-Interface-td24952i20.html
> >> > > > > > >
> >> > > > > > > [10] https://issues.apache.org/jira/browse/FLINK-12047
> >> > > > > > >
> >> > > > > > > [11]
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-33-Terminate-Suspend-Job-with-Savepoint-td26927.html
> >> > > > > > >
> >> > > > > > > [12]
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-36%3A+Support+Interactive+Programming+in+Flink
> >> > > > > > >
> >> > > > > > > [13] https://issues.apache.org/jira/browse/FLINK-10705
> >> > > > > > >
> >> > > > > > > [14] https://issues.apache.org/jira/browse/FLINK-9953
> >> > > > > > >
> >> > > > > > > [15] https://issues.apache.org/jira/browse/FLINK-9311
> >> > > > > > >
> >> > > > > > > [16] https://issues.apache.org/jira/browse/FLINK-11333
> >> > > > > > >
> >> > > > > > > [17]
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-38-Support-python-language-in-flink-TableAPI-td28061.html
> >> > > > > > > [18]
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-39-Flink-ML-pipeline-and-ML-libs-td28633.html
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>

Reply via email to