It would be great if hash-based combine (FLINK-3477) could make it in to be tested for this release. We've seen impressive improvements in performance (though, admittedly, some sort-based enhancements are yet to be worked on). This PR looks to be ripe.
Also, as we tidy up a few things with Gelly and documentation, what is the schedule for a feature freeze and creating a 1.1 branch off master? Thanks, Greg On Mon, Jun 27, 2016 at 7:23 AM, Robert Metzger <rmetz...@apache.org> wrote: > Sure Ufuk! Thanks a lot for taking care of the release management. > I'll be on vacation in three weeks, for three weeks and I'm not sure if we > get the release done until then. > > On Mon, Jun 27, 2016 at 12:08 PM, Ufuk Celebi <u...@apache.org> wrote: > > > I would like to do it if that's OK with you Robert. I would follow > > your suggestion and wait a few days until the following important > > fixes are in: > > - Savepoint headers and proper disposal (FLINK-4067 and > > https://github.com/apache/flink/pull/2083) > > - Metrics (https://github.com/apache/flink/pull/2146) > > - Table API time support (https://github.com/apache/flink/pull/2150) > > - Kafka at-least-once Producer ( > https://github.com/apache/flink/pull/2108) > > - Cassandra connector fixes (https://github.com/apache/flink/pull/2163) > > - YARN client fixes (FLINK-3675, FLINK-3904 @Max: is there something > else?) > > > > > > > > > > On Thu, Jun 23, 2016 at 1:33 PM, Robert Metzger <rmetz...@apache.org> > > wrote: > > > Hi, > > > it doesn't seem that there are volunteers for the RM, so I'll probably > do > > > it. > > > > > > I try to do the first release candidate (mostly for testing) next week > > (it > > > depends on the JIRAs fixed by then) > > > > > > On Thu, Jun 16, 2016 at 10:56 PM, Henry Saputra < > henry.sapu...@gmail.com > > > > > > wrote: > > > > > >> Thanks for the reply, @Max. I was not aware it was about dynamic > > scaling, > > >> which I think also asked for YARN support. > > >> I agree to list all related half merge JIRA for the ResourceManager. > > >> > > >> Looking forward for the Apache Mesos integration design for sure =) > > >> > > >> - Henry > > >> > > >> On Thu, Jun 16, 2016 at 2:12 AM, Maximilian Michels <m...@apache.org> > > >> wrote: > > >> > > >> > Hi Robert, hi Henry, > > >> > > > >> > +1 for a 1.1.0 release soon! We have enough new features that > justify > > >> > a major release. > > >> > > > >> > @Henry We have plans to extend the ResourceManager to interact with > > >> > the Scheduler which will be a prerequisite for dynamic scaling. I > > >> > think this is out of scope for 1.1.0. The upcoming Mesos integration > > >> > won't require additional refactoring of the ResourceManager. > Instead, > > >> > we will create a new "Dispatcher" component that takes care of > > >> > bootstrapping the initial node with the JobManager/ResourceManager. > > >> > From there on, everything will be handled by the Mesos > > >> > ResourceManager. I recently discussed this with Eron (CC) who came > up > > >> > with this design and he plans to publish it to the mailing list > soon. > > >> > > > >> > How about listing relevant JIRA issues here? "Half Merged" is kind > of > > >> > hard to get for people who are not involved in the different > > >> > components. > > >> > > > >> > The Cassandra adapter seems like a pretty important thing to have > for > > >> > the next release. In addition, I would like to merge FLINK-3667 and > > >> > FLINK-3937. Robert is doing a review at the moment :) Those are a) > > >> > refactoring of the command-line and client classes b) adding > > >> > capability to resume cluster programmatically. > > >> > > > >> > Then we should also have a look at any other critical/major bugs > > listed > > >> in > > >> > JIRA. > > >> > > > >> > Cheers, > > >> > Max > > >> > > > >> > On Wed, Jun 15, 2016 at 10:50 PM, Henry Saputra < > > henry.sapu...@gmail.com > > >> > > > >> > wrote: > > >> > > Hi Robert, > > >> > > > > >> > > Thanks for staying the discussion. > > >> > > > > >> > > Do you know if there any open tasks for the Resource Manager left? > > >> > > > > >> > > That is probably needed for Mesos integration? > > >> > > > > >> > > - Henry > > >> > > > > >> > > On Wed, Jun 15, 2016 at 12:55 PM, Robert Metzger < > > rmetz...@apache.org> > > >> > > wrote: > > >> > > > > >> > >> Hi, > > >> > >> > > >> > >> Flink 1.0.0 was released early March, so three months have passed > > and > > >> I > > >> > >> think we should start discussing the scope of the next major > > release > > >> > >> (1.1.0). > > >> > >> > > >> > >> From a high level point of view, we've added the following new > > >> features: > > >> > >> in master: > > >> > >> - Table API Refactoring, SQL, StreamSQL > > >> > >> - The metrics system > > >> > >> - Kinesis Connector > > >> > >> - Persistent file sources for streaming > > >> > >> > > >> > >> Half merged: > > >> > >> - Resource manager refactoring > > >> > >> > > >> > >> Unmerged features: > > >> > >> - Cassandra connector > > >> > >> - Key groups ("rescaling from savepoints") > > >> > >> - Queryable state > > >> > >> > > >> > >> I'm pretty sure I forgot many other features / pull requests, > > please > > >> > post > > >> > >> them to this thread. I'll collect them and create a Wiki page out > > of > > >> it. > > >> > >> > > >> > >> Some immediate TODOs for us: > > >> > >> - Which of the unmerged features are we going to add to the > > release? > > >> > >> - Which blockers do we need to address before releasing? > > >> > >> - Are there any volunteers for the release manager? > > >> > >> > > >> > >> > > >> > >> Regards, > > >> > >> Robert > > >> > >> > > >> > > > >> > > >
