Yet another one for the release: FLINK-4144 https://github.com/apache/flink/pull/2191
On Fri, Jul 1, 2016 at 5:18 PM, Maximilian Michels <m...@apache.org> wrote: > There is also FLINK-4141. We just found this during testing. PR is > waiting to be merged here: https://github.com/apache/flink/pull/2190 > > On Fri, Jul 1, 2016 at 3:26 PM, Maximilian Michels <m...@apache.org> wrote: >> FLINK-3904 is not Yarn related. Not pressing to fix for this release >> and actually a bit tricky to fix. I've addressed the other issues and >> merged all pending pull requests. Good to go from my side. >> >> On Fri, Jul 1, 2016 at 11:29 AM, Chesnay Schepler <ches...@apache.org> wrote: >>> There are also 2 fixes for Cassandra that should be included: >>> https://github.com/apache/flink/pull/2167 >>> https://github.com/apache/flink/pull/2183 >>> >>> We should also include the documentation for the metrics stuff (hopefully >>> merged today) >>> https://github.com/apache/flink/pull/2158 >>> >>> In regards to metrics: To add a counter metric a user currently has to call >>> "counter(...)" on >>> a MetricGroup. The point was raised in the documentation PR that we may want >>> to give >>> them a more descriptive name like "addCounter(...)". >>> >>> I would be in favor of changing them but would like others to weigh in on >>> this. IMO we >>> should nail this down before 1.1 . >>> >>> Regards, >>> Chesnay >>> >>> >>> On 30.06.2016 22:59, Greg Hogan wrote: >>>> >>>> It would be great if hash-based combine (FLINK-3477) could make it in to >>>> be >>>> tested for this release. We've seen impressive improvements in performance >>>> (though, admittedly, some sort-based enhancements are yet to be worked >>>> on). >>>> This PR looks to be ripe. >>>> >>>> Also, as we tidy up a few things with Gelly and documentation, what is the >>>> schedule for a feature freeze and creating a 1.1 branch off master? >>>> >>>> Thanks, >>>> Greg >>>> >>>> On Mon, Jun 27, 2016 at 7:23 AM, Robert Metzger <rmetz...@apache.org> >>>> wrote: >>>> >>>>> Sure Ufuk! Thanks a lot for taking care of the release management. >>>>> I'll be on vacation in three weeks, for three weeks and I'm not sure if >>>>> we >>>>> get the release done until then. >>>>> >>>>> On Mon, Jun 27, 2016 at 12:08 PM, Ufuk Celebi <u...@apache.org> wrote: >>>>> >>>>>> I would like to do it if that's OK with you Robert. I would follow >>>>>> your suggestion and wait a few days until the following important >>>>>> fixes are in: >>>>>> - Savepoint headers and proper disposal (FLINK-4067 and >>>>>> https://github.com/apache/flink/pull/2083) >>>>>> - Metrics (https://github.com/apache/flink/pull/2146) >>>>>> - Table API time support (https://github.com/apache/flink/pull/2150) >>>>>> - Kafka at-least-once Producer ( >>>>> >>>>> https://github.com/apache/flink/pull/2108) >>>>>> >>>>>> - Cassandra connector fixes (https://github.com/apache/flink/pull/2163) >>>>>> - YARN client fixes (FLINK-3675, FLINK-3904 @Max: is there something >>>>> >>>>> else?) >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Jun 23, 2016 at 1:33 PM, Robert Metzger <rmetz...@apache.org> >>>>>> wrote: >>>>>>> >>>>>>> Hi, >>>>>>> it doesn't seem that there are volunteers for the RM, so I'll probably >>>>> >>>>> do >>>>>>> >>>>>>> it. >>>>>>> >>>>>>> I try to do the first release candidate (mostly for testing) next week >>>>>> >>>>>> (it >>>>>>> >>>>>>> depends on the JIRAs fixed by then) >>>>>>> >>>>>>> On Thu, Jun 16, 2016 at 10:56 PM, Henry Saputra < >>>>> >>>>> henry.sapu...@gmail.com >>>>>>> >>>>>>> wrote: >>>>>>> >>>>>>>> Thanks for the reply, @Max. I was not aware it was about dynamic >>>>>> >>>>>> scaling, >>>>>>>> >>>>>>>> which I think also asked for YARN support. >>>>>>>> I agree to list all related half merge JIRA for the ResourceManager. >>>>>>>> >>>>>>>> Looking forward for the Apache Mesos integration design for sure =) >>>>>>>> >>>>>>>> - Henry >>>>>>>> >>>>>>>> On Thu, Jun 16, 2016 at 2:12 AM, Maximilian Michels <m...@apache.org> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Robert, hi Henry, >>>>>>>>> >>>>>>>>> +1 for a 1.1.0 release soon! We have enough new features that >>>>> >>>>> justify >>>>>>>>> >>>>>>>>> a major release. >>>>>>>>> >>>>>>>>> @Henry We have plans to extend the ResourceManager to interact with >>>>>>>>> the Scheduler which will be a prerequisite for dynamic scaling. I >>>>>>>>> think this is out of scope for 1.1.0. The upcoming Mesos integration >>>>>>>>> won't require additional refactoring of the ResourceManager. >>>>> >>>>> Instead, >>>>>>>>> >>>>>>>>> we will create a new "Dispatcher" component that takes care of >>>>>>>>> bootstrapping the initial node with the JobManager/ResourceManager. >>>>>>>>> From there on, everything will be handled by the Mesos >>>>>>>>> ResourceManager. I recently discussed this with Eron (CC) who came >>>>> >>>>> up >>>>>>>>> >>>>>>>>> with this design and he plans to publish it to the mailing list >>>>> >>>>> soon. >>>>>>>>> >>>>>>>>> How about listing relevant JIRA issues here? "Half Merged" is kind >>>>> >>>>> of >>>>>>>>> >>>>>>>>> hard to get for people who are not involved in the different >>>>>>>>> components. >>>>>>>>> >>>>>>>>> The Cassandra adapter seems like a pretty important thing to have >>>>> >>>>> for >>>>>>>>> >>>>>>>>> the next release. In addition, I would like to merge FLINK-3667 and >>>>>>>>> FLINK-3937. Robert is doing a review at the moment :) Those are a) >>>>>>>>> refactoring of the command-line and client classes b) adding >>>>>>>>> capability to resume cluster programmatically. >>>>>>>>> >>>>>>>>> Then we should also have a look at any other critical/major bugs >>>>>> >>>>>> listed >>>>>>>> >>>>>>>> in >>>>>>>>> >>>>>>>>> JIRA. >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> Max >>>>>>>>> >>>>>>>>> On Wed, Jun 15, 2016 at 10:50 PM, Henry Saputra < >>>>>> >>>>>> henry.sapu...@gmail.com >>>>>>>>> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Hi Robert, >>>>>>>>>> >>>>>>>>>> Thanks for staying the discussion. >>>>>>>>>> >>>>>>>>>> Do you know if there any open tasks for the Resource Manager left? >>>>>>>>>> >>>>>>>>>> That is probably needed for Mesos integration? >>>>>>>>>> >>>>>>>>>> - Henry >>>>>>>>>> >>>>>>>>>> On Wed, Jun 15, 2016 at 12:55 PM, Robert Metzger < >>>>>> >>>>>> rmetz...@apache.org> >>>>>>>>>> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> Flink 1.0.0 was released early March, so three months have passed >>>>>> >>>>>> and >>>>>>>> >>>>>>>> I >>>>>>>>>>> >>>>>>>>>>> think we should start discussing the scope of the next major >>>>>> >>>>>> release >>>>>>>>>>> >>>>>>>>>>> (1.1.0). >>>>>>>>>>> >>>>>>>>>>> From a high level point of view, we've added the following new >>>>>>>> >>>>>>>> features: >>>>>>>>>>> >>>>>>>>>>> in master: >>>>>>>>>>> - Table API Refactoring, SQL, StreamSQL >>>>>>>>>>> - The metrics system >>>>>>>>>>> - Kinesis Connector >>>>>>>>>>> - Persistent file sources for streaming >>>>>>>>>>> >>>>>>>>>>> Half merged: >>>>>>>>>>> - Resource manager refactoring >>>>>>>>>>> >>>>>>>>>>> Unmerged features: >>>>>>>>>>> - Cassandra connector >>>>>>>>>>> - Key groups ("rescaling from savepoints") >>>>>>>>>>> - Queryable state >>>>>>>>>>> >>>>>>>>>>> I'm pretty sure I forgot many other features / pull requests, >>>>>> >>>>>> please >>>>>>>>> >>>>>>>>> post >>>>>>>>>>> >>>>>>>>>>> them to this thread. I'll collect them and create a Wiki page out >>>>>> >>>>>> of >>>>>>>> >>>>>>>> it. >>>>>>>>>>> >>>>>>>>>>> Some immediate TODOs for us: >>>>>>>>>>> - Which of the unmerged features are we going to add to the >>>>>> >>>>>> release? >>>>>>>>>>> >>>>>>>>>>> - Which blockers do we need to address before releasing? >>>>>>>>>>> - Are there any volunteers for the release manager? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> Robert >>>>>>>>>>> >>>
