There are also 2 fixes for Cassandra that should be included:
https://github.com/apache/flink/pull/2167
https://github.com/apache/flink/pull/2183
We should also include the documentation for the metrics stuff
(hopefully merged today)
https://github.com/apache/flink/pull/2158
In regards to metrics: To add a counter metric a user currently has to
call "counter(...)" on
a MetricGroup. The point was raised in the documentation PR that we may
want to give
them a more descriptive name like "addCounter(...)".
I would be in favor of changing them but would like others to weigh in
on this. IMO we
should nail this down before 1.1 .
Regards,
Chesnay
On 30.06.2016 22:59, Greg Hogan wrote:
It would be great if hash-based combine (FLINK-3477) could make it in to be
tested for this release. We've seen impressive improvements in performance
(though, admittedly, some sort-based enhancements are yet to be worked on).
This PR looks to be ripe.
Also, as we tidy up a few things with Gelly and documentation, what is the
schedule for a feature freeze and creating a 1.1 branch off master?
Thanks,
Greg
On Mon, Jun 27, 2016 at 7:23 AM, Robert Metzger <rmetz...@apache.org> wrote:
Sure Ufuk! Thanks a lot for taking care of the release management.
I'll be on vacation in three weeks, for three weeks and I'm not sure if we
get the release done until then.
On Mon, Jun 27, 2016 at 12:08 PM, Ufuk Celebi <u...@apache.org> wrote:
I would like to do it if that's OK with you Robert. I would follow
your suggestion and wait a few days until the following important
fixes are in:
- Savepoint headers and proper disposal (FLINK-4067 and
https://github.com/apache/flink/pull/2083)
- Metrics (https://github.com/apache/flink/pull/2146)
- Table API time support (https://github.com/apache/flink/pull/2150)
- Kafka at-least-once Producer (
https://github.com/apache/flink/pull/2108)
- Cassandra connector fixes (https://github.com/apache/flink/pull/2163)
- YARN client fixes (FLINK-3675, FLINK-3904 @Max: is there something
else?)
On Thu, Jun 23, 2016 at 1:33 PM, Robert Metzger <rmetz...@apache.org>
wrote:
Hi,
it doesn't seem that there are volunteers for the RM, so I'll probably
do
it.
I try to do the first release candidate (mostly for testing) next week
(it
depends on the JIRAs fixed by then)
On Thu, Jun 16, 2016 at 10:56 PM, Henry Saputra <
henry.sapu...@gmail.com
wrote:
Thanks for the reply, @Max. I was not aware it was about dynamic
scaling,
which I think also asked for YARN support.
I agree to list all related half merge JIRA for the ResourceManager.
Looking forward for the Apache Mesos integration design for sure =)
- Henry
On Thu, Jun 16, 2016 at 2:12 AM, Maximilian Michels <m...@apache.org>
wrote:
Hi Robert, hi Henry,
+1 for a 1.1.0 release soon! We have enough new features that
justify
a major release.
@Henry We have plans to extend the ResourceManager to interact with
the Scheduler which will be a prerequisite for dynamic scaling. I
think this is out of scope for 1.1.0. The upcoming Mesos integration
won't require additional refactoring of the ResourceManager.
Instead,
we will create a new "Dispatcher" component that takes care of
bootstrapping the initial node with the JobManager/ResourceManager.
From there on, everything will be handled by the Mesos
ResourceManager. I recently discussed this with Eron (CC) who came
up
with this design and he plans to publish it to the mailing list
soon.
How about listing relevant JIRA issues here? "Half Merged" is kind
of
hard to get for people who are not involved in the different
components.
The Cassandra adapter seems like a pretty important thing to have
for
the next release. In addition, I would like to merge FLINK-3667 and
FLINK-3937. Robert is doing a review at the moment :) Those are a)
refactoring of the command-line and client classes b) adding
capability to resume cluster programmatically.
Then we should also have a look at any other critical/major bugs
listed
in
JIRA.
Cheers,
Max
On Wed, Jun 15, 2016 at 10:50 PM, Henry Saputra <
henry.sapu...@gmail.com
wrote:
Hi Robert,
Thanks for staying the discussion.
Do you know if there any open tasks for the Resource Manager left?
That is probably needed for Mesos integration?
- Henry
On Wed, Jun 15, 2016 at 12:55 PM, Robert Metzger <
rmetz...@apache.org>
wrote:
Hi,
Flink 1.0.0 was released early March, so three months have passed
and
I
think we should start discussing the scope of the next major
release
(1.1.0).
From a high level point of view, we've added the following new
features:
in master:
- Table API Refactoring, SQL, StreamSQL
- The metrics system
- Kinesis Connector
- Persistent file sources for streaming
Half merged:
- Resource manager refactoring
Unmerged features:
- Cassandra connector
- Key groups ("rescaling from savepoints")
- Queryable state
I'm pretty sure I forgot many other features / pull requests,
please
post
them to this thread. I'll collect them and create a Wiki page out
of
it.
Some immediate TODOs for us:
- Which of the unmerged features are we going to add to the
release?
- Which blockers do we need to address before releasing?
- Are there any volunteers for the release manager?
Regards,
Robert
