Will we remove the legacy mode for 1.6?
I can see value in keeping it for now so that legacy issues are still
visible on master, but at the same time removing this code would reduce
a lot of complexity and ambiguity in the codebase...
On 04.06.2018 11:21, Stephan Ewen wrote:
Hi Flink Community!
The release of Apache Flink 1.5 has happened (yay!) - so it is a good
time to start talking about what to do for release 1.6.
*== Suggested release timeline ==*
I would propose to release around _end of July_ (that is 8-9 weeks
from now).
The rational behind that: There was a lot of effort in release testing
automation (end-to-end tests, scripted stress tests) as part of
release 1.5. You may have noticed the big set of new modules under
"flink-end-to-end-tests" in the Flink repository. It delayed the 1.5
release a bit, and needs to continue as part of the coming release
cycle, but should help make releasing more lightweight from now on.
(Side note: There are also some nightly stress tests that we created
and run at data Artisans, and where we are looking whether and in
which way it would make sense to contribute them to Flink.)
*== Features and focus areas ==*
We had a lot of big and heavy features in Flink 1.5, with FLIP-6, the
new network stack, recovery, SQL joins and client, ... Following
something like a "tick-tock-model", I would suggest to focus the next
release more on integrations, tooling, and reducing user friction.
Of course, this does not mean that no other pull request gets
reviewed, an no other topic will be examined - it is simply meant as a
help to understand where to expect more activity during the next
release cycle. Note that these are really the coarse focus areas -
don't read this as a comprehensive list.
This list is my first suggestion, based on discussions with
committers, users, and mailing list questions.
- Support Java 9 and Scala 2.12
- Smoothen the integration in Container environment, like "Flink as
a Library", and easier integration with Kubernetes services and other
proxies.
- Polish the remaing parts of the FLIP-6 rewrite
- Improve state backends with asynchronous timer snapshots,
efficient timer deletes, state TTL, and broadcast state support in
RocksDB.
- Extends Streaming Sinks:
- Bucketing Sink should support S3 properly (compensate for
eventual consistency), work with Flink's shaded S3 file systems, and
efficiently support formats that compress/index arcoss individual rows
(Parquet, ORC, ...)
- Support ElasticSearch's new REST API
- Smoothen State Evolution to support type conversion on snapshot
restore
- Enhance Stream SQL and CEP
- Add support for "update by key" Table Sources
- Add more table sources and sinks (Kafka, Kinesis, Files, K/V
stores)
- Expand SQL client
- Integrate CEP and SQL, through MATCH_RECOGNIZE clause
- Improve CEP Performance of SharedBuffer on RocksDB
