Hi, Community By the way, there is a very important feature I think it should be. Currently, the BucketingSink does not support when a bucket is ready for user use. This situation will be very obvious when flink work with offline end. We called that real time/offline integration in business. In this case, we should let the user can do some extra work when the bucket is ready. And here is the JIRA for this https://issues.apache.org/jira/browse/FLINK-9609 <https://issues.apache.org/jira/browse/FLINK-9609>
Cheers Minglei > 在 2018年6月4日，下午5:21，Stephan Ewen <se...@apache.org> 写道： > > Hi Flink Community! > > The release of Apache Flink 1.5 has happened (yay!) - so it is a good time to > start talking about what to do for release 1.6. > > == Suggested release timeline == > > I would propose to release around end of July (that is 8-9 weeks from now). > > The rational behind that: There was a lot of effort in release testing > automation (end-to-end tests, scripted stress tests) as part of release 1.5. > You may have noticed the big set of new modules under > "flink-end-to-end-tests" in the Flink repository. It delayed the 1.5 release > a bit, and needs to continue as part of the coming release cycle, but should > help make releasing more lightweight from now on. > > (Side note: There are also some nightly stress tests that we created and run > at data Artisans, and where we are looking whether and in which way it would > make sense to contribute them to Flink.) > > == Features and focus areas == > > We had a lot of big and heavy features in Flink 1.5, with FLIP-6, the new > network stack, recovery, SQL joins and client, ... Following something like a > "tick-tock-model", I would suggest to focus the next release more on > integrations, tooling, and reducing user friction. > > Of course, this does not mean that no other pull request gets reviewed, an no > other topic will be examined - it is simply meant as a help to understand > where to expect more activity during the next release cycle. Note that these > are really the coarse focus areas - don't read this as a comprehensive list. > > This list is my first suggestion, based on discussions with committers, > users, and mailing list questions. > > - Support Java 9 and Scala 2.12 > > - Smoothen the integration in Container environment, like "Flink as a > Library", and easier integration with Kubernetes services and other proxies. > > - Polish the remaing parts of the FLIP-6 rewrite > > - Improve state backends with asynchronous timer snapshots, efficient timer > deletes, state TTL, and broadcast state support in RocksDB. > > - Extends Streaming Sinks: > - Bucketing Sink should support S3 properly (compensate for eventual > consistency), work with Flink's shaded S3 file systems, and efficiently > support formats that compress/index arcoss individual rows (Parquet, ORC, ...) > - Support ElasticSearch's new REST API > > - Smoothen State Evolution to support type conversion on snapshot restore > > - Enhance Stream SQL and CEP > - Add support for "update by key" Table Sources > - Add more table sources and sinks (Kafka, Kinesis, Files, K/V stores) > - Expand SQL client > - Integrate CEP and SQL, through MATCH_RECOGNIZE clause > - Improve CEP Performance of SharedBuffer on RocksDB >
