Hi all! I have collected a few next steps to push the streaming API and runtime a step further. Below is the list of issues, together with some comments gathered from offline discussions.
Feel free to complement and extend. Greetings, Stephan ------------------------------------------------------------------- Improve Kafka connector to commit consumer offsets to Zookeeper (not only hold them Flink internally) https://issues.apache.org/jira/browse/FLINK-1935 -> Robert is working on this, as far as I know, Finalize and document windowing semantics -> Kostas, Aljoscha, Gyula, Paris, and me are syncing on that today and will post a document with a description of semantics and concepts Add a unified way to specify chaining and slot sharing -> Open pull request from Gyula at https://github.com/apache/flink/pull/634 Rework timestamps such that they are assigned at the data sources. These timestamps should be used to trigger time windows. This is crucial in order to get the windows independent of the system time -> No JIRA for this, yet Make time windows based on source timestamps. Followup of the previous point. -> No JIRA, yet Integrate an out-of-core state solution that also works as a distributed state backup solution -> Fabian and me have started to look into Apache Ignite as a backend for streaming state or at least state backup. We'll post updates as we make progress. Extend the Table API to support windows -> This hinges on the finalization and documentation of the window semantics. -> No JIRA, yet Add a way to cancel/suspend the streaming topology (checkpoint and end) -> controlled failure/quit -> No JIRA, yet
