Hi Becket, Thanks for the reply, it is good to know that there is activity on FLIP-27.
A while ago I was wondering if event time alignment is on the radar [1], can you please clarify that? There is a parallel discussion of adding it to the existing Kafka consumer [2], could you please take a look and help figuring out the path forward? Thanks, Thomas [1] https://lists.apache.org/thread.html/c7336f259bbbb4c482de0580ecf2472a6b3e5a0fe2b7dd9258b159f2@%3Cdev.flink.apache.org%3E [2] https://issues.apache.org/jira/browse/FLINK-12675 On Mon, Nov 4, 2019 at 5:48 AM Becket Qin <becket....@gmail.com> wrote: > Hi Steven and Thomas, > > Sorry about missing the update of FLIP-27. > > I am working on the implementation of FLIP-27 at this point. It is about > 70% done. Right now I am integrating the source coordinator to the job > master. Hopefully I can get the basics of Kafka connector work from end to > end by this Sunday. Then I'll update FLIP-27 wiki page. > > Thanks, > > Jiangjie (Becket) Qin > > > On Sat, Nov 2, 2019 at 8:18 AM Thomas Weise <t...@apache.org> wrote: > >> Is there any activity on FLIP-27 that would make it relevant for 1.10 >> release? >> >> Thanks Gary for the update, it provides excellent visibility on current >> activity and what we can expect with the release. >> >> >> On Fri, Nov 1, 2019 at 1:52 PM Steven Wu <stevenz...@gmail.com> wrote: >> >>> Gary, FLIP-27 seems to get omitted in the 2nd update. below is the info >>> from update #1. >>> >>> - FLIP-27: Refactor Source Interface [20] >>> - FLIP accepted. Implementation is in progress. >>> >>> >>> >>> On Fri, Nov 1, 2019 at 7:01 AM Gary Yao <g...@ververica.com> wrote: >>> >>> > Hi community, >>> > >>> > Because we have approximately one month of development time left until >>> the >>> > targeted Flink 1.10 feature freeze, we thought now would be a good >>> time to >>> > give another progress update. Below we have included a list of the >>> ongoing >>> > efforts that have made progress since our last release progress update >>> > [1]. As >>> > always, if you are working on something that is not included here, feel >>> > free >>> > to use this thread to share your progress. >>> > >>> > - Support Java 11 [2] >>> > - Implementation is in progress (18/21 subtasks resolved) >>> > >>> > - Table API improvements >>> > - Full Data Type Support in Planner [3] >>> > - Implementing (1/8 subtasks resolved) >>> > - FLIP-66 Support Time Attribute in SQL DDL [4] >>> > - Implementation is in progress (1/7 subtasks resolved). >>> > - FLIP-70 Support Computed Column [5] >>> > - FLIP voting [6] >>> > - FLIP-63 Rework Table Partition Support [7] >>> > - Implementation is in progress (3/15 subtasks resolved). >>> > - FLIP-51 Rework of Expression Design [8] >>> > - Implementation is in progress (2/12 subtasks resolved). >>> > - FLIP-64 Support for Temporary Objects in Table Module [9] >>> > - Implementation is in progress >>> > >>> > - Hive compatibility completion (DDL/UDF) to support full Hive >>> integration >>> > - FLIP-57 Rework FunctionCatalog [10] >>> > - Implementation is in progress (6/9 subtasks resolved) >>> > - FLIP-68 Extend Core Table System with Modular Plugins [11] >>> > - Implementation is in progress (2/8 subtasks resolved) >>> > >>> > - Finer grained resource management >>> > - FLIP-49: Unified Memory Configuration for TaskExecutors [12] >>> > - Implementation is in progress (6/10 subtasks resolved) >>> > - FLIP-53: Fine Grained Operator Resource Management [13] >>> > - Implementation is in progress (1/9 subtasks resolved) >>> > >>> > - Finish scheduler re-architecture [14] >>> > - Integration tests are being enabled for new scheduler >>> > >>> > - Executor/Client refactoring [15] >>> > - FLIP-81: Executor-related new ConfigOptions [16] >>> > - done >>> > - FLIP-73: Introducing Executors for job submission [17] >>> > - Implementation is in progress >>> > >>> > - FLIP-36 Support Interactive Programming [18] >>> > - Is built on top of FLIP-67 [19], which has been accepted >>> > - Implementation in progress >>> > >>> > - FLIP-58: Flink Python User-Defined Stateless Function for Table [20] >>> > - Implementation is in progress (12/22 subtask resolved) >>> > - FLIP-50: Spill-able Heap Keyed State Backend [21] >>> > - Implementation is in progress (2/11 subtasks resolved) >>> > >>> > - RocksDB Backend Memory Control [22] >>> > - FLIP for resource management on state backend will be opened soon >>> > - Write Buffer Manager will be backported to FRocksDB due to >>> > performance regression [23] in new RocksDB versions >>> > >>> > - Unaligned Checkpoints >>> > - FLIP-76 [24] was published and received positive feedback >>> > - Implementation is in progress >>> > >>> > - Separate framework and user class loader in per-job mode [25] >>> > - First PR is almost done. Remaining PRs will be ready next week >>> > >>> > - Active Kubernetes Integration [26] >>> > - Implementation is in progress (6/11 in review, 3/11 in progress, >>> > 2/11 todo) >>> > >>> > - FLIP-39 Flink ML pipeline and ML libs [27] >>> > - A few abstract ML classes have been merged (FLINK-13339, >>> > FLINK-13513) >>> > - Starting review of algorithms >>> > >>> > Again, the feature freeze is targeted to be at the end of November. >>> Please >>> > make sure that all important work threads can be completed until that >>> date. >>> > Feel free to use this thread to communicate any concerns about features >>> > that >>> > might not be finished until then. We will send another announcement >>> later >>> > in >>> > the release cycle to make the date of the feature freeze official. >>> > >>> > Best, >>> > Yu & Gary >>> > >>> > [1] https://s.apache.org/wc0dc >>> > [2] https://issues.apache.org/jira/browse/FLINK-10725 >>> > [3] https://issues.apache.org/jira/browse/FLINK-14079 >>> > [4] >>> > >>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-66%3A+Support+time+attribute+in+SQL+DDL >>> > [5] >>> > >>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-70%3A+Flink+SQL+Computed+Column+Design >>> > [6] >>> > >>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-70-Flink-SQL-Computed-Column-Design-td34385.html >>> > [7] >>> > >>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-63%3A+Rework+table+partition+support >>> > [8] >>> > >>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-51%3A+Rework+of+the+Expression+Design >>> > [9] >>> > >>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-64%3A+Support+for+Temporary+Objects+in+Table+module >>> > [10] >>> > >>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-57%3A+Rework+FunctionCatalog >>> > [11] >>> > >>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-68%3A+Extend+Core+Table+System+with+Pluggable+Modules >>> > [12] >>> > >>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-49%3A+Unified+Memory+Configuration+for+TaskExecutors >>> > [13] >>> > >>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-53%3A+Fine+Grained+Operator+Resource+Management >>> > [14] https://issues.apache.org/jira/browse/FLINK-10429 >>> > [15] >>> > >>> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E >>> > [16] >>> > >>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=133631524 >>> > [17] >>> > >>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-73%3A+Introducing+Executors+for+job+submission >>> > [18] >>> > >>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-36%3A+Support+Interactive+Programming+in+Flink >>> > [19] >>> > >>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-67%3A+Cluster+partitions+lifecycle >>> > [20] >>> > >>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-58%3A+Flink+Python+User-Defined+Stateless+Function+for+Table >>> > [21] >>> > >>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-50%3A+Spill-able+Heap+Keyed+State+Backend >>> > [22] https://issues.apache.org/jira/browse/FLINK-7289 >>> > [23] https://github.com/facebook/rocksdb/issues/5774 >>> > [24] >>> > >>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-76%3A+Unaligned+Checkpoints >>> > [25] https://issues.apache.org/jira/browse/FLINK-13993 >>> > [26] https://issues.apache.org/jira/browse/FLINK-9953 >>> > [27] >>> > >>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-39+Flink+ML+pipeline+and+ML+libs >>> > >>> >>
