Hi community,

Because we have approximately one month of development time left until the
targeted Flink 1.10 feature freeze, we thought now would be a good time to
give another progress update. Below we have included a list of the ongoing
efforts that have made progress since our last release progress update [1].
As
always, if you are working on something that is not included here, feel free
to use this thread to share your progress.

- Support Java 11 [2]
    - Implementation is in progress (18/21 subtasks resolved)

- Table API improvements
    - Full Data Type Support in Planner [3]
        - Implementing (1/8 subtasks resolved)
    - FLIP-66 Support Time Attribute in SQL DDL [4]
        - Implementation is in progress (1/7 subtasks resolved).
    - FLIP-70 Support Computed Column [5]
        - FLIP voting [6]
    - FLIP-63 Rework Table Partition Support [7]
        - Implementation is in progress (3/15 subtasks resolved).
    - FLIP-51 Rework of Expression Design [8]
        - Implementation is in progress (2/12 subtasks resolved).
    - FLIP-64 Support for Temporary Objects in Table Module [9]
        - Implementation is in progress

- Hive compatibility completion (DDL/UDF) to support full Hive integration
    - FLIP-57 Rework FunctionCatalog [10]
        - Implementation is in progress (6/9 subtasks resolved)
    - FLIP-68 Extend Core Table System with Modular Plugins [11]
        - Implementation is in progress (2/8 subtasks resolved)

- Finer grained resource management
    - FLIP-49: Unified Memory Configuration for TaskExecutors [12]
        - Implementation is in progress (6/10 subtasks resolved)
    - FLIP-53: Fine Grained Operator Resource Management [13]
        - Implementation is in progress (1/9 subtasks resolved)

- Finish scheduler re-architecture [14]
    - Integration tests are being enabled for new scheduler

- Executor/Client refactoring [15]
    - FLIP-81: Executor-related new ConfigOptions [16]
        - done
    - FLIP-73: Introducing Executors for job submission [17]
        - Implementation is in progress

- FLIP-36 Support Interactive Programming [18]
    - Is built on top of FLIP-67 [19], which has been accepted
    - Implementation in progress

- FLIP-58: Flink Python User-Defined Stateless Function for Table [20]
    - Implementation is in progress (12/22 subtask resolved)
- FLIP-50: Spill-able Heap Keyed State Backend [21]
    - Implementation is in progress (2/11 subtasks resolved)

- RocksDB Backend Memory Control [22]
    - FLIP for resource management on state backend will be opened soon
    - Write Buffer Manager will be backported to FRocksDB due to
performance regression [23] in new RocksDB versions

- Unaligned Checkpoints
    - FLIP-76 [24] was published and received positive feedback
    - Implementation is in progress

- Separate framework and user class loader in per-job mode [25]
    - First PR is almost done. Remaining PRs will be ready next week

- Active Kubernetes Integration [26]
    - Implementation is in progress (6/11 in review, 3/11 in progress, 2/11
todo)

- FLIP-39 Flink ML pipeline and ML libs [27]
    - A few abstract ML classes have been merged (FLINK-13339, FLINK-13513)
    - Starting review of algorithms

Again, the feature freeze is targeted to be at the end of November. Please
make sure that all important work threads can be completed until that date.
Feel free to use this thread to communicate any concerns about features that
might not be finished until then. We will send another announcement later in
the release cycle to make the date of the feature freeze official.

Best,
Yu & Gary

[1] https://s.apache.org/wc0dc
[2] https://issues.apache.org/jira/browse/FLINK-10725
[3] https://issues.apache.org/jira/browse/FLINK-14079
[4]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-66%3A+Support+time+attribute+in+SQL+DDL
[5]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-70%3A+Flink+SQL+Computed+Column+Design
[6]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/VOTE-FLIP-70-Flink-SQL-Computed-Column-Design-td34385.html
[7]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-63%3A+Rework+table+partition+support
[8]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-51%3A+Rework+of+the+Expression+Design
[9]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-64%3A+Support+for+Temporary+Objects+in+Table+module
[10]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-57%3A+Rework+FunctionCatalog
[11]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-68%3A+Extend+Core+Table+System+with+Pluggable+Modules
[12]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-49%3A+Unified+Memory+Configuration+for+TaskExecutors
[13]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-53%3A+Fine+Grained+Operator+Resource+Management
[14] https://issues.apache.org/jira/browse/FLINK-10429
[15]
https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
[16]
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=133631524
[17]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-73%3A+Introducing+Executors+for+job+submission
[18]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-36%3A+Support+Interactive+Programming+in+Flink
[19]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-67%3A+Cluster+partitions+lifecycle
[20]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-58%3A+Flink+Python+User-Defined+Stateless+Function+for+Table
[21]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-50%3A+Spill-able+Heap+Keyed+State+Backend
[22] https://issues.apache.org/jira/browse/FLINK-7289
[23] https://github.com/facebook/rocksdb/issues/5774
[24]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-76%3A+Unaligned+Checkpoints
[25] https://issues.apache.org/jira/browse/FLINK-13993
[26] https://issues.apache.org/jira/browse/FLINK-9953
[27]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-39+Flink+ML+pipeline+and+ML+libs

Reply via email to