reswqa commented on code in PR #751: URL: https://github.com/apache/flink-web/pull/751#discussion_r1679349683
########## docs/content/posts/2024-07-15-release-1.20.0.md: ########## @@ -0,0 +1,510 @@ +--- +authors: +- reswqa: + name: "Weijie Guo" + twitter: "WeijieGuo12" +- 1996fanrui: + name: "Rui Fan" + twitter: "1996fanrui" + +date: "2024-07-15T08:00:00Z" +subtitle: "" +title: Announcing the Release of Apache Flink 1.20 +aliases: +- /news/2024/07/15/release-1.20.0.html +--- + +The Apache Flink PMC is pleased to announce the release of Apache Flink 1.20.0. As usual, we are +looking at a packed release with a wide variety of improvements and new features. Overall, 142 +people contributed to this release completing 13 FLIPs and 300+ issues. Thank you! + +Let's dive into the highlights. + +# Standing on the Eve of Apache Flink 2.0 + +The Flink community expects a 1.19 → 1.20 → 2.0 release sequence, with a normal 4-5 months release cycle. +We expect to deliver the 2.0 release by the end of 2024, as long as not compromising the quality. + +Start from Flink 1.19, the community has decided to officially deprecate multiple APIs that were approaching +end of life for a while. In 1.20, we further sorted through all relevant APIs that might need to be replaced +or deprecated to clear the way for the 2.0 release: +- Configuration Improvements: As Flink moves toward 2.0, we have revisited all runtime & table/sql +configurations and identified several improvements to enhance user-friendliness and maintainability +- Deprecate the Legacy `SinkFunction` API: Since its introduction in Flink 1.12, the Unified Sink API +has undergone extensive development and testing. Over multiple release cycles, the API has demonstrated +stability and robustness, aligning with the criteria set forth in FLIP-197 for API stability graduation. +So we promote the Unified Sink API v2 to `@Public` and deprecate the legacy `SinkFunction`. + +It has been seven years since the Flink community's last major release, and we have great expectations for Flink 2.0. +We'll have a lot of killer features released in `2.x`, and some of them were released as MVP(minimum viable product) +in 1.20: +- Introduce a New Materialized Table for Simplifying Data Pipelines: FLIP-435 designed to simplify the development of +data processing pipelines. With dynamic table with uniform SQL statements and freshness, users can define batch +and streaming transformations to data in the same way, accelerate ETL pipeline development, and manage task scheduling +automatically. +- Unified File Merging Mechanism for Checkpoints: The unified file merging mechanism for checkpointing is introduced to +Flink 1.20 as an MVP feature, which allows scattered small checkpoint files to be written into larger files, reducing +the number of file creations and file deletions and alleviating the pressure of file system metadata management raised by +the file flooding problem during checkpoints. + +# Flink SQL Improvements + +## Introduce a New Materialized Table for Simplifying Data Pipelines + +We have introduced the Materialized Table in Flink SQL, a new table type designed to simplify both batch and stream +data pipelines while providing a consistent development experience. + +By specifying data freshness and query at creation, the engine automatically derives the schema and creates a data +refresh pipeline to maintain the specified freshness. + +Here is an example to create a materialized table that is constantly refreshed with a data freshness of `30` seconds. + +```sql +CREATE MATERIALIZED TABLE continuous_users_shops +PARTITIONED BY (ds) +WITH ( + 'format' = 'debezium-json', + 'sink.rolling-policy.rollover-interval' = '10s', + 'sink.rolling-policy.check-interval' = '10s' +) +FRESHNESS = INTERVAL '30' SECOND +AS SELECT + user_id, + ds, + SUM (payment_amount_cents) AS payed_buy_fee_sum, + SUM (1) AS PV +FROM ( + SELECT user_id, order_created_at AS ds, payment_amount_cents + FROM json_source + ) AS tmp +GROUP BY user_id, ds; +``` + +**More Information** +* [FLINK-35187](https://issues.apache.org/jira/browse/FLINK-35187) +* [FLIP-435](https://cwiki.apache.org/confluence/display/FLINK/FLIP-435%3A+Introduce+a+New+Materialized+Table+for+Simplifying+Data+Pipelines) +* [Materialized Table Overview](https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/dev/table/materialized-table/overview/) + + +## Introduce Catalog-Related Syntax + +As the application scenario of `Catalog` expands, which widely applied in services such as JDBC/Hive/Paimon, +`Catalog` plays an increasingly crucial role in Flink. + +Now in Flink 1.20, you can use the `DQL` syntax to obtain detailed metadata from existing catalogs, and the +`DDL` syntax to modify metadata such as properties or comment in the specified catalog. Review Comment: Make sense, I have added a example. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
