LiuHao0606 opened a new issue, #13211: URL: https://github.com/apache/seatunnel/issues/13211
### Search before asking Yes, I had searched in the [feature](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement. The closest ones are #9207 (upgrade to Paimon 1.x, closed) and #9530 (update to the latest version, closed), both from previous upgrade rounds. ### Description Apache Paimon 2.0.0 has been officially released: https://paimon.apache.org/releases/2.0.0 Paimon 2.0 is a major release that brings, among other things: - New data types for multimodal workloads: `BLOB`, `VECTOR`, `VARIANT` - Data Evolution / Row Tracking (add or backfill columns without rewriting existing files) - Global index framework (BTree / Bitmap / vector / full-text) - REST catalog, PyPaimon, and other ecosystem improvements However, the SeaTunnel Paimon connector still pins the Paimon runtime to **1.1.1** (see `seatunnel-connectors-v2/connector-paimon/pom.xml`: `<paimon.version>1.1.1</paimon.version>`). This means: 1. `connector-paimon` source/sink are not qualified against Paimon 2.0 warehouses. 2. Tables created with Paimon 2.0 that use new features/data types cannot be read or written by the connector's bundled 1.1.1 runtime. 3. Users building a lakehouse on Paimon 2.0 cannot use SeaTunnel as their data integration engine against those tables. **Request:** upgrade the Paimon dependency of `connector-paimon` (source and sink) from 1.1.1 to 2.0.x, so SeaTunnel can read from and write to Paimon 2.0 tables. If feasible, please keep backward compatibility with existing 1.x tables. ### Usage Scenario We run a streaming lakehouse built on EMR (HDFS HA) + StarRocks + Apache Fluss + Paimon + SeaTunnel (SeaTunnel Zeta as the batch/streaming integration engine). New Paimon tables are being created on Paimon 2.0 to leverage its improvements, and we would like to keep using SeaTunnel jobs (e.g. database CDC / batch sync -> Paimon sink) against those 2.0 tables. With the connector stuck on the 1.1.1 runtime, these pipelines cannot target 2.0 tables, forcing us to either downgrade our Paimon warehouse or look for another ingestion tool. ### Related issues - https://github.com/apache/seatunnel/issues/9207 (previous upgrade to Paimon 1.x, closed) - https://github.com/apache/seatunnel/issues/9530 (update to the latest version, closed) - https://github.com/apache/seatunnel/issues/13140 (Paimon REST catalog support, open - REST catalog is also a 2.x-era feature, a version upgrade would help that effort too) ### Are you willing to submit a PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) --- ### Chinese summary (补充说明) Apache Paimon 2.0.0 已正式发布(多模态类型 BLOB/VECTOR/VARIANT、Data Evolution、全局索引、REST Catalog 等),但 SeaTunnel 的 `connector-paimon` 目前仍固定依赖 Paimon 1.1.1,无法读写 2.0 表。希望社区将 Paimon connector 升级到 2.0.x,以支持基于 Paimon 2.0 的湖仓读写场景(我们的流式湖仓 EMR + StarRocks + Fluss + Paimon + SeaTunnel 正受此限制)。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
