Hi everyone, I am working on revisiting the PR created by Liwei Li and Kyle Bendickson where they were implementing the Flink SinkV2 interface in the Iceberg Flink connector. The discussion has been halted around a year ago, but some of our users would love to have the automatic small file compaction for which the first step is implementing the SinkV2 API, so I would like to restart the conversation.
I have created the PR based on the previous work done by the team: https://github.com/apache/iceberg/pull/8653 I have created a document describing the changes: https://docs.google.com/document/d/1K1M4wb9r_Tr-SDsUvqLyBaI5F14eRcqe3-ud6io0Da0/edit#heading=h.ovwx4ivyetma Since there are missing features, I am proposing the following approach: 1. Review the design document 2. Update the PR 3. PR reviews 4. Merge the PR to the Iceberg source 5. Restart the discussion about the missing features in the Flink community by creating a FLIP 6. Discuss/review/merge the relevant Flink changes 7. Release the Flink changes 8. Create a PR in the Iceberg repo to start using the new Flink features 9. Merge the Iceberg PR 10. Be happy 😀 Thanks, Peter
