Hi Ajantha, Yes, there is still the coordinator piece to add before the sink is functional at all. There have been discussions with some in the community around the best path forward for that part to ensure we have a good foundation to build on, which is why we have held off on opening the PR. I'd very much like to target that for 1.6.0 also.
Bryan > On Apr 15, 2024, at 10:30 AM, Ajantha Bhat <ajanthab...@gmail.com> wrote: > > Hey everyone, > > We've been testing the Kafka Connect connector from the 1.5.0 release, and > unfortunately, it seems it's still not ready for use. We're not quite up to > date on its current status. > > We've noticed that three pull requests have been merged so far: #8701, #9466, > and #9641. There's also a discussion in the comments of PR #9641 about > holding off on supporting upsert > and CDC until we've thoroughly analyzed all the gaps: > https://github.com/apache/iceberg/pull/9641#issuecomment-1970183410 > > At least to make the connector usable for append operations only, it seems we > still need to work on the commit coordinator and integration tests. > What's the plan for addressing this? We're eager to pitch in and contribute. > Additionally, could we aim for a 1.6.0 release to at least have a usable > connector for append operations? > > We'd also like to discuss the gaps related to upsert (equality deletes) and > brainstorm potential solutions, as CDC is a crucial feature for the connector. > Could we also set up an issue tracker to keep track of what tasks are > completed and what's still pending? > > - Ajantha
