Re: [DISCUSS] KIP-762 Delete Committed Connect Records

2021-08-20 Thread Ryanne Dolan
Mickael, thanks for taking a look. I wouldn't be opposed to making this a per-connector configuration. I guess each topic-partition is consumed by one WorkerSinkTask, so it should be safe for the worker to truncate such partitions if all associated Connectors opt-in. I think in that case it may ma

Re: [DISCUSS] KIP-762 Delete Committed Connect Records

2021-08-19 Thread Mickael Maison
Hi Ryanne, Thanks for the KIP, this is an interesting use case. I agree it's best to have the runtime take care of the deletion. However, I wonder if it would make more sense to have the configuration on connectors instead of the runtime. While Workers perform the deletion, it's very much driven

Re: [DISCUSS] KIP-762 Delete Committed Connect Records

2021-07-17 Thread Ryanne Dolan
Whoops, looks like I got the KIP number wrong in the original email subject line. Please use this corrected thread. Ryanne On Fri, Jul 16, 2021, 3:45 PM Ryanne Dolan wrote: > Hey y'all, please review the following small proposal: > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-762%3