Re: [DISCUSS] KIP-89: Allow sink connectors to decouple flush and offset commit

2016-11-07 Thread Ewen Cheslack-Postava
I don't have much to say here since I reviewed a draft of this already. But I did want to point out that although it expands the API for Connectors (which I think we should be conservative about), it has the really nice impact that it allows connectors that really care about high throughput to avoi

[DISCUSS] KIP-89: Allow sink connectors to decouple flush and offset commit

2016-11-04 Thread Shikhar Bhushan
Hi all, I created KIP-89 for making a Connect API change that allows for sink connectors to decouple flush and offset commits. https://cwiki.apache.org/confluence/display/KAFKA/KIP-89%3A+Allow+sink+connectors+to+decouple+flush+and+offset+commit I'd welcome your input. Best, Shikhar