Re: A query on log truncation.

2023-03-06 Thread Vinoth
Hi Luke , Thanks for acknowledging my mail. Sorry for the late reply. My query was not on keeping uncommitted records but on how teams managed the loss of committed data in case of unclean leader election. Is there a means to track lost data?. Is this a common problem?. I am asking ba

Re: A query on log truncation.

2023-01-15 Thread Luke Chen
Hi Vinoth, I'm wondering what's the use case or pain point you're trying to resolve? Like you said, the client will be notified the data is not successfully sent or propagated and handle the error, why should we keep the un-commited records? Could you elaborate more on the motivation? Thank you.

A query on log truncation.

2023-01-15 Thread Vinoth
I was reading through about kafka , the way leader election works , log truncation etc. One thing that kind of struck me was how records which were written to log but then were not committed (It has not propagated successfully through to all of the isr and and the high watermark has not increased a