Re: [DISCUSS] KIP-838 Simulate batching and compression

2022-05-17 Thread Sergio Troiano
I’ll start with the code if you consider this feature could be useful, what do you think ? Regarding to the code I’ll use the same classes dump-logs uses for reading records and add the logic on top of it. Best regards Sergio Troiano Sent from my iPhone > On 16 May 2022, at 20:48, Sergio Da

Re: [DISCUSS] KIP-838 Simulate batching and compression

2022-05-16 Thread Sergio Daniel Troiano
Hi Divij, First of all thanks for your time and dedication. About point one: You are right, the idea is to have "in real time" visibility of the way the clients are using the service as that is translated into a lot of money saving. I agree with the further vision although I think we are still fa

Re: [DISCUSS] KIP-838 Simulate batching and compression

2022-05-16 Thread Divij Vaidya
Thank you for the KIP Sergio. High level thoughts: 1\ I understand that the idea here is to provide better visibility to the admins about potential improvements using compression and modifying batch size. I would take it a step further and say that we should be providing this visibility in a progr

[DISCUSS] KIP-838 Simulate batching and compression

2022-05-15 Thread Sergio Daniel Troiano
Hey guys! I would like to start an early discussion on this: https://cwiki.apache.org/confluence/display/KAFKA/KIP-838+Simulate+batching+and+compression Thanks!