I was investigating a big topic, trying to find the reason why the retention limits were not working. I was able to read messages from 2019 even though the retention was for 5 days. Until I found the log segment file that contains those 2019 messages, and checking the rest of the messages I realized its have messages from Dec 2020. Yep, messages from the future.
| offset: 4513685442 CreateTime: 1609420521000 keysize: -1 valuesize: 197 sequence: -1 headerKeys: [] payload: 2020-12-31T13:15:21+00:00 ... Besides fixing how we use the CreateTime for ingesting data, I need to remove that data. So my two questions are: a) How is the strategy kafka uses to decide if a log segment needs to be deleted. It's using Max CreateTime and compare with retention limits? b) How can I delete that whole log segment file, and of course, without messing the system? Thanks.- -- Richard Rossel Atlanta - GA
