You have a couple options: 1) You can adjust log.segment.bytes to make the segments smaller so that individual segments can be trimmed 2) You can set log.roll.hours, which will roll to a new log segment even if the size hasn't been reached Note: #1 and #2 also have per-topic setting controls
https://kafka.apache.org/08/documentation/#brokerconfigs Thanks, -- KYLE TINKER Lead Architect/Team Lead | WorkForce Software T: +1 734 742 2616 | E: ktin...@workforcesoftware.com -----Original Message----- From: Muruganandham, Ashokkumar [mailto:ashokkumar.muruganand...@fluke.com] Sent: Monday, March 12, 2018 1:19 AM To: dev@kafka.apache.org Subject: Log Retention Period of Kafka Messages Hi Team, Form the documentation I could see that log retention period is relative to the partition segments. So unless the segment is closed (1Gb/1 Week) the message will never be deleted. Consider I have a very critical use case where I will need to delete the data every day and my segment reaches 1GB after three days. So according the above statement, then there is no way I could configure Kafka to delete the data in one day. Is my understanding correct ? Regards, AK ________________________________ Please be advised that this email may contain confidential information. If you are not the intended recipient, please notify us by email by replying to the sender and delete this message. The sender disclaims that the content of this email constitutes an offer to enter into, or the acceptance of, any agreement; provided that the foregoing does not invalidate the binding effect of any digital or other electronic reproduction of a manual signature that is included in any attachment. This message is intended exclusively for the individual or entity to which it is addressed. This communication may contain information that is proprietary, privileged, confidential or otherwise legally exempt from disclosure. If you are not the named addressee, or have been inadvertently and erroneously referenced in the address line, you are not authorized to read, print, retain, copy or disseminate this message or any part of it. If you have received this message in error, please notify the sender immediately by e-mail and delete all copies of the message. (ID m031214)
