Hi Rajat I am not 100% sure, but I think the roll logic is based on incoming messages. When a message is received, it will compare the timestamp of the first message in the log vs the timestamp of the incoming message. If it is greater than log.roll.ms, it will roll the segment
In your scenario, it could be that your data is not well partioned, so this given partition doesn't receive a lot of message, resulting in the behavior you have observed Vincent On Tue, May 21, 2019 at 1:43 PM Rajat Gangwar <rajatgangwar...@gmail.com> wrote: > Hi, > > We have 1 topic with 8 partitions. > *log.roll* is set to the default value of 7 days. > *log.segment.bytes* is also set to default : 1 GB. > *log.retention.hours* is set to 30 days. > > Inflow is not much so every 7 days, logs are getting segmented into new > file even though it never reaches 1 GB. > But for 1 partition it didn't spawn new segments even after 30 days. And > since retention policy is for 30 days, it deleted that only segment which > had all the messages. > > We end up losing messages in the production. > > *Is this the intended behavior, that until the log file reaches segment max > size, it may or may not get split into new segments, no matter what value > is set for log.roll?* > > Thanks, > Rajat >
